Metadata-Version: 2.4
Name: latenspect
Version: 0.14.0
Summary: A local privacy preflight for hidden file content and metadata
Keywords: privacy,metadata,redaction,ooxml,pdf,security
Author: Latenspect contributors
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
License-File: THIRD_PARTY_NOTICES.md
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Requires-Dist: jinja2>=3.1.6,<4
Requires-Dist: lxml>=6.1.1,<7
Requires-Dist: pillow>=12.3.0,<13
Requires-Dist: pydantic>=2.13.4,<3
Requires-Dist: pypdf>=6.14.2,<7
Requires-Dist: pypdfium2>=5.12.1,<6
Requires-Dist: rich>=15.0.0,<16
Requires-Dist: typer>=0.27.1,<0.28
Requires-Python: >=3.11, <3.15
Project-URL: Homepage, https://github.com/xvorki/latenspect
Project-URL: Documentation, https://github.com/xvorki/latenspect#readme
Project-URL: Issues, https://github.com/xvorki/latenspect/issues
Project-URL: Security, https://github.com/xvorki/latenspect/security/policy
Description-Content-Type: text/markdown

# Latenspect

**See what your files don’t show.**

Latenspect is a local-first privacy preflight for hidden document content:
tracked changes, deleted text, speaker notes, hidden slides and sheets,
ineffective PDF redactions, embedded resources, external references,
and sensitive metadata.

```bash
uvx latenspect inspect suspicious-file.pdf
```

Latenspect does not prove that a file is universally safe. It reports findings under the enabled rules and states when analysis is incomplete.

![Latenspect inspection and cleaning architecture](demo/architecture.svg)

## Hero example: a rectangle is not a redaction

A PDF can look redacted while its original text remains selectable, searchable, or extractable. Latenspect’s flagship rule enumerates page objects, identifies opaque filled rectangle-like paths, compares their geometry with extractable text, and considers drawing order. A later, highly opaque cover that substantially overlaps recoverable text produces an evidence-backed finding with page, bounds, object order, confidence, and a masked excerpt.

```console
$ latenspect inspect demo/generated/demo_fake_redaction.pdf --no-color
LATENSPECT PREFLIGHT
demo_fake_redaction.pdf

Scan status: complete
Risk: HIGH
Findings: 1
Rules executed: 5
File modified: no

HIGH      LS-PDF-001  Opaque vector object covers extractable text
          Location: page 1, cover object 1
          Evidence: R•••••••••• text: “Project N•••••••• - Internal budget: E•• ••••••”
          Confidence: high
          Why it matters: The text can remain selectable, searchable, copyable,
          or recoverable from the PDF content stream.

The original file was not modified.
Use --reveal-evidence to show full hidden excerpts.
Verified against Latenspect 0.1.0 and ruleset 2026.1.
```

The applied-redaction control in the demo corpus has the underlying text removed and does not trigger `LS-PDF-001`. Backgrounds, transparent rectangles, and stroke-only rectangles are covered by separate negative controls.

## Quick start

Latenspect supports CPython 3.11 through 3.14.

```bash
# Run without a permanent installation.
uvx latenspect --version
uvx latenspect inspect contract.docx

# Scan a directory and make high-or-worse findings fail CI.
uvx latenspect inspect release-files/ --recursive --fail-on high

# Write machine-readable or self-contained reports.
uvx latenspect inspect deck.pptx --format json --output deck.latenspect.json
uvx latenspect inspect evidence.pdf --format html --output evidence.latenspect.html
uvx latenspect inspect release-files/ --recursive --format sarif --output latenspect.sarif
```

The default terminal report uses masked evidence. JSON, HTML, and SARIF are deterministic apart from documented report timestamps and timings.

## Supported formats

| Format | Inspection | Explicit cleaning in 0.1.0 |
|---|---|---|
| PDF | Opaque covers over extractable text, unapplied redaction annotations, attachments, active actions, document/XMP metadata | None; PDF cleaning is deliberately refused |
| DOCX | Deleted, moved, and inserted revisions; comments; Track Changes; hidden text; templates; external and embedded content; properties | Properties, comments, attached-template relationship, selected external relationships |
| PPTX | Speaker notes, hidden slides, comments, hidden objects, external and embedded content, macros, properties | Properties, notes, comments, selected external relationships |
| XLSX | Hidden and very-hidden sheets, hidden data-bearing rows/columns, comments, external links, embedded content, macros, properties | Properties, comments, selected external relationships |
| JPEG | EXIF/GPS, camera and identity data, thumbnails, XMP/IPTC/comments | Lossless marker-level metadata stripping; ICC preserved by default |
| PNG | Text chunks, EXIF, XMP, identity and descriptive metadata | Lossless chunk-level metadata stripping; color information preserved by default |

Unsupported, encrypted, malformed, truncated, or resource-limited inputs are reported honestly as unsupported or incomplete. A parser failure is never converted into a clean result.

## Built-in rules

Rules have stable IDs, independent severity and confidence, remediation guidance, evidence locations, fix metadata, and documented limitations.

| Family | Rules | Representative findings |
|---|---|---|
| PDF | `LS-PDF-001`–`LS-PDF-005` | Ineffective redaction, unapplied annotation, attachment, active content, metadata |
| Word | `LS-DOCX-001`–`LS-DOCX-007` | Recoverable revision text, comments, hidden text, templates, embedded objects |
| PowerPoint | `LS-PPTX-001`–`LS-PPTX-006` | Speaker notes, hidden slides and shapes, comments, linked or embedded content |
| Excel | `LS-XLSX-001`–`LS-XLSX-006` | Hidden sheets and data, comments, external workbooks, embedded content |
| Images | `LS-IMG-001`–`LS-IMG-005` | GPS, device identity, creator data, thumbnails, descriptive metadata |
| OOXML package | `LS-OOXML-001`–`LS-OOXML-004` | VBA, external relationships, OLE/packages, document properties |
| Generic | `LS-GEN-001`–`LS-GEN-002` | Extension mismatch and incomplete analysis |

```bash
latenspect rules list
latenspect rules show LS-PDF-001
```

The complete catalog is in [docs/RULE_REFERENCE.md](docs/RULE_REFERENCE.md).

## Inspect, clean, and verify

Inspection never changes the input:

```bash
latenspect inspect confidential-proposal.docx
latenspect inspect documents/ --recursive --include '*.docx' --exclude 'archive/**'
latenspect inspect documents/ --recursive --quiet --fail-on medium
```

Cleaning always targets a different path, uses an atomic write, and rescans the output:

```bash
latenspect clean photo.jpg \
  --strip-metadata \
  --output photo.clean.jpg

latenspect clean presentation.pptx \
  --strip-metadata \
  --remove-notes \
  --remove-comments \
  --output presentation.clean.pptx \
  --yes
```

`--remove-notes` and `--remove-external-relationships` are destructive and require confirmation or `--yes`. Latenspect does not resolve tracked Word revisions, delete hidden slides or sheets, or flatten PDFs in 0.1.0. It refuses to overwrite an existing input or output unless the applicable explicit `--overwrite` option is present; it never cleans in place.

Verification rescans a file and can compare it with a prior JSON scan or clean report:

```bash
latenspect verify presentation.clean.pptx
latenspect verify presentation.clean.pptx --against before.latenspect.json
```

Comparisons distinguish resolved, still-present, new, and not-re-evaluated findings. An incomplete rescan remains incomplete.

## Reports and exit codes

`--format terminal|json|html|sarif` selects the reporter. HTML is a self-contained local file with no CDN, remote fonts, analytics, or server requirement. SARIF maps document locations into artifact and result properties rather than pretending that pages, slides, or worksheets are source-code lines.

| Exit | Meaning |
|---:|---|
| 0 | Scan completed and no finding met `--fail-on` |
| 1 | One or more findings met `--fail-on` |
| 2 | Invalid usage or operational error |
| 3 | At least one requested scan was incomplete, unsupported, or errored |

`--fail-on` accepts `critical`, `high`, `medium`, `low`, `info`, or `never`. For multi-file scans, the aggregate exit code reflects all per-file statuses while JSON and SARIF retain each file’s result.

## Local-only privacy model

Normal inspection and cleaning perform no network requests. Latenspect has no account, telemetry, server, database, background process, cloud API, or AI dependency. It never follows Office external relationships, loads remote XML resources, executes macros or PDF JavaScript, opens attachments, or launches external programs.

Installing Latenspect and vulnerability-checking dependencies can use the network; scanning does not. The threat model and hostile-input boundaries are documented in [docs/THREAT_MODEL.md](docs/THREAT_MODEL.md) and [docs/PRIVACY.md](docs/PRIVACY.md).

## Evidence is sensitive

The default `masked` policy keeps excerpts short and obscures most hidden text. Use `--evidence none` to omit excerpts entirely.

```bash
latenspect inspect evidence.pdf --evidence none
latenspect inspect evidence.pdf --reveal-evidence
```

`--reveal-evidence` is equivalent to full evidence and prints a warning to stderr. Full-evidence terminal, JSON, and HTML reports can themselves disclose sensitive content; store and share them accordingly. Latenspect avoids absolute paths in ordinary reports and never emits full hidden content merely because verbose logging is enabled.

## Configuration

No configuration is required. A `latenspect.toml` in the current working directory or one of its
parents may set defaults. Configuration is deliberately **not** discovered from the scanned file's
own directory tree, so a `latenspect.toml` that ships inside untrusted content cannot silently
disable rules or relax limits:

```toml
[scan]
fail_on = "high"
evidence = "masked"
max_file_size = "500MiB"
max_uncompressed_size = "1GiB"

[rules]
disable = []
enable = []

[paths]
exclude = [".git", ".venv", "node_modules", "__pycache__"]
```

Supported command-line options override environment variables, environment variables override the project file, and built-in defaults apply last. Supported variables are `LATENSPECT_FAIL_ON`, `LATENSPECT_EVIDENCE`, `LATENSPECT_MAX_FILE_SIZE`, and `LATENSPECT_MAX_UNCOMPRESSED_SIZE`. Unknown fields are rejected. `--no-config` disables file and environment configuration discovery for that command.

### Rule profiles

`--profile <name>` (or `[scan] profile`) is a named starting point that sets a default `fail-on` and a set of disabled rules; explicit config values and command-line options still win. Available profiles: `default` (never fail), `ci` (fail on high), `strict` (fail on medium), and `disclosure` (fail on high; mute low-signal metadata rules).

### Suppressing accepted findings

To keep a reviewed finding from failing a gate while preserving an audit trail, list its stable fingerprint in `latenspect-suppressions.toml` (discovered in the working directory or an ancestor, or passed with `--suppress <path>`; `--no-suppress` ignores it):

```toml
[[suppress]]
fingerprint = "…"          # from: latenspect inspect f.pdf --format json | jq '.findings[].fingerprint'
reason = "Reviewed: intentional watermark, not a redaction."
expires = 2026-12-31        # optional; the finding reappears after this date
```

A suppressed finding is still reported (marked, and emitted as a SARIF `suppressions` entry) but does not raise the risk level or trigger a `--fail-on` exit. Suppressions are never read from the scanned file's own directory tree.

## Installation

With [uv](https://docs.astral.sh/uv/):

```bash
uvx latenspect inspect suspicious-file.pdf
uv tool install latenspect
```

With pipx:

```bash
pipx install latenspect
latenspect doctor
```

With pip in an isolated environment:

```bash
python -m venv .venv
# POSIX: source .venv/bin/activate
# Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install latenspect
```

The Latenspect wheel is pure Python. Its PDF, XML, and image dependencies select maintained platform wheels during installation; run `latenspect doctor` to see the available local backends.

## Python API

The public API is synchronous, typed, `pathlib`-native, and does not expose parser-engine objects:

```python
from pathlib import Path

from latenspect import inspect_path
from latenspect.models import EvidencePolicy, ScanPolicy, ScanReport

policy = ScanPolicy(evidence=EvidencePolicy.MASKED)
report = inspect_path(Path("contract.docx"), policy=policy)

assert isinstance(report, ScanReport)  # A file scan; directories return ScanBundle.
print(report.status, report.risk)
print(report.model_dump_json(indent=2))
```

Cleaning requires explicit options and a distinct destination:

```python
from pathlib import Path

from latenspect import clean_path
from latenspect.models import CleanOptions

result = clean_path(
    Path("photo.jpg"),
    Path("photo.clean.jpg"),
    options=CleanOptions(strip_metadata=True),
)
assert result.original_unchanged
```

## CI and SARIF

This repository includes a composite action:

```yaml
- name: Privacy preflight
  uses: xvorki/latenspect@v0.14.0
  with:
    path: release-files
    recursive: "true"
    fail-on: high
    format: sarif
    output: latenspect.sarif
```

Or invoke the tool directly and upload its SARIF with the code-scanning action:

```yaml
- run: >-
    uvx latenspect inspect release-files --recursive
    --format sarif --output latenspect.sarif --fail-on high
- uses: github/codeql-action/upload-sarif@v4
  if: always()
  with:
    sarif_file: latenspect.sarif
```

The action installs the released package before scanning; the scan itself remains local and does not retrieve referenced document content.

## Demo corpus

All fixtures are synthetic and reproducible. They contain fictional names, `.invalid` addresses, and harmless test data—never real secrets or personal files.

```bash
uv run python scripts/generate_demo_fixtures.py
uv run latenspect inspect demo/generated/demo_fake_redaction.pdf --fail-on never
```

See [demo/README.md](demo/README.md) for the fixture matrix, deterministic sample reports, and regeneration commands.

## Limitations

- No OCR: image-only scanned pages and text hidden inside pixels are outside 0.1.0 analysis.
- PDF geometry can be uncertain with unusual clipping, transparency groups, blend modes, malformed fonts, uncommon user units, or deeply nested forms; confidence and limitations reflect that uncertainty.
- Detection is broader than repair. PDF redactions, Word revision resolution, hidden slide/sheet deletion, macros, and embedded objects require a compatible editor or specialized sanitizer.
- Metadata findings identify fields and evidence; Latenspect is not a general PII classifier, antivirus product, legal-compliance certificate, or proof of admissibility.
- Resource limits intentionally stop hostile or extremely large inputs. A stopped or partial scan is reported as incomplete.
- A complete result means only: no findings under the enabled Latenspect rules in content parsed successfully.

More detail is in [docs/LIMITATIONS.md](docs/LIMITATIONS.md) and [docs/FILE_FORMAT_SUPPORT.md](docs/FILE_FORMAT_SUPPORT.md).

## Contributing

Bug fixes, false-positive controls, deterministic synthetic fixtures, format research, and documentation improvements are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md) before opening a pull request.

Do not attach private documents to public issues. Reduce a case to a synthetic fixture or use the private security-reporting path.

## Security reporting

Please do not disclose parser crashes, path traversal, evidence leaks, unexpected network access, cleaning corruption, or dependency-compromise concerns in a public issue. Follow [SECURITY.md](SECURITY.md) and use GitHub private vulnerability reporting when available.

## Roadmap

Post-0.1 work may explore additional formats, stronger nested-PDF analysis, optional isolated OCR, richer policy profiles, and more conservative verified fixers. These are directions, not promises; inspection evidence and cleaning safety take priority over format count.

## License

Latenspect is licensed under the [Apache License 2.0](LICENSE). Third-party components retain their own licenses; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).

## Credits

| Role | Handle |
|------|--------|
| Owner | [Xvorki](https://github.com/xvorki) |
| Co-Owner | [Constripacity](https://github.com/constripacity) |
