Metadata-Version: 2.4
Name: vector-forget
Version: 0.1.0
Summary: Hard-delete AND verify erasure of a data subject from your vector store — the GDPR right-to-be-forgotten that a plain .delete() fails.
Project-URL: Homepage, https://github.com/privatebydefault/vector-forget
Project-URL: Issues, https://github.com/privatebydefault/vector-forget/issues
Project-URL: Source, https://github.com/privatebydefault/vector-forget
Author: privatebydefault
License: MIT License
        
        Copyright (c) 2026 privatebydefault
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: ccpa,data-erasure,embeddings,gdpr,ghost-vectors,pgvector,pii,rag,right-to-be-forgotten,vector-database
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Security
Classifier: Typing :: Typed
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: psycopg[binary]>=3.1; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Provides-Extra: pgvector
Requires-Dist: psycopg[binary]>=3.1; extra == 'pgvector'
Description-Content-Type: text/markdown

# vector-forget

[![CI](https://github.com/privatebydefault/vector-forget/actions/workflows/ci.yml/badge.svg)](https://github.com/privatebydefault/vector-forget/actions/workflows/ci.yml)

**Hard-delete _and verify_ erasure of a data subject from your vector store.**

When a GDPR / CCPA "right to be forgotten" request lands, `store.delete()` is not
enough — most vector databases **soft-delete**: the vectors vanish from search while the
raw bytes stay on disk, reconstructible. (See *Ghost Vectors*, arXiv:2606.18497 — deleted
embeddings recovered at high rates from raw index files.) A single-subject delete also
never trips the background compaction/vacuum that would actually reclaim them.

`vector-forget` does the two things a plain delete doesn't:

1. **Hard-delete** — forces the physical reclaim (e.g. pgvector `REINDEX` + `VACUUM`) so
   the ghost vectors are actually gone, not just hidden.
2. **Verify + report** — proves it with the backend's own residue signal and hands you a
   structured `ForgetReport`.

> ⚠️ **Alpha / v0.1 — pgvector only.** Qdrant, Weaviate and framework integrations are
> deliberately deferred (see [Non-goals](#non-goals)).

## Install

```bash
pip install "vector-forget[pgvector]"
```

Requires Python 3.9+ and Postgres with the `vector` extension. The physical-residue proof
(the wedge) additionally needs `pgstattuple` and table-owner / `MAINTAIN` rights — i.e. a
**self-hosted / owned** Postgres. On managed Postgres it still deletes and verifies
visibility, but the residue gate degrades honestly to `UNVERIFIED-managed` (see below).

## Quickstart (pgvector)

```python
import psycopg
from vector_forget import stamp, forget, verify
from vector_forget.pgvector import PgVectorStore

# 0) connect to your Postgres (self-hosted, for the full residue proof)
conn = psycopg.connect("postgresql://user:pass@localhost:5432/mydb")
store = PgVectorStore(conn, table="documents", subject_col="subject_id")

# 1) at INGEST — tag each row so it can be forgotten later (one-line shim)
metadata = stamp(metadata, subject_id="user_42")   # wrap your existing add_documents/insert

# 2) on an ERASURE request — hard-delete + force physical reclaim + verify
report = forget("user_42", store)
print(report.to_json(indent=2))
print("erased & verified:", report.ok)             # True only on visibility PASS + VERIFIED

# safe on a hot prod DB — delete + verify only, no REINDEX/VACUUM (residue stays PENDING)
report = forget("user_42", store, force=False)

# standalone proof pass, no delete
report = verify("user_42", store)
```

## What it honestly claims — and does not

`ForgetReport` carries this boundary **in the report itself** (not just the docs):

> Scoped to the live store(s) you pointed it at, at the time it ran. Does **not** cover
> backups/snapshots, replicas, WAL/logs, other stores (caches, a second DB, object
> storage, logs), embedding-provider-side copies, or data baked into **fine-tuned model
> weights**. It is a removal **report**, not a signed **certificate** of erasure.

On managed Postgres (RDS/Neon/Supabase) the physical-residue check may be unavailable →
the report says `residue: UNVERIFIED-managed` rather than claiming more than it can prove.
Under-claiming is the design rule.

Package-provenance attestations (PEP 740) emitted by the PyPI publish pipeline attest the
**wheel's** build origin — they are **not** an erasure attestation and never sign a
`ForgetReport`. Don't conflate a signed package with a signed proof of deletion.

### Residue states

| `residue` | meaning |
|---|---|
| `VERIFIED` | dead tuples reclaimed + index rebuilt (not "prior bytes overwritten") |
| `PENDING-lock-timeout` | reclaim hit the bounded lock wait — transient, retry |
| `PENDING-<pin>` | dead tuples pinned by a long txn / replication slot / prepared xact |
| `UNVERIFIED-managed` | managed PG / least-privilege role blocked the reclaim |
| `UNVERIFIED-index-not-rebuilt` | a vector index left invalid (ghost nodes may survive) |
| `UNVERIFIED-dead-tuples-remain` | tuples remained, no transient pin found |

`report.ok` is `True` **only** on visibility `PASS` and `residue == VERIFIED`.

## Operational caveats

- **`force_reclaim` can wait on long-running transactions.** `REINDEX INDEX CONCURRENTLY`
  must wait for older snapshots to finish — correct Postgres behaviour, but on a busy DB
  it could otherwise block indefinitely. The reclaim connection therefore runs with a
  bounded `lock_timeout` (default **5 s**, `reclaim_lock_timeout_ms`; `0` disables). On
  timeout the run degrades to `residue: PENDING-lock-timeout` — a **transient, retryable**
  state, *not* the permanent `UNVERIFIED-managed` — and `forget()` returns instead of
  hanging. A timed-out `REINDEX CONCURRENTLY` can leave an `INVALID` index behind; drop it
  (`DROP INDEX …`) or a later run will flag it as `UNVERIFIED-index-not-rebuilt`.
- **Retry is an opt-in poll:** `forget(..., retries=N, backoff_ms=M)` re-attempts the
  reclaim while the residue is a transient `PENDING-*` (e.g. the blocking txn clears
  between attempts). It **never** turns an `UNVERIFIED-*` or a visibility `FAIL` into a
  pass, and total added wait is bounded.
- **Zero-risk mode:** `force=False` does delete + verify only (no `REINDEX`/`VACUUM`), so
  it can never block on a lock. It reports `residue: PENDING` (compaction not performed).
- **`reclaim_statement_timeout_ms`** (default `0` / off) is an extra hard cap on reclaim
  runtime. Leave it off unless you want to bound total work — unlike `lock_timeout` (which
  targets *waits*), it can cancel a legitimately long `VACUUM`/`REINDEX`.

## Non-goals

- **Not a signed erasure certificate / crypto-shred / cryptographic proof** — `ForgetReport`
  is deliberately an unsigned removal report.
- **Not machine-unlearning of fine-tuned weights** — out of a library's reach; PII baked
  into model weights is explicitly out of scope.
- **Not a hosted service or daemon** — library only.
- **v0.1 is pgvector-only** — Qdrant, Weaviate, and framework (LangChain/LlamaIndex)
  integrations are deferred.

## Status

Alpha (v0.1).

## License

MIT © privatebydefault
