Metadata-Version: 2.5
Name: fuju-trace-vexdb
Version: 0.1.9
Summary: VexDB storage and native BM25/vector search adapter for Fuju Trace
Author: Fuju Trace
License-Expression: MIT
Requires-Python: >=3.10
Requires-Dist: fuju-trace==0.1.9
Provides-Extra: driver
Requires-Dist: psycopg2-binary<3,>=2.9.5; extra == 'driver'
Description-Content-Type: text/markdown

# Fuju Trace × VexDB

[中文：本地 wheel 安装与数据库参数](README.zh-CN.md)

An optional VexDB backend for Fuju Trace's Python SDK. The core Rust engine and the normal Python SDK do not depend on VexDB. This adapter stores the SDK's original events and a folded span read model in VexDB. Native `fulltext`/BM25 and `graph_index` perform the two search branches; Fuju Trace combines their ranks with RRF.

## Install and connect

Install the published SDK with its VexDB extra. This installs the adapter and the generic `psycopg2-binary` driver used in the VexDB smoke test:

```bash
python -m pip install 'fuju-trace[vexdb]==0.1.9'
```

If `psycopg2-binary>=2.9.5,<3` is already installed, pip reuses it; this includes AgenticData's pinned 2.9.5. If your deployment supplies another compatible `psycopg2` driver itself, install `fuju-trace-vexdb==0.1.9` without the `driver` extra. From a source checkout, use `python -m pip install -e fuju-trace-sdk/python -e 'fuju-trace-vexdb[driver]'`. The VexDB vendor driver is not required for the tested adapter path.

Set `VEXDB_DSN` locally. Do not commit credentials. Choose the dimension of the embedding model you will use; it becomes part of the table schema. The example uses three dimensions only so it can be run without an embedding service.

```python
import os
from fuju_trace import DbExporter, Tracer, connect

with connect(vexdb_dsn=os.environ["VEXDB_DSN"], tenant_id=1,
             vector_dim=3, initialize=True) as db:
    tracer = Tracer(exporter=DbExporter(db, tenant_id=1), node_id=1)
    with tracer.trace("风控", tenant_id=1) as trace:
        with trace.span("研判") as span:
            span.log("疑似盗刷，需要人工复核")
    tracer.close()

    # Native VexDB BM25 on the folded span's input/output text and logs.
    hits = db.search(text="盗刷", k=10)
    print(hits)

    # Embeddings are supplied by the caller's model, not generated by this adapter.
    if hits:
        db.set_embedding(hits[0]["trace_id"], hits[0]["span_id"], [0.1, 0.2, 0.3])
        print(db.search(vector=[0.1, 0.2, 0.3], k=10))
        print(db.search(text="盗刷", vector=[0.1, 0.2, 0.3], k=10))
```

`initialize=True` creates four tables and native indexes under the `fuju_trace` prefix. Run it before the first use; later opens may omit it. `table_prefix` can isolate test data or another installation and accepts only lowercase SQL identifiers. A stored `vector_dim` mismatch raises an error. The adapter does not create embeddings; use `set_embedding` after ingest, or `set_embeddings([(trace_id, span_id, vector), ...])` to update existing spans in one transaction. The batch method returns the number of existing spans updated, rejects duplicate IDs within a batch, and validates all vectors before writing. It supports `trace(trace_id)` and `span(trace_id, span_id)` point reads, `list_spans(filters={"externalSessionId": task_id})`, and `list_trace_ids()` for session-scoped replay. `prune_before(cutoff_ns)` removes whole expired traces in bounded transactions. It does not implement every local embedded DB method.

## Data and query behavior

- Each event is deduplicated by `(tenant_id, ext_span_id, seq, event_type)`; its 64-bit FNV event ID is checked against the SDK and Rust engine. Rows are tenant scoped. Events, folded span, and exact attribute filter rows commit in one transaction. Writers for the same span lock its row before rebuilding it. `ingest(events)` combines multi-row SQL within that transaction and bounds statements to 256 rows.
- `search(text=...)` uses VexDB `search_text @~@ %s` and `bm25_score()`. `search(vector=...)` uses `floatvector` cosine distance and `graph_index`. Both supplied uses RRF with the same constant 60 as the Rust engine. Scores from different databases are not directly comparable.
- `tenant_id` is bound when opening the store. Search and reads always filter by it, and body or per-call overrides for another tenant are rejected. `filter` supports `trace_id`, `agent_name`, `status`, `time_from`, `time_to`, and arbitrary JSON values in `attrs`, plus direct `project_id`/`skill`/`mode`/`call_site` aliases. Unsupported filters raise rather than being ignored.
- The event table is the source of truth. The span and attribute tables are materialized read models. Future migrations/backfills must keep this boundary; indexes are VexDB-managed.
- VexDB's `graph_index` is approximate. Native BM25 tokenization and ANN filter behavior may differ from the local Rust engine. Validate recall and query plans on the actual VexDB version and data distribution before production use.
- `DbExporter` sends each event synchronously. For a remote VexDB connection with high event volume, the SDK's optional `BufferedDbExporter` can batch events (`max_batch=128`, `drop_when_full=False`). It changes visibility and durability timing: call `flush(timeout=...)` when a write barrier is needed, then inspect `health()` for `dropped` and `write_errors`. Keep `DbExporter` when each call must wait for persistence.
- Concurrent sessions can share one store per tenant in one process. The store guards its single database connection with a lock, so its writes and reads are serialized. Multiple Tracers with the same `node_id` in that process share one ID counter; different processes or hosts still need distinct node IDs (0–1023). The adapter indexes `session_id` for exact filtering through `list_spans()` and `list_trace_ids()`; it does not provide the full Fuju Trace session API.

## Live integration test

With a disposable VexDB database and `VEXDB_DSN` set, run:

```bash
python fuju-trace-vexdb/tests/live_smoke.py
python fuju-trace-vexdb/tests/live_bulk.py  # 260-row SQL chunk boundary
```

The test uses a unique table prefix, verifies deduplication, tenant rejection, BM25, vector, mixed search, and point reads, and then drops its test tables. Do not run it against a database where DDL is prohibited. The unit tests do not need a database:

```bash
python -m unittest discover -s fuju-trace-vexdb/tests -p 'test_*.py'
```

## Performance test

A reproducible end-to-end benchmark is in [`bench/bench_vexdb.py`](bench/bench_vexdb.py). It uses unique temporary tables and drops them after the run. Set `VEXDB_DSN` locally, then run from the repository root:

```bash
python fuju-trace-vexdb/bench/bench_vexdb.py --spans 1000 --single-spans 0 \
  --batch 128 --embedding-batch 128 --queries 20 --report /tmp/fuju-vexdb-bench.json
```

The [measured 1000-span report](../docs/reports/2026-09-25_fuju-trace-vexdb-performance.md) preserves the original baseline and the optimized runs, with raw JSON for each stage. It includes write throughput, BM25/vector/hybrid latency, recall, and index plans. These are remote Python-client measurements on a synthetic corpus, not VexDB engine-only results. Synchronous single-event ingest remains much slower than batched ingest.
