Evidence-driven, local-first AI copilot for data quality. Detect · Explain · Validate · Repair — with statistical evidence, AI assistance, and human approval. Your data never leaves your machine.
One scan produces a six-dimension quality score; every issue carries a statistical evidence chain — samples, ratios, confidence. AI proposes; humans approve.
Missingness, dates, encodings, cross-field rules, cross-table foreign keys, exact + fuzzy duplicates, Isolation Forest / LOF outliers.
Completeness, validity, uniqueness, consistency, integrity, timeliness — explainable weights, per-dimension contributions.
Propose → preview (rules re-run) → apply (fingerprinted copy) → rollback. AI suggests; you decide.
Compare historical scans: schema, row-count, score and issue-distribution drift between versions.
scan --fail-on blocks releases; reports as
JSON / Markdown / HTML / JUnit / SARIF.
Persistent job queue in SQLite: cron expressions, manual triggers, run history, webhooks, quality gates and change-aware skip (no re-scan when content is unchanged).
Any instance runs as a worker
(datasentry worker); a pool with round-robin routing, failover,
cooldown and health checks, plus parallel dispatch
(DATASENTRY_MAX_WORKERS).
PII redacted before every prompt, encrypted vault with key rotation, calls audited, rules approved after preflight simulation. Runs on local Ollama.
CLI, REST API, server-rendered Web UI with cross-scan trends, and an MCP stdio server (20 tools) for LLM agents.
DuckDB execution engine, 1e6 rows in ~10s. No cloud, no telemetry, no vendor lock-in.
Detector plugin API (plugins/ auto-load
with SHA-256 integrity locks), contract exports to Pandera / Great
Expectations, 105 ADRs of design rationale.
200 rows of order data with 15 injected quality issues — scanned and reported in one
command. This is the actual report export --as html output, no custom rendering.
Install from PyPI and scan your first file in under a minute.
# install
$ pip install datasentry-ai
# scan → fuse → score → persist, one step
$ datasentry scan orders.csv
{
"scan_run_id": "scan_3426008c0380",
"status": "completed",
"row_count": 200,
"issues_count": { ... }
}
# issues, score, repair, drift
$ datasentry issues list
$ datasentry score scan_3426008c0380
$ datasentry repair propose <issue_id> --file orders.csv
$ datasentry drift latest orders
# scheduled jobs + distributed workers
$ datasentry job create nightly --cron "0 9 * * *" orders.csv
$ datasentry worker --port 8001 # execution node
$ DATASENTRY_WORKERS="http://localhost:8001:secret" datasentry-server
# Web UI + REST API
$ datasentry-server
→ http://localhost:8000/ui/ (scans, issues, repair workbench, trends)Per-step implementation notes, conventions, testing practices.
docs/DEVELOPMENT.md →