Metadata-Version: 2.4
Name: agentflow-runtime
Version: 1.3.0
Summary: Real-time data platform serving context to AI agents
License: MIT
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: bcrypt<6,>=5
Requires-Dist: confluent-kafka<3,>=2.5
Requires-Dist: dagster<2,>=1.13.1
Requires-Dist: duckdb<2,>=1.1
Requires-Dist: fastapi<1,>=0.111
Requires-Dist: httpx<1,>=0.27
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc<2,>=1.41
Requires-Dist: opentelemetry-instrumentation-fastapi<1,>=0.62b0
Requires-Dist: opentelemetry-instrumentation-httpx<1,>=0.62b0
Requires-Dist: opentelemetry-sdk<2,>=1.41
Requires-Dist: pandera<1,>=0.20
Requires-Dist: prometheus-client<1,>=0.21
Requires-Dist: pyarrow<19,>=17
Requires-Dist: pydantic-settings<3,>=2.5
Requires-Dist: pydantic<3,>=2.9
Requires-Dist: pyyaml<7,>=6
Requires-Dist: sqlglot<31,>=30
Requires-Dist: structlog<26,>=24.4
Requires-Dist: uvicorn[standard]<1,>=0.30
Provides-Extra: cloud
Requires-Dist: boto3<2,>=1.35; extra == 'cloud'
Requires-Dist: pyiceberg[pyiceberg-core]<1,>=0.7; extra == 'cloud'
Provides-Extra: contract
Requires-Dist: schemathesis==4.10.2; extra == 'contract'
Provides-Extra: dev
Requires-Dist: bandit<2,>=1.9; extra == 'dev'
Requires-Dist: build<2,>=1.2; extra == 'dev'
Requires-Dist: hatchling<2,>=1.25; extra == 'dev'
Requires-Dist: hypothesis<7,>=6; extra == 'dev'
Requires-Dist: jsonschema<5,>=4; extra == 'dev'
Requires-Dist: mypy<2,>=1.11; extra == 'dev'
Requires-Dist: pytest-asyncio<1,>=0.24; extra == 'dev'
Requires-Dist: pytest-cov<6,>=5; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.6; extra == 'dev'
Requires-Dist: testcontainers[kafka]<5,>=4.9; extra == 'dev'
Provides-Extra: flink
Requires-Dist: apache-flink==1.19.1; extra == 'flink'
Provides-Extra: integrations
Requires-Dist: langchain-core<2,>=1.2.22; extra == 'integrations'
Requires-Dist: langchain-text-splitters<2,>=1.1.2; extra == 'integrations'
Requires-Dist: langchain<2,>=0.3.30; extra == 'integrations'
Requires-Dist: langsmith<1,>=0.7.31; extra == 'integrations'
Requires-Dist: llama-index-core<1,>=0.12; extra == 'integrations'
Provides-Extra: llm
Requires-Dist: anthropic<1,>=0.39; extra == 'llm'
Provides-Extra: load
Requires-Dist: locust<3,>=2.29; extra == 'load'
Description-Content-Type: text/markdown

# AgentFlow

> Real-time data platform for AI agents. Live entity lookups, typed contracts, dual-language SDKs, and release-gated delivery.

[![Release gate](https://img.shields.io/badge/release_gate-v1.1_published-brightgreen)](docs/release-readiness.md)
[![codecov](https://codecov.io/gh/brownjuly2003-code/agentflow/branch/main/graph/badge.svg)](https://codecov.io/gh/brownjuly2003-code/agentflow)
[![Python](https://img.shields.io/badge/python-3.11+-blue)](pyproject.toml)
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

## Why this exists

Most agent demos work until they have to answer from live business state. Support, ops, and merch workflows need current orders, metrics, and health signals while the conversation is happening, not a stale warehouse snapshot and not a pile of one-off service adapters.

AgentFlow turns that problem into one serving boundary:

- streaming ingestion for operational events
- a semantic layer that exposes entities, metrics, and query endpoints
- typed contracts so SDKs and callers know what shape to expect
- Python and TypeScript clients that speak the same API surface

## Highlights

- **Release-line gate:** 752 passed, 4 skipped on 2026-05-04; GitHub environments `staging` and `production` have required reviewers. The 2026-04-27 audit closure sprint (Codex p1–p9 + Opus) shipped six commits closing all P0/P1/P2 findings — see [docs/audits/2026-04-27/README.md](docs/audits/2026-04-27/README.md) and Release Readiness for the live status
- **Sub-second entity lookups in the checked-in baseline**: entity p50 `38-55 ms`, entity p99 `290-320 ms`, aggregate p50 `56 ms` at `50` users for `60s`
- **Historical performance remediation is documented**: the serving path moved from an original ~`26,000 ms` baseline to the current `43-55 ms` release range
- **Dual SDK parity** for Python and TypeScript, including retry policies, circuit breakers, batching, pagination, and contract pinning
- **Postgres/MySQL CDC path** through Debezium and Kafka Connect, with local compose, Helm manifests, and canonical CDC normalization
- **Security hardening in the hot path**: parameterized queries, `sqlglot` AST validation for NL-to-SQL, and a Bandit baseline gate for new findings only
- **Release workflow coverage**: chaos smoke on PRs, performance regression gate, contract drift checks, and a Terraform apply workflow with OIDC-ready auth

## Quick start

> **Upgrading from v1.0.x?** See the [v1.1 migration guide](docs/migration/v1.1.md) before installing.

Prerequisites:

- Python `3.11+`
- `make`
- Docker Compose (`make demo` starts Redis)

PowerShell 7+:

```powershell
git clone https://github.com/brownjuly2003-code/agentflow.git
cd agentflow
. .\scripts\setup.ps1
make demo
```

macOS / Linux:

```bash
git clone https://github.com/brownjuly2003-code/agentflow.git
cd agentflow
source ./scripts/setup.sh
make demo
```

`make demo` seeds local data, starts Redis, and serves the API on `http://localhost:8000`. Swagger UI is available at `http://localhost:8000/docs`.

Try it:

```bash
curl http://localhost:8000/v1/entity/order/ORD-20260404-1001

curl -X POST http://localhost:8000/v1/query \
  -H "Content-Type: application/json" \
  -d '{"question":"Show me top 3 products"}'
```

Local demo runs without API-key enforcement unless you explicitly configure `AGENTFLOW_API_KEYS_FILE`.

## Architecture

```text
Event sources -> Kafka -> Flink -> Iceberg ----\
                                                -> Semantic layer -> FastAPI -> Agent / SDK
Local demo   -> local_pipeline -> DuckDB ------/
```

Stack:

- **Ingestion**: Kafka producers, Debezium/Kafka Connect CDC, and a local synthetic pipeline
- **Processing**: Flink plus validation and enrichment stages
- **Storage**: Iceberg for production-shaped tables, DuckDB for the local serving path
- **Serving**: FastAPI, contract registry, lineage, search, and operational endpoints
- **Orchestration**: Dagster
- **IaC**: Terraform, Helm, Docker Compose, and a Fly.io demo config

See [docs/architecture.md](docs/architecture.md) for the detailed design, trade-offs, and deployment topologies.

CDC source capture is standardized on Debezium/Kafka Connect; downstream consumers use the canonical AgentFlow CDC contract defined in [ADR 0005](docs/decisions/0005-cdc-ingestion-strategy.md).

## What's inside

| Area | Files |
|------|-------|
| API core | `src/serving/api/` |
| Semantic layer | `src/serving/semantic_layer/` |
| Python SDK | `sdk/agentflow/` |
| TypeScript SDK | `sdk-ts/src/` |
| Agent integrations | `integrations/agentflow_integrations/` (LangChain, LlamaIndex, CrewAI, MCP) |
| Flink jobs | `src/processing/flink_jobs/` |
| Test suites | `tests/` |
| Planning trail | `docs/plans/` |
| Public site | `site/` |
| IaC | `infrastructure/terraform/`, `infrastructure/dv2/`, `helm/`, `k8s/` |
| DV2.0 warehouse | `warehouse/agentflow/dv2/` (hubs / links / satellites + X5 loader) |

## Documentation

- [Interactive Technical Walkthrough](docs/index.md) - MkDocs Material guide with Mermaid architecture, API, SDK, deployment, observability, and troubleshooting pages
- [Architecture](docs/architecture.md) - system context, data flow, failure modes
- [Operational Runbook](docs/runbook.md) - local stack, CDC capture, incident response, and maintenance commands
- [API Reference](docs/api-reference.md) - endpoint-by-endpoint examples for curl, Python, and TypeScript
- [Security Audit](docs/security-audit.md) - threat model, controls, and evidence
- [Competitive Analysis](docs/competitive-analysis.md) - positioning and trade-offs
- [CDC Deployment Plan](docs/plans/2026-04-debezium-kafka-connect-deployment-plan.md) - Debezium/Kafka Connect rollout decisions and implementation trail
- [Glossary](docs/glossary.md) - interview-ready explanations of the core technical terms
- [Release Readiness](docs/release-readiness.md) - checked release evidence through `v1.1.0` and current external-gate handoffs
- [Customer Discovery Tracker](docs/customer-discovery-tracker.md) - synthetic/modelled PMF planning surface and real-evidence gap
- [Pricing Validation Plan](docs/pricing-validation-plan.md) - pricing/WTP questions, evidence gates, and current real-evidence gap
- [AWS OIDC Setup](docs/operations/aws-oidc-setup.md) - Terraform apply readiness handoff
- [Immutable Retention Evidence Handoff](docs/operations/immutable-retention-evidence-handoff.md) - external audit-retention evidence checklist
- [Production CDC Source Onboarding](docs/operations/cdc-production-onboarding.md) - production CDC decision record and no-go gates
- [External Gate Evidence Intake Checklist](docs/operations/external-gate-evidence-intake.md) - required owner evidence before blocked external gates can close
- [Public Production-Hardware Benchmark Plan](docs/perf/public-production-hardware-benchmark-plan.md) - benchmark evidence plan for `c8g.4xlarge+`
- [External Pen-Test Attestation Handoff](docs/operations/external-pen-test-attestation-handoff.md) - third-party pen-test evidence checklist
- [Audit History](docs/audit-history.md) - baseline-to-release remediation trail
- [Publication Checklist](docs/publication-checklist.md) - final GitHub publishing checklist
- [Fly.io Demo Deploy](deploy/fly/README.md) - minimal hosted demo instructions
- [DV2.0 Multi-Branch Extension](docs/dv2-multi-branch/SESSION_HANDOFF.md) - Data Vault 2.0 model for mid-market e-com with 5 locations / 3 jurisdictions; [schema](docs/dv2-multi-branch/schema_dv2.md), [end-to-end flow](docs/dv2-multi-branch/architecture.md), [live demo evidence](docs/dv2-multi-branch/demo_evidence.md), [k8s manifests](infrastructure/dv2/README.md)
- [Contributing](CONTRIBUTING.md) - development and PR expectations
- [Changelog](CHANGELOG.md) - project release notes

## Development

```bash
# verified release slice
python -m pytest tests/unit tests/integration tests/sdk -q

# benchmark and regression gate
python scripts/run_benchmark.py
python scripts/check_performance.py --baseline docs/benchmark-baseline.json --current .artifacts/load/results.json --max-regress 20

# benchmark trend: [.github/perf-history.json](.github/perf-history.json) is appended on every main push;
# render the history locally with `make perf-plot` (writes docs/perf/history.html).

# contracts and security
python scripts/generate_contracts.py --check
bandit -r src sdk --ini .bandit --severity-level medium -f json -o .tmp/bandit-current.json
python scripts/bandit_diff.py .bandit-baseline.json .tmp/bandit-current.json
```

## Status

**v1.1.0** is published to PyPI, npm, and GitHub.
The 2026-04-27 audit closure sprint landed six commits on `main`
that close all P0/P1/P2 findings from the Claude
Opus + Codex p1–p9 audits: tenant isolation across the control plane,
SQL guard centralization, entity allowlist enforcement on every read
surface, secrets scrubbed and rotated, helm `runAsNonRoot` /
NetworkPolicy / PodDisruptionBudget, npm lockfile + `npm audit` clean,
vulnerable dep bumps (`dagster>=1.13.1`, `langchain-core>=1.2.22`),
trivy pinned, OpenAPI drift gate, branch protection with 12 required
status checks, GitHub Actions environment reviewers, and Python SDK
alignment with the server v1 contract (F1–F10). Recent local full-suite
verification: `752 passed, 4 skipped` on 2026-05-04 after clarifying the
external-gate handoff. The post-v1.1 CDC operationalization
for Debezium / Kafka Connect is checked in, while production source
onboarding remains pending; see [docs/release-readiness.md](docs/release-readiness.md).
Remaining external gates are AWS OIDC role setup for real Terraform apply,
external immutable audit retention if claimed beyond local hash-chain evidence,
production CDC source onboarding, real PMF/pricing evidence, public benchmark
publication on production hardware, external pen-test attestation, and legacy
npm `NPM_TOKEN` revocation after a successful new-package trusted-publish run.
npm Trusted Publishing readback for the new package is complete. A project-local
Pi skill for evidence intake lives at `.pi/skills/external-gate-evidence-intake`.

## Screenshots

| Admin UI | API docs |
|----------|----------|
| <img src="docs/screenshots/admin-ui.png" alt="AgentFlow admin UI" width="420"> | <img src="docs/screenshots/swagger-docs.png" alt="AgentFlow API docs" width="420"> |

| Landing page | Benchmark run |
|--------------|---------------|
| <img src="docs/screenshots/landing-page.png" alt="AgentFlow landing page" width="420"> | <img src="docs/screenshots/benchmark-terminal.png" alt="AgentFlow benchmark terminal" width="420"> |

Capture notes and publish-time checks are listed in [docs/publication-checklist.md](docs/publication-checklist.md).

## License

MIT. See [LICENSE](LICENSE).

## Credits

Built as a data-engineering reference project during the `2026-04-10` -> `2026-04-20` release cycle, with the full implementation trail preserved in `docs/plans/`.
