Metadata-Version: 2.5
Name: axiom-data-pipeline-topology
Version: 3.0.1
Summary: Pure scalable Axiom topology validation and immutable graph compilation.
Project-URL: Homepage, https://github.com/zephytiju/AxiomDataPipelineTopology
Project-URL: Repository, https://github.com/zephytiju/AxiomDataPipelineTopology.git
Author-email: Juntai Team <dev@juntai.com>
License-File: LICENSE
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# Axiom Data Pipeline Topology

[简体中文](README_zh.md) · [Development](DEVELOPMENT.md)

`axiom-data-pipeline-topology` is the pure, runtime-neutral graph compiler shared
by Axiom Definition and Axiom Runtime. It validates normalized workflow nodes,
edges, and released `axiom.source-binding/v1` values, then emits exact immutable
manifest, chunk, and routing-index bytes.

The package contains no Meridian client, persistence adapter, service, runtime
state, physical storage choice, provider API, or deployment authority.

## Compilation

```python
from juntai.axiom.data_pipeline.topology import (
    ResourceRequest,
    TaskNode,
    TopologyCompiler,
)

connector = TaskNode(
    node_id="connector",
    workflow_type="ingestion",
    image_label="portable-python",
    workflow_import_path="acme.social:SocialConnector",
    workflow_folder="artifact://sha256:" + "1" * 64,
    resource_request=ResourceRequest(cpu=1, memory_mb=512),
    config={},
    security_labels={"classification": "internal"},
)

compiled = TopologyCompiler(chunk_size=512).compile(
    pipeline_id="social-pipeline",
    nodes=[connector],
    edges=[],
    source_bindings=[released_source_binding],
)

for relative_path, exact_bytes in compiled.artifacts().items():
    publish(relative_path, exact_bytes)
```

The compiler accepts source-binding mappings or frozen models exposing
`model_dump()`, including the public values released by `axiom-data-workflow`
2.1. It does not duplicate that package's source semantics.

## Artifacts

- `topology-manifest.json` records the definition digest, counts, ranks, chunks,
  and exact routing-index digest.
- `topology-chunks/NNNNNN.json` contains bounded normalized nodes and each
  outgoing edge exactly once. Nodes include deterministic upstream, downstream,
  and scheduling-rank data.
- `topology-routing-index.json` maps every node and exact
  `(source fingerprint, raw contract)` pair to its connector chunk.

Every artifact exposes a SHA-256 `digest` alongside its exact canonical bytes.
Equivalent input permutations compile to byte-identical artifacts. Validation
issues use stable codes and paths, and ambiguous source routes fail before any
artifact is produced.

Graph validation, adjacency construction, rank assignment, chunk population,
and route construction are O(nodes + edges). Artifact-boundary ordering makes
publication deterministic. The scale suite compiles both sparse and high-fanout
10,000-node fixtures without a graph-size ceiling.

## Compatibility and release

Version 3 retains the pure 2.x `TaskNode`, `ResourceRequest`, `ExecutionGraph`,
`TopologyValidator`, and `TopologyRevisionCreate` inputs. The legacy 256-node
limit is removed. KES persistence, activation services, schedules, and
provider-shaped database schemas were intentionally removed because they were
outside this repository's authority.

Run the complete verification before release:

```bash
python -m pytest
ruff check src tests scripts
python -m build
python scripts/verify_clean_consumer.py
```

The merged `3.0.1` commit is published through PyPI Trusted Publishing and the
matching immutable GitHub release workflow.

## License

See [LICENSE](LICENSE).
