Metadata-Version: 2.4
Name: trellis-pipelines
Version: 0.3.0
Summary: Modular agentic pipeline system for document and data workflows
Author: Ciaran Davies
License: MIT License
        
        Copyright (c) 2026 Ciaran Davies
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/ciarandavies7191/trellis
Project-URL: Repository, https://github.com/ciarandavies7191/trellis
Project-URL: Documentation, https://ciarandavies7191.github.io/trellis
Project-URL: Bug Tracker, https://github.com/ciarandavies7191/trellis/issues
Keywords: pipeline,agentic,dag,document,extraction,llm,pdf,sec,etl,workflow,orchestration,automation,finance,business,productivity
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Office/Business
Classifier: Typing :: Typed
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: litellm>=1.0.0
Requires-Dist: PyMuPDF>=1.24.0
Requires-Dist: pypdf>=3.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: typer>=0.9
Requires-Dist: numpy>=1.24
Requires-Dist: rank-bm25>=0.2.2
Requires-Dist: python-dateutil>=2.8
Provides-Extra: retrieval
Requires-Dist: faiss-cpu>=1.7.4; extra == "retrieval"
Requires-Dist: sentence-transformers>=2.6; extra == "retrieval"
Provides-Extra: api
Requires-Dist: fastapi>=0.104; extra == "api"
Requires-Dist: uvicorn[standard]>=0.24; extra == "api"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: httpx>=0.24; extra == "dev"
Requires-Dist: black>=23.0; extra == "dev"
Requires-Dist: isort>=5.12; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Requires-Dist: ruff>=0.1; extra == "dev"
Provides-Extra: all
Requires-Dist: trellis-pipelines[api]; extra == "all"
Dynamic: license-file

# Trellis

**Modular agentic pipeline system for document and data workflows.**

Trellis lets you define multi-step AI pipelines as declarative YAML — ingest PDFs, fetch SEC filings, call LLMs, extract structured fields, fan out over lists — and run them from the CLI, a REST API, or directly from Python.

**[Documentation](https://ciarandavies7191.github.io/trellis)** · [Installation](https://ciarandavies7191.github.io/trellis/installation) · [Pipeline DSL Reference](https://ciarandavies7191.github.io/trellis/PIPELINE-DSL) · [Examples](examples/pipelines/)

---

## Install

```bash
pip install trellis-pipelines
```

With [uv](https://github.com/astral-sh/uv):

```bash
uv add trellis-pipelines
```

Requires Python 3.12+. Set at least one LLM provider key before running pipelines:

```bash
export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_API_KEY, etc.
```

---

## Quickstart

### CLI

```bash
# Validate a pipeline file
trellis validate examples/pipelines/pdf_summarize.yaml

# Run a pipeline
trellis run examples/pipelines/pdf_summarize.yaml

# Pass runtime parameters
trellis run examples/pipelines/fetch_10k_parametrized.yaml \
  --params '{"ticker": "AAPL", "year": "2024"}'

# Override the LLM model for all tasks
trellis run examples/pipelines/pdf_summarize.yaml --model anthropic/claude-haiku-4-5
```

### A pipeline file

```yaml
pipeline:
  id: pdf_summarize
  goal: "Load a PDF and summarize key points"
  tasks:
    - id: ingest_pdf
      tool: ingest_document
      inputs:
        path: "https://example.com/report.pdf"

    - id: extract_content
      tool: extract_from_texts
      inputs:
        document: "{{ingest_pdf.output}}"
        prompt: "Extract the main topics and key metrics"

    - id: summarize
      tool: llm_job
      inputs:
        prompt: |
          Summarize in 5 bullet points for a busy executive:
          {{extract_content.output}}
        max_tokens: 256
```

Dependencies are inferred from `{{task_id.output}}` references — no explicit wiring needed.

### Python

```python
import asyncio
from trellis.models.pipeline import PipelineSpec
from trellis.execution.orchestrator import Orchestrator

spec = PipelineSpec.from_yaml("examples/pipelines/pdf_summarize.yaml")
orchestrator = Orchestrator()
result = asyncio.run(orchestrator.run_pipeline(spec))

print(result.outputs)
```

### REST API

```bash
# Start the server (requires trellis-pipelines[api])
pip install "trellis-pipelines[api]"
uvicorn trellis_api.main:app --reload
```

```bash
curl -X POST http://localhost:8000/pipelines/run \
  -H "Content-Type: application/json" \
  -d '{
    "pipeline": {
      "id": "hello",
      "goal": "Say hello",
      "tasks": [
        {
          "id": "greet",
          "tool": "llm_job",
          "inputs": { "prompt": "Say hello in one sentence." }
        }
      ]
    }
  }'
```

---

## Key features

- **Declarative YAML DSL** — flat task list, dependencies inferred from template references
- **Template resolution** — `{{task_id.output}}`, `{{params.key}}`, `{{session.key}}`, `{{item}}` (fan-out)
- **Fan-out / parallel_over** — scatter a task over a list, collect results automatically
- **`await` barriers** — explicit synchronization across parallel branches
- **Retry & backoff** — per-task `retry` with exponential backoff and jitter
- **Structured extraction** — `extract_fields` with JSON Schema, `extract_from_texts` for free-form
- **PDF + web ingestion** — `ingest_document` (PDF/HTML), `fetch_url`, `search_web`
- **Multi-tenancy** — tenant-scoped blackboard (`store` / `{{session.key}}`) for stateful workflows
- **CLI, REST API, and Python SDK** — three interfaces, one engine

---

## Project structure

```
trellis/            # Core: models, execution engine, tool registry
trellis_api/        # FastAPI REST server (optional extra: [api])
trellis_cli/        # Typer CLI
trellis_mcp/        # MCP server adapter (roadmap)
examples/           # Example pipelines and data
docs/               # MkDocs source
tests/              # Test suite
```

---

## Optional extras

| Extra | What it adds |
|---|---|
| `trellis-pipelines[api]` | FastAPI + uvicorn REST server |
| `trellis-pipelines[dev]` | pytest, ruff, mypy, black, isort |
| `trellis-pipelines[all]` | All extras |

---

## Documentation


- [Installation](https://ciarandavies7191.github.io/trellis/installation)
- [Quickstart](https://ciarandavies7191.github.io/trellis/quickstart)
- [Pipeline DSL Reference](https://ciarandavies7191.github.io/trellis/PIPELINE-DSL)
- [Tools & Registry](https://ciarandavies7191.github.io/trellis/tools-index)
- [CLI reference](https://ciarandavies7191.github.io/trellis/interfaces-cli)
- [REST API reference](https://ciarandavies7191.github.io/trellis/interfaces-api)

---

## License

MIT — see [LICENSE](LICENSE) or [https://opensource.org/license/mit](https://opensource.org/license/mit).
