Metadata-Version: 2.4
Name: pipeio
Version: 0.1.0
Summary: Pipeline registry, notebook lifecycle, and flow management for research repositories
Author-email: Arash Shahidi <A.Shahidi@campus.lmu.de>
License-Expression: MIT
Project-URL: Homepage, https://github.com/arashshahidi1997/pipeio
Project-URL: Documentation, https://arashshahidi1997.github.io/pipeio/
Project-URL: Repository, https://github.com/arashshahidi1997/pipeio
Project-URL: Issues, https://github.com/arashshahidi1997/pipeio/issues
Keywords: pipeline,workflow,snakemake,notebook,registry,research
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Build Tools
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6.0
Requires-Dist: pydantic>=2.0
Provides-Extra: bids
Requires-Dist: snakebids>=0.15; extra == "bids"
Provides-Extra: notebook
Requires-Dist: jupytext>=1.16; extra == "notebook"
Requires-Dist: nbconvert>=7; extra == "notebook"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9; extra == "docs"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# pipeio

Agent-facing authoring and discovery layer for computational pipelines in research repositories.

Part of the [projio](https://github.com/arashshahidi1997/projio) ecosystem.

## What pipeio does

pipeio makes pipeline knowledge queryable and actionable for AI agents. It does not compete with execution engines (Snakemake), provenance systems (DataLad), app lifecycle managers (snakebids), or path resolvers (snakebids `bids()`).

Each **flow** is a self-contained snakebids app producing one derivative directory. The `pipe` field is a category tag grouping related flows. **Mods** are logical modules (rule groups) within a flow.

### Core features

- **Registry & discovery** — scan, load, validate, and query the flow/mod hierarchy
- **AI-safe authoring** — `rule_insert`, `config_patch`, `mod_create` with validation
- **Contract semantics** — declarative I/O validation and cross-flow wiring
- **Flow config** — declarative output registry (data contracts) in `config.yml`
- **Notebook lifecycle** — pair, sync, execute, and publish Jupytext notebooks
- **Scaffolding** — create new flows and mods from templates
- **Documentation** — collect, nav generation, modkey bibliography

## Install

```bash
pip install pipeio                # core
pip install pipeio[notebook]      # + jupytext/nbconvert
pip install pipeio[bids]          # + snakebids adapter
```

## CLI

```
pipeio init                       # scaffold .pipeio/ in the current project
pipeio flow list [--pipe PIPE]    # list all flows
pipeio flow new <pipe> <flow>     # scaffold a new flow
pipeio registry scan              # discover flows from filesystem
pipeio registry validate          # validate registry consistency
```

## Python API

```python
from pipeio import FlowConfig, PipelineContext

cfg = FlowConfig.from_yaml(Path("code/pipelines/preprocess/ieeg/config.yml"))
ctx = PipelineContext.from_config(cfg, root=Path("."))

sess = ctx.session(subject="01", session="pre")
path = sess.get("badlabel", "npy")
paths = sess.bundle("badlabel")  # {'npy': Path(...), 'featuremap': Path(...)}
```

## Implementation status

| Module | Status | Description |
|--------|--------|-------------|
| Registry | **Done** | scan, load, validate, query, YAML round-trip |
| Flow config | **Done** | load, extra inputs, groups/products, validation |
| Path resolution | **Done** | SimpleResolver, PipelineContext, Session, Stage |
| CLI | **Done** | init, flow list/new, registry scan/validate |
| MCP tools | **Done** | 35 tools across 7 categories (authoring, discovery, contracts, notebooks, docs) |
| Contracts | **Done** | Models, I/O validation, cross-flow wiring |
| Notebook lifecycle | **Done** | pair, sync, execute, publish, pipeline composite |
| BIDS adapter | Stub | Requires `pipeio[bids]` |

### Delegation

pipeio delegates execution concerns to specialized tools:

- **Execution**: snakebids `run.py` → Snakemake
- **Provenance**: DataLad run records
- **Path resolution**: snakebids `bids()` + `generate_inputs()`
- **App lifecycle**: snakebids deployment modes

## Development

```bash
pip install -e ".[dev]"
make test
```
