Metadata-Version: 2.4
Name: shmpipeline
Version: 1.0.2
Summary: Process-based shared-memory compute pipelines built on pyshmem.
Author: Jacob Taylor
License-Expression: GPL-3.0-only
Project-URL: Homepage, https://github.com/jacotay7/shmpipeline
Project-URL: Repository, https://github.com/jacotay7/shmpipeline
Project-URL: Issues, https://github.com/jacotay7/shmpipeline/issues
Project-URL: Changelog, https://github.com/jacotay7/shmpipeline/blob/main/CHANGELOG.md
Keywords: shared-memory,pipeline,multiprocessing,numba,cuda
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numba>=0.60
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pyshmem>=1.0.0
Requires-Dist: PyYAML>=6.0
Provides-Extra: test
Requires-Dist: fastapi>=0.115; extra == "test"
Requires-Dist: httpx>=0.27; extra == "test"
Requires-Dist: pytest>=8.3; extra == "test"
Requires-Dist: pytest-cov>=5; extra == "test"
Requires-Dist: ruff>=0.11; extra == "test"
Requires-Dist: uvicorn>=0.30; extra == "test"
Provides-Extra: control
Requires-Dist: fastapi>=0.115; extra == "control"
Requires-Dist: httpx>=0.27; extra == "control"
Requires-Dist: uvicorn>=0.30; extra == "control"
Provides-Extra: docs
Requires-Dist: myst-parser>=3.0; extra == "docs"
Requires-Dist: Sphinx>=8.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=3.0; extra == "docs"
Provides-Extra: gpu
Requires-Dist: torch>=2.2; extra == "gpu"
Provides-Extra: gui
Requires-Dist: fastapi>=0.115; extra == "gui"
Requires-Dist: httpx>=0.27; extra == "gui"
Requires-Dist: PySide6>=6.8; extra == "gui"
Requires-Dist: pyqtgraph>=0.14; extra == "gui"
Requires-Dist: uvicorn>=0.30; extra == "gui"
Dynamic: license-file

# shmpipeline

## Links

- [PyPI](https://pypi.org/project/shmpipeline/)
- [Read the Docs](https://shmpipeline.readthedocs.io/en/latest/)
- [Issues](https://github.com/jacotay7/shmpipeline/issues)

## What It Is

`shmpipeline` builds local, process-based compute pipelines on top of named
shared-memory streams from `pyshmem`.

It is a good fit when you want explicit dataflow, process isolation, and
shared-memory throughput without giving up a small Python API, YAML-driven
configuration, or interactive tooling.

Typical uses include:

- adaptive optics and other real-time sensor / control pipelines
- CPU or GPU processing graphs around shared-memory streams
- validating, inspecting, and running pipelines from Python or the CLI
- editing and supervising pipelines from a desktop GUI
- extending the runtime with custom kernels, sources, and sinks

Good starting points:

- [Installation guide](https://shmpipeline.readthedocs.io/en/latest/getting-started/installation.html)
- [Quickstart](https://shmpipeline.readthedocs.io/en/latest/getting-started/quickstart.html)
- [Worked examples](https://shmpipeline.readthedocs.io/en/latest/examples/index.html)

## Installation

Full docs: [Installation guide](https://shmpipeline.readthedocs.io/en/latest/getting-started/installation.html)

Choose the smallest install that matches your workflow:

- Base runtime and CLI: `pip install shmpipeline`
- GPU support: `pip install "shmpipeline[gpu]"`
- Desktop GUI: `pip install "shmpipeline[gui]"`
- Remote control service: `pip install "shmpipeline[control]"`
- Editable source install: `pip install -e .`
- Full local development environment: `pip install -e ".[control,gpu,gui,test,docs]"`

Quick smoke test against a checked-in example:

```bash
shmpipeline validate examples/affine_transformation/pipeline.yaml
shmpipeline describe examples/affine_transformation/pipeline.yaml --json
```

## Open The GUI For The Observatory AO Example

Relevant docs:

- [GUI guide](https://shmpipeline.readthedocs.io/en/latest/guides/gui.html)
- [Observatory AO system example](https://shmpipeline.readthedocs.io/en/latest/examples/observatory-ao-system.html)

From the repository root:

```bash
pip install "shmpipeline[gui]"
shmpipeline-gui examples/observatory_ao_system/pipeline.yaml
```

The full GUI can auto-launch a local loopback control server when you press
`Build` or `Start`.

If you want to launch the server yourself first:

```bash
shmpipeline serve examples/observatory_ao_system/pipeline.yaml --host 127.0.0.1 --port 8765
shmpipeline-gui examples/observatory_ao_system/pipeline.yaml
```

## Quickstart For The Python API

Full docs:

- [Quickstart](https://shmpipeline.readthedocs.io/en/latest/getting-started/quickstart.html)
- [Configuration guide](https://shmpipeline.readthedocs.io/en/latest/getting-started/configuration.html)

Minimal pipeline config:

```yaml
shared_memory:
  - name: input_frame
    shape: [4]
    dtype: float32
    storage: cpu

  - name: scaled_frame
    shape: [4]
    dtype: float32
    storage: cpu

kernels:
  - name: scale_stage
    kind: cpu.scale
    input: input_frame
    output: scaled_frame
    parameters:
      factor: 2.0
```

Run it from Python:

```python
import numpy as np

from shmpipeline import PipelineConfig, PipelineManager

config = PipelineConfig.from_yaml("pipeline.yaml")
manager = PipelineManager(config)
manager.build()
manager.start()

manager.get_stream("input_frame").write(np.array([1, 2, 3, 4], dtype=np.float32))
result = manager.get_stream("scaled_frame").read_new(timeout=2.0)
print(result)

manager.stop()
manager.shutdown()
```

If you prefer the CLI for the same pipeline:

```bash
shmpipeline validate pipeline.yaml
shmpipeline describe pipeline.yaml
shmpipeline run pipeline.yaml --duration 5.0
```

## Popular Example Pages

- [Affine transformation](https://shmpipeline.readthedocs.io/en/latest/examples/affine-transformation.html)
- [Basic AO system](https://shmpipeline.readthedocs.io/en/latest/examples/basic-ao-system.html)
- [Observatory AO system](https://shmpipeline.readthedocs.io/en/latest/examples/observatory-ao-system.html)
- [Source and sink plugins](https://shmpipeline.readthedocs.io/en/latest/examples/source-sink-plugins.html)

## More Documentation

Start with these in roughly the order most users need them:

1. [Configuration guide](https://shmpipeline.readthedocs.io/en/latest/getting-started/configuration.html) for the full YAML model, parameters, and shared-memory definitions.
2. [Worked examples](https://shmpipeline.readthedocs.io/en/latest/examples/index.html) for complete CPU, GPU, custom-operation, AO, and plugin-backed pipelines.
3. [CLI guide](https://shmpipeline.readthedocs.io/en/latest/guides/cli.html) for `validate`, `describe`, `run`, and `serve`.
4. [GUI guide](https://shmpipeline.readthedocs.io/en/latest/guides/gui.html) for editing configs, validating locally, inspecting graphs, and launching viewers.
5. [Runtime guide](https://shmpipeline.readthedocs.io/en/latest/guides/runtime.html) for lifecycle, metrics, worker health, and synthetic inputs.
6. [Extensions guide](https://shmpipeline.readthedocs.io/en/latest/guides/extensions.html) for custom kernels, sources, and sinks.
7. [Control plane guide](https://shmpipeline.readthedocs.io/en/latest/guides/control-plane.html) for remote management, SSE events, and the Python client.
8. [API reference](https://shmpipeline.readthedocs.io/en/latest/reference/index.html), [core API](https://shmpipeline.readthedocs.io/en/latest/reference/core.html), and [kernel catalog](https://shmpipeline.readthedocs.io/en/latest/reference/kernels.html) for the detailed reference surface.
9. [Troubleshooting](https://shmpipeline.readthedocs.io/en/latest/guides/troubleshooting.html) for common setup and runtime issues.
