Metadata-Version: 2.5
Name: linkinpy
Version: 0.0.10
Summary: A python package to bring uv managed workflows to all common image analysis tools
Project-URL: Homepage, https://github.com/HenriquesLab/LinkinPy
Project-URL: Repository, https://github.com/HenriquesLab/LinkinPy
Project-URL: Issues, https://github.com/HenriquesLab/LinkinPy/issues
Project-URL: Changelog, https://github.com/HenriquesLab/LinkinPy/blob/main/LinkinPy/CHANGELOG.md
Author-email: "Bruno M. Saraiva" <bruno.msaraiva2@gmail.com>
License-Expression: MIT
License-File: LICENSE.txt
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >=3.9
Requires-Dist: ipyfilechooser>=0.6.0
Requires-Dist: ipywidgets>=8.0
Requires-Dist: magicgui>=0.10.0
Requires-Dist: numpy>=1.26
Requires-Dist: pyyaml>=6.0
Requires-Dist: stackview>=0.14.0
Requires-Dist: tifffile>=2024.8.10
Provides-Extra: gui
Requires-Dist: pyqt6>=6.7.0; extra == 'gui'
Provides-Extra: napari
Requires-Dist: napari>=0.5.0; extra == 'napari'
Description-Content-Type: text/markdown

# linkinpy

`linkinpy` is the Python core for running YAML-described bioimage-analysis methods in
isolated `uv` environments. It provides:

- a CLI: `list`, `install`, `run`, `validate`, `schema`, `batch`, `pipeline`, `envs`, `gc`,
  `serve` and `gui`
- spec parsing, linting (`validate`) and runtime validation of inputs
- per-spec environment management (locked installs, automatic recreation when the Python version changes)
- image loading, writing, and shared-memory transport helpers
- result normalization for single-output and multi-output methods, with real CSV/JSON/TIFF writers
- batch runs, spec-declared pipelines and a provenance record (`linkinpy_run.json`) for every run
- progress reporting and live log streaming
- a remote host and client (`serve`, `remote list|install`, `run --remote`) for running on another machine
- Colab helper widgets
- a Python facade API plus lower-level functions

Run commands in this document from the `LinkinPy/` folder.

## Development Setup

```bash
uv sync
uv run ruff check src tests
uv run pytest          # tests/test_core.py and tests/test_features.py
```

Optional developer commands:

```bash
uv run ruff format .
uv run python -m build
uv run pre-commit install
```

The same workflow is available through `make`:

```bash
make sync
make lint
make test
make format
make build
```

## CLI Tutorial

List available methods:

```bash
uv run linkinpy list ../Library/nanopyx.yaml
```

Install a spec for the CLI interface:

```bash
uv run linkinpy install ../Library/nanopyx.yaml --interface cli
```

Each spec gets its own environment under `~/.linkinpy/<spec-name>/`. If the spec contains
`how_to_cite`, the install command prints that citation text after the install summary.

Run a method and save declared outputs to a folder:

```bash
uv run linkinpy run ../Library/nanopyx.yaml eSRRF \
  --output-dir outputs \
  image=input.tif \
  magnification=2
```

Use explicit output destinations when needed:

```bash
uv run linkinpy run ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --output labels=outputs/labels.tif \
  image=input.tif \
  model_name=2D_versatile_fluo
```

Arguments are passed as `name=value`. For inputs declared as `np.ndarray`, the CLI accepts
image paths and `.npy` files.

## CLI Reference

| Command | Purpose |
| --- | --- |
| `linkinpy list SPEC` | List the methods a spec declares |
| `linkinpy install SPEC [--interface I]` | Create or update the spec's environment (prints `how_to_cite`) |
| `linkinpy run SPEC METHOD [name=value ...]` | Run a method (`--output-dir`, `--output name=path`, or `--remote URL` to run on a host) |
| `linkinpy validate SPEC [--strict]` | Lint a spec; non-zero exit on errors (or warnings with `--strict`) |
| `linkinpy schema` | Print the JSON Schema for spec files |
| `linkinpy batch SPEC METHOD --input-dir D --output-dir O` | Run one method over many images |
| `linkinpy pipeline list SPEC` / `run SPEC NAME` | List or run the spec's pipelines |
| `linkinpy envs list` / `refresh NAME` / `remove NAME` | Inspect and manage environments |
| `linkinpy gc [--dry-run] [--older-than DAYS]` | Remove stale temp files and orphaned shared memory |
| `linkinpy serve [--port N]` | Serve installed environments to remote clients (the "host") |
| `linkinpy remote list` / `install SPEC` | List what a host has installed, or install a spec on it |
| `linkinpy gui SPEC METHOD` | Open a generated Qt (magicgui) window for one method |

`SPEC` is a local path or an `https://` URL. Every command has `--help`.

### Validate and edit specs with completion

```bash
uv run linkinpy validate ../Library/StarDist.yaml            # 0 error(s), 0 warning(s)
uv run linkinpy validate --strict my_spec.yaml               # warnings fail too (used in CI)
uv run linkinpy schema > spec.schema.json
```

Add this as the first line of a spec to get completion in editors that use `yaml-language-server`:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/HenriquesLab/LinkinPy/main/docs/spec.schema.json
```

`validate` reports errors (duplicate names, `min` > `max`, a `default` outside `choices`, unsafe
output names, unknown pipeline steps...) and warnings (unknown type/role/format, missing
metadata). At run time the declared `type`, `min`, `max` and `choices` are enforced before the
method is called, so a bad value fails fast with a clear message:

```text
linkinpy.runner.RunnerError: Input 'gain' must be <= 10, got 99.0.
```

### Batch: one method over a folder

```bash
uv run linkinpy batch ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --input-dir raw/ --output-dir results/ --recursive --pattern "*.tif" --jobs 2 \
  model_name=2D_versatile_fluo
```

```text
[1/3] ok      a.tif (4.2s)
[2/3] ok      b.tif (3.9s)
[3/3] failed  c.tif (0.8s) ValueError: ...
2 ok, 1 failed, summary in results/batch_summary.csv
```

Each image gets its own folder (`results/<relative path>/<image stem>/`) with its outputs and
`linkinpy_run.json`. The environment is installed once up front, one failing image never stops
the batch, `--skip-existing` resumes an interrupted run, and the exit code is 1 if anything failed.
`--image-arg` names the image input when a method has several.

### Pipelines: chain methods inside one spec

```yaml
pipelines:
  denoise_then_segment:
    steps:
      - id: denoise
        method: Denoise
        inputs: {image: "$input.image", strength: 2}
      - id: segment
        method: Segment
        inputs: {image: "$denoise.result"}
```

```bash
uv run linkinpy pipeline list my_spec.yaml
uv run linkinpy pipeline run my_spec.yaml denoise_then_segment --output-dir out/ image=cells.tif
```

`$input.<name>` is a pipeline argument and `$<step id>.<output name>` is an earlier step's
output. Step outputs land in `out/<step id>/`. See [METHOD_AUTHORING.md](METHOD_AUTHORING.md).

### Environments and cleanup

```bash
uv run linkinpy envs list                 # name, Python version, size, stored spec
uv run linkinpy envs refresh stardist     # reinstall packages (e.g. after upgrading LinkinPy)
uv run linkinpy envs remove stardist
uv run linkinpy gc --dry-run              # preview; drop --dry-run to delete
```

`gc` removes `*.part` downloads, old `linkinpy-*` temp folders and, where the platform lists them
(Linux `/dev/shm`), orphaned shared-memory segments.

### Provenance

Runs that write outputs also write `linkinpy_run.json` next to them:

```json
{
  "callable": "Blur",
  "interface": "cli",
  "spec": {"display_name": "E2E Demo", "version": "1", "sha256": "6d8c63db..."},
  "environment": {"python": "3.12.13", "packages": {"linkinpy": "0.0.8"}},
  "arguments": {"gain": 4.0, "image": "/data/a.tif"},
  "outputs": [{"name": "result", "path": "out/result.tif", "sha256": "afba6106...", "bytes": 1234}],
  "started": "2026-10-16T12:00:00+00:00", "finished": "2026-10-16T12:00:02+00:00"
}
```

### Progress

A method reports progress with `linkinpy.progress(fraction, message)`. `linkinpy run` prints it
as `[ 50%] halfway` on a terminal and as a raw JSON line when piped (hosts read that form).

### Running on another machine

**1. On the host machine** (the GPU box) start the server. It needs only `linkinpy`; the option
you will normally set is the port:

```bash
uv run linkinpy serve --port 8765        # prints a bearer token; binds to localhost by default
```

**2. From your laptop**, tunnel the port, then connect, install and run. None of this needs the
tool's packages (TensorFlow, PyTorch...) on your laptop:

```bash
ssh -N -L 8765:localhost:8765 gpu-host &
export LINKINPY_REMOTE=http://localhost:8765
export LINKINPY_TOKEN=<token printed by serve>

uv run linkinpy remote list                              # what the host has installed
uv run linkinpy remote install StarDist.yaml             # install a Library spec ON THE HOST
uv run linkinpy remote install my_tool.yaml              # ...or a local file (uploaded) or an https URL

uv run linkinpy run ../Library/StarDist.yaml "Predict 2D Pretrained Model" \
  --remote http://localhost:8765 --output-dir results/ image=input.tif model_name=2D_versatile_fluo
```

Image inputs are uploaded and outputs are downloaded. The server only runs specs installed on
it; installing is done by the host, with a streamed log. For use beyond an SSH tunnel pass
`--certfile` and `--keyfile` for TLS; the token is the only access control, so treat it like a
password.

**napari, Fiji and QuPath** talk to the same server from a separate **Remote** submenu with
three entries: *Connect to host*, *Install on host* and *Run on host*. The run entry lists only
the specs and methods the host reports as installed, and builds its parameter dialog from the
host's description, so nothing has to be installed locally. See each plugin's README.

### Without `--output-dir`

If you give neither `--output-dir` nor `--output`, the CLI writes outputs to a temporary folder
and prints its path. (Image outputs cannot be returned over a plain pipe, and an unreleased
shared-memory segment would leak.)

### Environment variables

| Variable | Meaning |
| --- | --- |
| `LINKINPY_ENV_HOME` | Where specs and environments live (default `~/.linkinpy`) |
| `LINKINPY_REMOTE` | Default server URL for `run --remote` |
| `LINKINPY_TOKEN` | Bearer token for `serve`, `run --remote` and `remote ...` |

## Python API Tutorial

### The facade

```python
from linkinpy import LinkinPy

lp = LinkinPy("../Library/nanopyx.yaml")
print(lp.list_callables())
print(lp.validate())                       # [] when the spec has no errors or warnings
lp.install(interface="cli")
stdout = lp.run(
    "eSRRF",
    {"image": "input.tif", "magnification": 2},
    output_dir="outputs",
)
```

`run` returns the worker's output; turn it into result objects with `parse_run_result`:

```python
from linkinpy import parse_run_result

for item in parse_run_result(stdout):
    print(item.spec.name, item.path)       # e.g. "result outputs/result.tif"
```

### Batch, pipelines and streaming

```python
from linkinpy import LinkinPy, load_spec, parse_progress_line, run_spec_callable_in_environment

lp = LinkinPy("my_spec.yaml")

batch = lp.batch("Blur", input_dir="raw/", output_dir="results/", arguments={"gain": 2})
print(len(batch.ok), "ok,", len(batch.failed), "failed")        # BatchItem: input_path, status, error, outputs

pipeline = lp.pipeline("denoise_then_segment", {"image": "cells.tif"}, output_dir="out/")
print(pipeline.final["labels"].path)                             # outputs of the last step

def on_line(line):                                               # called for every output line, live
    progress = parse_progress_line(line)                         # (fraction | None, message) or None
    print(progress if progress else line)

spec = load_spec("my_spec.yaml")
run_spec_callable_in_environment(spec, "Blur", {"image": "a.tif"}, output_dir="out/", on_line=on_line)
```

### Remote runs

```python
from linkinpy import run_remote

outputs = run_remote(
    "http://localhost:8765", token, "stardist",           # server URL, token, installed spec name
    method="Predict 2D Pretrained Model",
    arguments={"image": "cells.tif", "model_name": "2D_versatile_fluo"},
    output_dir="results/",
    on_line=print,
)
print(outputs["labels"])                                  # local path of the downloaded output
```

`RemoteClient(url, token)` exposes the individual steps: `info()` (server version), `specs()`
(everything installed on the host, with full input/output descriptions), `install(...)`,
`upload()`, `submit()`, `wait()` and `download()`:

```python
from linkinpy import RemoteClient

client = RemoteClient("http://localhost:8765", token)
print(client.info())                                      # {'ok': True, 'linkinpy': '0.0.8'}
job = client.install(spec_name="StarDist.yaml", interface="cli")   # or spec_yaml=..., spec_url=...
client.wait(job, on_line=print)                           # streams the install log
print([spec["name"] for spec in client.specs()])
```

### Results in memory (display-only methods)

```python
from linkinpy import load_spec, run_for_display

spec = load_spec("my_spec.yaml")                      # a method with output: {napari: [display]}
results = run_for_display(spec, "Segment", {"image": image_array}, interface="napari")
labels = results["labels"].value                      # NumPy array, copied out of shared memory
print(spec.callables["Segment"].actions_for("napari"))   # (display, io) == (True, False)
```

Nothing is written to disk; only outputs with `display: true` are returned. See
[METHOD_AUTHORING.md](METHOD_AUTHORING.md#display-io-or-both) and the host protocol in
[CONTRIBUTING.md](CONTRIBUTING.md#host-protocol-for-display-only-runs).

### Reporting progress from a method

```python
from linkinpy import progress

def run(image, n_tiles: int = 8):
    for i in range(n_tiles):
        ...
        progress((i + 1) / n_tiles, f"tile {i + 1}/{n_tiles}")
```

### Errors

A failure inside the spec's environment (bad argument, exception in the method) raises
`LinkinPyEnvironmentError` carrying the worker's message, e.g.
`... RunnerError: Input 'gain' must be <= 10, got 99.0.`. Calling a method in-process
(`run_spec_callable(..., ensure_environment=False)`) raises `RunnerError` directly. Spec problems
raise `SpecError`.

### Lower-level APIs

- `load_spec` / `parse_spec`, `validate_spec` (returns `Issue` objects with `level`, `where`, `message`)
- `EnvironmentManager`, `list_environments`, `refresh_environment`, `remove_environment`, `collect_garbage`
- `run_spec_callable`, `run_spec_callable_in_environment`, `run_for_display`, `parse_run_result`
- `run_batch`, `run_pipeline`, `run_remote`, `RemoteClient`
- `read_image` / `write_image`, `create_shared_array` / `attach_shared_array`
- `progress`, `parse_progress_line`

## Colab Tutorial

Install LinkinPy in a notebook:

```python
!pip install linkinpy
```

Mount Google Drive:

```python
from linkinpy import mount_google_drive

mount_google_drive()
```

Install a selected spec:

```python
from linkinpy import install_colab_spec

install_colab_spec("/content/LinkinPy/Library")
```

Run a method from installed specs:

```python
from linkinpy import run_installed_colab_method

run_installed_colab_method(
    output_dir="/content/drive/MyDrive/LinkinPy/outputs"
)
```

Colab displays image and labels outputs when possible and prints metadata/path outputs in
the notebook output area.

## Spec Contract

Specs are YAML files. Each method declares `inputs`, `outputs`, and `output`.

```yaml
display_name: Example Package
python_version: "3.12"
how_to_cite: "Author A, Author B. Example Package. Journal, year."

install:
  python_packages:
    - name: example-package

segment:
  callable: example_package.segment
  inputs:
    - name: image
      display_name: Input Image
      type: np.ndarray
      role: image
      required: true
  outputs:
    - name: labels
      display_name: Label Image
      type: np.ndarray
      role: labels
      format: tif
      display: true
  output:
    cli: [io]
    colab: [display, io]
    napari: [display, io]
    imagej: [display, io]
    qupath: [display, io]
```

`outputs` is the return-value contract. Tuple/list returns map by output order. Dict
returns map by output name.

`output:` says, per interface, what happens to the results: `[display]` shows them in the host and writes
**nothing to disk** (images through shared memory, tables and shapes inline, no output folder), `[io]` saves
them, `[display, io]` does both. If shared memory cannot be used, display-only runs fall back to temporary
files that are deleted after display.

Output `role` tells hosts how to present a result: `image`, `labels`, `scalar`, `metadata`, `text`,
`path`, `folder`, plus `table` (CSV), `points` (CSV of `y,x[,...]` rows) and `geojson` (GeoJSON
FeatureCollection). A spec can also declare `pipelines:`, and a method can embed its code with
`code:` instead of a package. All of this, with examples, is in
[METHOD_AUTHORING.md](METHOD_AUTHORING.md).

## More Documentation

- [GETTING_STARTED.md](GETTING_STARTED.md) for contributor setup
- [METHOD_AUTHORING.md](METHOD_AUTHORING.md) for package authors
- [CONTRIBUTING.md](CONTRIBUTING.md) for the contribution workflow
- [CHANGELOG.md](CHANGELOG.md) for what changed
- [Repository tutorials](../USAGE_TUTORIALS.md) for all interfaces, with examples
