Metadata-Version: 2.4
Name: zyexp
Version: 0.1.0
Summary: Local-first experiment records for humans and coding agents
Keywords: experiment-tracking,machine-learning,reproducibility
Author: frostime
License-Expression: AGPL-3.0-or-later
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: jsonschema>=4.23,<5
Requires-Dist: scikit-learn>=1.4,<2 ; extra == 'sklearn'
Requires-Dist: torch>=2.1,<3 ; extra == 'torch'
Requires-Dist: torchvision>=0.16,<1 ; extra == 'torch'
Requires-Python: >=3.10, <4
Project-URL: Repository, https://bitbucket.org/aeonsfrostime/zyplib/src/main/packages/zyexp/
Project-URL: Issues, https://bitbucket.org/aeonsfrostime/zyplib/issues
Provides-Extra: sklearn
Provides-Extra: torch
Description-Content-Type: text/markdown

# zyexp

English · [简体中文](README.zh-CN.md)

**A lightweight, local experiment record manager for coding agents.**

Let your agent write and run experiment code in the way the task requires. zyexp gives that code a consistent way to organize runs and save parameters, metrics, checkpoints, and results. A later analysis can read those records without having to interpret a new collection of ad hoc logs.

The package includes an **Agent Skill** that explains how to structure experiments, record observations, validate outputs, and read results. Your agent writes the recording calls into the script; the script saves observations as the experiment runs. zyexp does not execute experiments or impose a training loop.

Python 3.10+. The only direct runtime dependency is `jsonschema`; the base installation requires neither PyTorch, scikit-learn, NumPy, nor zyplib. Records stay in local files, with no tracking server or account required.

## Get started with your agent

Install zyexp in the Python environment your agent uses for the experiment:

```bash
pip install zyexp
```

Then ask the agent to read the bundled guide:

```bash
zyexp skill
```

This prints the Skill and the installed paths to its reference files. An agent with shell and file access can read them as needed. No separate Skill download is required.

For example, give your agent this task:

> Compare logistic regression and random forest on Iris using the same data split. First run `zyexp skill` and follow its guidance. Write and execute the experiment, recording the parameters, data split identity, metrics, and saved models with zyexp. Validate the Run, then compare the models by reading the saved results. Report the Run directory and the evidence behind your comparison.

The agent chooses how to load data, build models, and run the comparison. It uses zyexp to save the facts needed to inspect the experiment later. Repeats, cross-validation, and parameter searches remain ordinary code, not a separate scheduling language.

Use the libraries your experiment needs. Optional installation shortcuts are available:

```bash
pip install 'zyexp[sklearn]'  # scikit-learn
pip install 'zyexp[torch]'    # PyTorch and torchvision
```

These extras are not needed to read or validate saved results.

## What an experiment leaves behind

A comparison might use one Run for the execution and one Trial per model:

```text
experiments/classifier-comparison/runs/run-001/
├── state.json
├── records.jsonl
├── report.json
└── models/
    ├── state.json
    ├── records.jsonl
    ├── logistic-regression/
    │   ├── state.json
    │   ├── records.jsonl
    │   └── checkpoints/
    └── random-forest/
        ├── state.json
        ├── records.jsonl
        └── checkpoints/
```

A **Run** is one execution. A **Trial** is a unit with its own metrics, checkpoints, result, and failure status. A **Group**, such as `models` above, organizes related Trials and may contain other Groups. Epoch and batch positions are metric coordinates inside a Trial, not directories.

The experiment script can record:

| Fact | What a later analysis can establish |
|---|---|
| Parameters and runtime environment | Which settings and software were used |
| Dataset descriptions, split seeds, or content hashes | Which data each Trial used |
| Raw metrics with coordinates | How the recorded observations changed over epochs, batches, or other steps |
| Top-K checkpoints and evaluated-from lineage | Which saved model produced the declared final result |
| Objectives, summaries, and artifacts | What was measured and which supporting files are available |

These facts must be submitted by the experiment code; zyexp does not infer data splits or scientific conclusions. Normal context exit marks a node completed. An escaping exception marks it failed and propagates to the caller.

## Read and analyze results

After the experiment exits, the same agent or a new session can inspect its Run without rerunning training:

```bash
zyexp inspect <RUN_DIR>
zyexp records <RUN_DIR> --node models/logistic-regression --type metric
zyexp validate <RUN_DIR> --verify-artifacts
```

`inspect` prints the report tree, including final results and checkpoint metadata. `records` reads the observations. Validation checks structure and referenced files; `--verify-artifacts` also checks artifact and checkpoint hashes. It does not judge experimental design or the agent's interpretation.

Python readers work on the saved files too:

```python
from zyexp import iter_metrics, load_run

run_dir = 'experiments/classifier-comparison/runs/run-001'
report = load_run(run_dir)
models = report['run']['children'][0]
for trial in models['children']:
    print(trial['name'], trial['result']['objectives'])

for observation in iter_metrics(run_dir, node_path='models/logistic-regression'):
    print(observation['name'], observation['coordinates'], observation['value'])
```

`state.json` and `report.json` hold results; `records.jsonl` holds submitted observations. Live metric recorders provide within-Trial aggregate views. After process exit, read raw metrics to reconstruct the views needed for analysis. The agent computes cross-Trial comparisons and may record its conclusions explicitly.

## How scripts integrate

Recording calls sit beside the experiment's own execution code. This runnable example uses illustrative values, not a trained model:

```python
from zyexp import open_run

with open_run(root='experiments', experiment='recording-demo', run_name='run-001') as run:
    run.record_hyper_params({'learning_rate': 0.01})
    with run.trial('training') as trial:
        metrics = trial.metrics(levels=('epoch',))
        metrics.define('validation/loss')
        for epoch, loss in enumerate([0.8, 0.5, 0.3]):
            # In an experiment, use the value computed by your evaluation code.
            metrics.record('validation/loss', loss, epoch=epoch)
        trial.record_result(objectives={'validation/loss': loss})

print(run.run_dir)
```

Choose a new `run_name` for another execution. Training frameworks, optimization order, and checkpoint serialization/loading stay in caller code. zyexp owns the paths and retention policy for managed Top-K checkpoints.

Current limits:

- One writer per Run; concurrent or distributed writing is not supported.
- An existing Run cannot be reopened for continued writing after a process restart.
- Each metric name has one coordinate depth. Compute non-decomposable metrics such as F1 or AUROC over the appropriate data; do not average batch scores.
- `zyexp repair <RUN_DIR>` marks abandoned running nodes interrupted and rebuilds the report. It does not resume execution or recover missing observations.

## Guides and examples

- `zyexp skill` — installed Agent guide with paths to references for recording, checkpoints, profiling, and inspection.
- `zyexp sourcecode` — absolute path to the installed Python source, for checking an API question against the version your experiment uses. The Skill includes a source map.
- [Python API guide](docs/python-api.md) — API examples for metrics, Top-K, artifacts, relations, framework-owned loops, and data profiling.
- [sklearn Iris example](example/sklearn-iris/README.md) — fit, evaluate, save a model, and record its lineage.
- [PyTorch MNIST example](example/torch-mnist/README.md) — caller-owned training loop, batch/epoch metrics, and Top-K checkpoints.

Both READMEs, the Python API guide, and example scripts are included in the source distribution. The installed wheel includes the Agent Skill and its references.

License: [AGPL-3.0-or-later](LICENSE).
