Metadata-Version: 2.4
Name: multimodal-mllog
Version: 1.2.1
Summary: Experiment logbook pipeline: automatic per-run capture + on-demand reporting
License: MIT
Keywords: experiment-tracking,logbook,coding-agents,mlflow
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: MIT License
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: pydantic>=2
Requires-Dist: click
Provides-Extra: mlflow
Requires-Dist: mlflow>=2; extra == "mlflow"
Provides-Extra: dashboard
Requires-Dist: django>=4.2; extra == "dashboard"
Requires-Dist: djangorestframework; extra == "dashboard"
Requires-Dist: django-cors-headers; extra == "dashboard"

# multimodal-mllog

[![Python](https://img.shields.io/pypi/pyversions/multimodal-mllog)](https://pypi.org/project/multimodal-mllog/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Experiment logbook pipeline for researchers training ML models with coding agents.

An external observer that **automatically captures** per-run facts (agent events, git state, optional MLflow info) as JSON records via a sensor/adapter/sink architecture, and **on demand** renders a markdown logbook from them.

```
coding agent  -->  mllog capture (automatic via hook)  -->  local JSON store
researcher    -->  /logbook (manual)                    -->  markdown logbook
```

## Architecture

```
sensors (data producers) --> core (deterministic) --> sinks (optional side effects)

  Sensors:   agent (always), git, env, mlflow (optional)
  Adapters:  Claude Code JSONL --> AgentEvent normal form
  Core:      validate, store (fsync), query, project to markdown
  Sinks:     mlflow_attach (runs after write+fsync, failures warn)
```

Record envelope (`./mllog/records/<YYYY-MM-DD>/<record_id>.json`):

```jsonc
{
  "schema_version": "0.1",
  "record_id": "<ULID>",
  "activity_type": "train | eval | analysis | attempt_failed",
  "status": "ok | failed",
  "started_at": "<iso8601>",
  "ended_at": "<iso8601>",
  "comparison_safety": { "code_touched": [], "config_touched": [], "safe_for_delta": true },
  "sources": {
    "agent":  { "event_count": 12, "edit_ledger": {}, "config_deltas": [], "commands": [] },
    "git":    { "commit": "abc123", "dirty": false, "changed_files": [] },
    "mlflow": { "run_id": "...", "params": {}, "metrics": {} },  // present only if active
    "env":    { "python_version": "3.13", "platform": "..." }
  },
  "digest": null,           // optional, evidence-gated
  "transcript_ref": null    // pointer to session JSONL, never raw content
}
```

## Installation

```bash
pip install multimodal-mllog              # core (capture, query, render)
pip install "multimodal-mllog[mlflow]"    # + optional MLflow sensor/sink
```

## Quickstart

1. Open your ML project in a coding agent (Claude Code, Codex, etc.).
2. Run your experiment: train, evaluate, or analyze.
3. The `Stop` hook captures the run automatically, or run `/mllog` manually.
4. Run `/logbook --from yesterday` to generate a logbook from stored records.

## CLI reference

```bash
# Capture a run (sensors gather git, agent events, optional MLflow automatically)
mllog capture --type {train|eval|analysis|attempt_failed} --status {ok|failed} \
              [--session-path <path>] [--mlflow-run-id <id>] [--auto]

# Query / render records
mllog get-logs --from <when> [--to <when>] [--json]
mllog get-logs --from <when> [--to <when>] --render [--out <path>]
# <when>: ISO date (2026-07-01), 'yesterday', 'today'

# Sensor / adapter / sink status
mllog doctor

# Checkpoint
mllog checkpoint [--show | --advance]
```

## Storage

- Records: `./mllog/records/<YYYY-MM-DD>/<record_id>.json` (one per run, fsynced)
- Checkpoint: `./mllog/checkpoint.json`
- Logbooks: `./mllog/logbooks/<from>_<to>.md`
- Override root with `MLLOG_DIR` env var.

## Development

```bash
git clone https://github.com/jean-johnson-zwix/multimodal_mllogger
cd multimodal_mllogger
make install         # create venv + install deps
make test            # run pytest
make lint            # check core purity (no banned imports)
```

## Module layout

```
src/mllog/
  cli.py                    # entrypoint
  core/
    schema.py               # Record, ComparisonSafety, Digest, TranscriptRef
    store.py                # date-partitioned JSON, fsync, time-window queries
    project.py              # Record[] -> markdown (pure, no I/O)
    events.py               # AgentEvent discriminated union
    extract/                # pure functions over list[AgentEvent]
      edits.py  config.py  commands.py  safety.py
  sensors/                  # data producers (entry points)
    agent.py  git.py  mlflow.py  env.py
  adapters/                 # vendor JSONL -> AgentEvent
    claude_code.py  codex.py
  sinks/                    # post-write side effects (entry points)
    mlflow_attach.py
```

## Roadmap

- **v1.0:** MLflow-centric pipeline with `/mllog` command.
- **v1.1:** Local JSON store, automatic capture via hooks, `/logbook` reporting.
- **v1.2 (current):** Sensor/adapter/sink architecture, AgentEvent normal form,
  deterministic extractors, comparison safety, evidence-gated digests, entry points.

## License

MIT
