Metadata-Version: 2.4
Name: robo-watson
Version: 0.1.0
Summary: Robotics logging, bag storage and experiment retrieval
Author-email: Lorenzo Magnino <lorenzo.magnino@gmail.com>, Antonio Terpin <aterpin@ethz.ch>, Francesco Banelli <fbanelli@ethz.ch>
Maintainer-email: Lorenzo Magnino <lorenzo.magnino@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/lorenzomagnino/watson
Project-URL: Repository, https://github.com/lorenzomagnino/watson
Project-URL: Documentation, https://github.com/lorenzomagnino/watson#readme
Project-URL: Issues, https://github.com/lorenzomagnino/watson/issues
Keywords: robotics,logging,observability,jax,wandb,experiment-tracking
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Logging
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ruamel.yaml
Requires-Dist: rich
Requires-Dist: typing-extensions>=4.15.0
Requires-Dist: numpy>=1.23
Requires-Dist: imageio>=2.37.0
Requires-Dist: pillow>=12.3.0
Requires-Dist: matplotlib>=3.10.7
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.0.0; extra == "dev"
Requires-Dist: hypothesis>=6.0.0; extra == "dev"
Requires-Dist: snowballstemmer==2.2.0; extra == "dev"
Requires-Dist: pre_commit==4.0.1; extra == "dev"
Requires-Dist: tomli; extra == "dev"
Requires-Dist: pyupgrade>=3.19.0; extra == "dev"
Requires-Dist: psutil; extra == "dev"
Requires-Dist: hydra-core>=1.3.2; extra == "dev"
Requires-Dist: basedpyright; extra == "dev"
Requires-Dist: pydoclint>=0.8.3; extra == "dev"
Provides-Extra: jax
Requires-Dist: jax>=0.4.3; extra == "jax"
Requires-Dist: jaxlib>=0.4.3; extra == "jax"
Provides-Extra: wandb
Requires-Dist: wandb[media]; extra == "wandb"
Requires-Dist: plotly>=5.20.0; extra == "wandb"
Dynamic: license-file

# Watson

Robotics logging, bag storage, and experiment retrieval. Watson records
and retrieves experiments; [robosim-prorokLab](https://github.com/lorenzomagnino/robosim-prorokLab)
plots and renders their recordings.

Watson builds on [Goggles](https://github.com/antonioterpin/goggles), preserving
its implementation, tests, MIT license, and Git history at upstream commit
`5b850d3d89b3b7cebb364f89722ff0fb9f1684b8`. See [GOGGLES.md](GOGGLES.md)
for the complete inherited API and examples.

## Install

Python 3.10 or newer:

```sh
pip install robo-watson
pip install 'robo-watson[wandb]'
pip install 'robo-watson[jax]'
```

These commands become available after the first PyPI release. Until then,
repository collaborators can clone the repository and run `pip install -e .`.

The distribution is `robo-watson`; the public import is `watson`. It also ships
`goggles` for compatibility. Use this distribution in place of `robo-goggles`;
installing both in the same environment would give them ownership of the same
package files. The GitHub repository is private; distributions uploaded to
PyPI are public and include the Python source code.

## Log experiments

```python
import watson as wt

wt.attach(wt.ConsoleHandler(name="navigation.console"))
logger = wt.get_logger("navigation", with_metrics=True)
logger.info("Experiment started")
logger.scalar("tracking_error", 0.12, step=1)
wt.finish()
```

All of Goggles' public top-level exports are available through `watson`:
console and JSONL logging, scoped routing, multiprocessing and shared-memory
transport, W&B metrics and media, grouped runs and tags, artifacts, timing,
exception tracing, YAML configuration, filtering, and graceful shutdown.
Specialized modules retain their existing imports: `goggles.history`,
`goggles.media`, `goggles.config`, `goggles.types`, and `goggles.filters`.
JAX histories require the `jax` extra. W&B features require the `wandb` extra.

Choose a separate logging socket for each application before importing Watson:

```sh
export GOGGLES_SOCKET=/tmp/watson-navigation.sock
```

Related worker processes must share that socket. Existing `GOGGLES_*`
environment settings continue to apply.

## Save and find bags

Save a completed recording file (`.bag`, `.db3`, `.mcap`, etc.) or an entire
recording directory. For run folders containing a `rosbag/` subdirectory,
use `--bag-path rosbag`; copying the full run preserves rollout sidecars.

```sh
watson save /data/flight --bag-path rosbag --name navigation \
  --tag baseline --robosim /path/to/robosim-prorokLab \
  --wandb-run team/navigation/run_id
watson list --name navigation --tag baseline
watson show BAG_ID
watson path BAG_ID
watson verify BAG_ID
```

Storage defaults to `~/.watson/bags`. Set `WATSON_BAGS` or use
`watson --root /data/bags ...` to choose another store.

```python
from watson import Bag, BagStore

store = BagStore("/data/saved-bags")
bag = store.save(
    "/data/flight", bag_path="rosbag", name="navigation",
    tags=("baseline",), robosim="/path/to/robosim-prorokLab",
    wandb_run="team/navigation/run_id",
)
bag = store.get(bag.id)
bag.verify()
print(bag.path)
print(store.list(name="navigation", tag="baseline"))
```

Each save has a unique ID and a portable `manifest.json` containing the
original source path, relative bag path, name, tags, creation time, W&B run
reference, file hashes, and optional RoboSim repository path, origin URL,
commit, and dirty state. `Bag.load(directory)` reads an exported recording.
The relative payload path still works after transferring the recording to
another machine. Repository provenance is a reference; it does not copy a
checkout or capture uncommitted changes.

Saves copy data, reject symlinks, and become visible only after the copy and
manifest succeed. Sources are never removed. Save only after recording has
stopped: a live recording cannot be copied as a consistent snapshot. Hashes
check accidental corruption; they do not authenticate the artifact publisher.

## W&B organization and retrieval

Use project, group, and tags on `WandBHandler` as described in
[GOGGLES.md](GOGGLES.md#experiment-tracking-with-wb). To attach a bag to an
existing SDK run:

```python
import wandb
from watson import BagStore
from watson.tracking import fetch_bag, find_runs, upload_bag

store = BagStore()
with wandb.init(project="navigation", group="baseline", tags=["flight"]) as run:
    bag = store.save("/data/flight", bag_path="rosbag", wandb_run=run.path)
    artifact = upload_bag(bag, run)

for run in find_runs("team/navigation", group="baseline", tag="flight"):
    for artifact in run.logged_artifacts():
        if artifact.type == "bag":
            print(artifact.qualified_name)

bag = fetch_bag("team/navigation/bag-BAG_ID:v0", BagStore("/data/retrieved"))
```

`upload_bag` verifies files and attaches the manifest and payload to the
caller's run. It does not start or finish a run. For Goggles-managed W&B runs,
use the inherited logger's `artifact` method with `path=str(bag.directory)`,
`name=f"bag-{bag.id}"`, and `type="bag"` in its payload. Keep the directory
available until `wt.finish()` completes the asynchronous upload.

```sh
wandb login
watson --root /data/retrieved fetch team/navigation/bag-BAG_ID:v0
```

Fetch validates payload paths and checks hashes before admitting a downloaded
recording to the store. Existing IDs are never overwritten. Choose a version
such as `:v0` for reproducible retrieval; `:latest` follows the latest upload.
W&B project access and visibility follow your W&B account settings, independently
of this repository's GitHub privacy. Uploads include source and repository paths
from the manifest; review those before sharing an artifact.

## Render with RoboSim

Watson returns a real local bag path. From the RoboSim checkout:

```sh
python -m robosim.bag "$(watson path BAG_ID)"
python -m robosim.build "bag.path=$(watson path BAG_ID)"
```

The first command decodes a summary; the second uses RoboSim's rendering
pipeline and configuration. RoboSim's multi-flight viewer discovers recordings
under its own `bags/` directory; Watson does not change that checkout or install
its rendering dependencies. Bag format support is determined by RoboSim.

## Development

```sh
uv sync --extra dev --extra wandb --extra jax
uv run pytest
uv run pre-commit run --all-files
uv run pre-commit run --all-files --hook-stage pre-push
uv build
```

Watson additions live in small modules under `watson/`: bag manifests,
local storage, repository provenance, optional W&B tracking, and the CLI.
New code has no inline comments. The inherited `goggles/` implementation is
kept intact to retain its features and make future upstream updates practical.
The `upstream` remote tracks Goggles. See [Publishing](docs/publishing.md)
for the one-time PyPI setup and version-tag release process.

CodeQL analysis for this private repository is saved as a GitHub Actions
artifact because GitHub code-scanning uploads are not enabled here.
