Metadata-Version: 2.5
Name: driftless-bundle-photometry
Version: 0.2.1
Summary: Trace-first multichannel fiber photometry acquisition and NWB recording
Project-URL: Homepage, https://github.com/mohebi-n-associates/DriftlessBundlePhotometry
Project-URL: Repository, https://github.com/mohebi-n-associates/DriftlessBundlePhotometry
Project-URL: Issues, https://github.com/mohebi-n-associates/DriftlessBundlePhotometry/issues
Project-URL: Changelog, https://github.com/mohebi-n-associates/DriftlessBundlePhotometry/blob/main/WHATS_NEW.md
Author: Driftless Bundle Photometry contributors
License: MIT
License-File: LICENSE
Keywords: NWB,acquisition,fiber photometry,neuroscience
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Win32 (MS Windows)
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Requires-Python: <3.14,>=3.11
Requires-Dist: h5py<4,>=3.12
Requires-Dist: ndx-fiber-photometry<0.3,>=0.2.3
Requires-Dist: numpy<3,>=2.0
Requires-Dist: nwbinspector<1,>=0.6
Requires-Dist: platformdirs<5,>=4.3
Requires-Dist: pydantic<3,>=2.10
Requires-Dist: pynwb<5,>=4.0
Requires-Dist: pyserial<4,>=3.5
Provides-Extra: dev
Requires-Dist: pyqtgraph<0.15,>=0.13.7; extra == 'dev'
Requires-Dist: pyside6<6.9,>=6.8; extra == 'dev'
Requires-Dist: pytest-cov<7,>=6; extra == 'dev'
Requires-Dist: pytest-qt<5,>=4.4; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.11; extra == 'dev'
Provides-Extra: gui
Requires-Dist: pyqtgraph<0.15,>=0.13.7; extra == 'gui'
Requires-Dist: pyside6<6.9,>=6.8; extra == 'gui'
Provides-Extra: release
Requires-Dist: build<2,>=1.2; extra == 'release'
Requires-Dist: pillow<13,>=11; extra == 'release'
Requires-Dist: pyinstaller<7,>=6.16; extra == 'release'
Requires-Dist: twine<8,>=7; extra == 'release'
Description-Content-Type: text/markdown

# Driftless Bundle Photometry

**Version 0.2.1** · [What's new](WHATS_NEW.md)

A Python-based, trace-first multichannel fiber-photometry acquisition system.

The application supports up to nine circular fiber ROIs, interleaved 405/470/565 nm
excitation, four edge-recording TTL inputs, optional lossless raw frames, and one
self-contained NWB-HDF5 output per ROI and animal. Windows is the physical-hardware
target; the simulator and GUI demo are cross-platform.

## Current implementation

- Immutable validation for sessions, channels, circular ROIs, camera settings, and
  TTL inputs.
- Deterministic simulator and a shared headless acquisition engine with explicit
  lifecycle states.
- Bounded, checksummed, chunked session spools that survive interruption and disk or
  finalizer errors.
- Pre-arm storage-capacity checks and structured live diagnostics for queue pressure,
  write latency, dropped frames, clock residuals, and NWB finalization stages.
- Typed acquisition faults are committed as system events and invalid-time markers
  before an incomplete spool closes whenever committed samples can be recovered.
- Canonical NWB finalization using PyNWB, `ndx-fiber-photometry`, and
  `ndx-ophys-devices`, including a separate subject, brain region, sensor type, and
  validated file for every ROI.
- Exact per-exposure commanded voltage, controller sequence/tick, camera frame ID,
  host receipt time, per-ROI saturation QC, explicit excitation events, and marked
  invalid continuous spans in every independently interpretable animal file.
- Trace-first PySide6/pyqtgraph desktop demo with a separate calibration image and
  draggable circular fiber ROIs, plus separate live camera panels for 405, 470, and
  565 nm exposures.
- Selectable 15 s, 1 min, 10 min, 1 h, and Full trace horizons with display-only
  extrema-preserving decimation, keeping long recordings responsive without changing
  acquired or stored samples.
- One live plot row per ROI with overlaid 405, 470, and 565 nm traces, independent
  wavelength visibility controls, and Absolute or display-only dF/F views with a
  configurable per-ROI/per-wavelength median baseline window.
- Optional display-only trailing-mean smoothing uses elapsed seconds, works with
  asynchronously sampled channels, and never changes acquired or stored values.
- Independent enable/disable controls for each ROI; disabled definitions remain in
  saved settings but are excluded from extraction, live traces, and NWB output.
- A required non-writing acquisition preview that validates timing, explicit
  wavelength coverage, commanded voltages, camera frames, ROI extraction, and
  saturation before enabling each recording.
- Synchronized numeric time axes across all ROI rows, horizontal/vertical drag-box
  zoom, double-click zoom reset, and page scrolling—not graph zoom—under the mouse
  wheel.
- Clean numeric entry fields without embedded increment/decrement stepper buttons.
- Versioned, human-readable JSON settings with GUI actions to save, load, set startup
  defaults, and restore the complete setup from a previous DBF NWB recording.
- Settings v2 discriminate the native camera/controller source from the planned RWD
  read-only bridge and migrate v1 files explicitly to native with a visible warning.
- A raw uint16 reference frame and fixed-scale ROI-annotated diagnostic view for
  every wavelength observed in a recording are saved inside each ROI NWB file.
- Versioned/checksummed controller protocol, fail-safe lifecycle guard, and
  single-owner serial transport that sends a best-effort `STOP` on close.
- Lazy Thorlabs SDK detection that does not break simulator mode on macOS.

The CS505MU camera transport, serial controller transport, controller firmware, and
LED intensity electronics are not yet physically validated. Those require the real
Windows rig, vendor SDK, trigger cabling, DAC/driver hardware, and bench timing and
safety measurements. Simulator success is not evidence of physical validation.

The RWD bridge currently has a documented wire contract, validated source settings,
typed raw domain records, a fixture-tested incremental fluorescence/event parser,
and a synthetic-data trace-only recovery/NWB path. The RWD spool retains checksummed
mixed records, exact raw ticks/values and scales, named ON/OFF events, connection
identity, faults, and invalid spans. Recovery creates one independently validated NWB
per mapped fiber without inventing camera frames, LED commands, native exposures, or
numbered TTL inputs. A headless single-owner TCP client now provides bounded
recording, timeout/disconnect/malformed-stream faults, prompt stop, storage preflight,
and machine-readable CLI output. The GUI workflow is still being implemented, and
the desktop setup now begins with a native/simulator or RWD read-only system choice,
hides camera/LED controls in RWD mode, retains per-fiber animal metadata, and routes
both sources through the same bounded live-trace presentation contract. Every RWD
fiber has an explicit device-channel number and an independent operator label. Live
status reports connection state, decoder counts, spool pressure, committed records,
and named event activity; raw, elapsed-time-smoothed, and dF/F views remain
presentation-only.
The TCP path has only synthetic local-server validation; current builds must not be
described as a physically validated RWD connection. See
[the RWD streaming contract](docs/rwd-streaming-protocol.md) and
[the RWD operator and bench-validation guide](docs/rwd-operator-guide.md).

The historical MATLAB, Arduino, and Bonsai sources remain under `old mescoscope/`
for reference and are not runtime dependencies.

## Saving and restoring settings

The Configuration panel provides **Save JSON**, **Load JSON**, **Load NWB**, and
**Set as default**. A settings file includes all session metadata, recording duration,
output location, native/RWD source selection, source-specific connection settings,
camera and retention settings, excitation states and voltages, TTL configuration,
live-display preferences, and every fiber's animal metadata. Settings v1 files are
accepted only through an explicit migration to the native source because they
predate system selection.

On Windows, DBF resolves the same Documents folder used by Explorer, including
OneDrive or other redirected locations, and uses its `Driftless Bundle Photometry`
folder for settings. **Set as default** writes `default_settings.json` there; DBF
loads that file automatically at startup. Save and Load dialogs open in the same
folder by default.

Every newly recorded per-ROI NWB file embeds the complete settings snapshot, so any
one of those files can restore the whole multi-ROI setup. DBF can also import older
per-ROI DBF NWBs: it gathers matching sibling files to recover all available ROIs and
warns when an old file never recorded a setting that must use a safe default.
Each file also embeds a separate runtime provenance snapshot with the application,
Python, operating-system, dependency, adapter, and protocol versions that governed
acquisition or recovery.

## Installation choices

End users on 64-bit Windows should use the standalone installer. Developers should
use the repository's Conda environment. PyPI is also available for users who already
manage Python environments. The standalone installer includes Python and all normal
application dependencies; it does not require Conda or a separate Python install.

### Windows installer for end users

1. Open the [latest GitHub release](https://github.com/mohebi-n-associates/DriftlessBundlePhotometry/releases/latest).
2. Download `Driftless-Bundle-Photometry-<version>-Windows-x64-Setup.exe`.
3. Optionally compare its SHA-256 digest with `SHA256SUMS.txt` from the same release.
4. Run the installer, then launch **Driftless Bundle Photometry** from the Start menu.

The installer is per-user, supports clean uninstall through Windows Settings, and
does not require administrator access. Current installers are not code-signed, so
Windows SmartScreen may show an unrecognized-publisher warning. The application can
run the simulator and RWD read-only path, but the physical Thorlabs/controller path
must not be treated as validated until the documented Windows bench gates pass.

### Conda environment for developers

Clone the repository and let `environment.yml` create the `dbf` environment plus an
editable install with GUI, test, lint, build, and release tools:

```bash
git clone https://github.com/mohebi-n-associates/DriftlessBundlePhotometry.git
cd DriftlessBundlePhotometry
conda env create --file environment.yml
conda activate dbf
python -m pytest
```

After pulling dependency changes, synchronize the existing environment with:

```bash
conda env update --file environment.yml --prune
```

### PyPI installation

Inside an existing Python 3.11–3.13 environment, install the desktop application
from PyPI and launch it with:

```bash
python -m pip install "driftless-bundle-photometry[gui]"
dbf --version
dbf --demo
```

The distribution name is `driftless-bundle-photometry`, the short command and Conda
environment are `dbf`, and Python imports use `driftless_photometry`. The long
`driftless-photometry` command remains available. On macOS, use Conda or PyPI for the
simulator, replay, NWB, analysis, and GUI-demo paths.

## Running developer workflows

Run a headless simulated acquisition:

```bash
dbf --headless --duration 5 --raw
```

Run the headless read-only RWD bridge from a validated settings-v2 file:

```bash
dbf --rwd-settings path/to/rwd.settings.json --duration 60
```

RWD must already be recording and listening on the host/port stored in that file.
DBF does not send hardware commands. It makes one connection for one session and
never reconnects automatically after a fault, because joining records across TCP
connections could hide a gap or changed device state.

Capture a checksummed exact-wire artifact during a physical bench run, inspect it,
and replay it unpaced through the production parser/recovery/NWB path:

```bash
dbf --rwd-settings path/to/rwd.settings.json --duration 300 \
    --rwd-wire-capture path/to/bench-001.rwd-wire
dbf --inspect-rwd-capture path/to/bench-001.rwd-wire
dbf --rwd-settings path/to/replay.settings.json \
    --rwd-replay path/to/bench-001.rwd-wire
```

Replay settings must preserve the capture's parsing scales, wavelengths, expected
machine, and channel/fiber mappings, but should use a new session ID and empty output
directory. Exact-wire captures can contain experimental data and must not be
committed to the repository.

Run the GUI demo:

```bash
dbf --demo
```

Recover all committed data from a complete or interrupted spool:

```bash
dbf --recover-spool path/to/session.photometry-spool
dbf --recover-spool path/to/session.rwd-spool
```

Inspect direct child spools without creating or removing files:

```bash
dbf --inspect-spools path/to/output-directory
```

Inspection recognizes native `*.photometry-spool` and RWD `*.rwd-spool` children,
verifies checksums, and reports source-specific counts plus completeness, invalid
times, system events, schema, and output-file count. It deliberately does not recurse
into unrelated directories. Successful recovery validates the complete
NWB set before removing the spool; add `--keep-spool` for forensic retention.
Recovery is idempotent and can resume if a process stopped after validating partials
or while promoting the multi-ROI set. Existing canonical files must match the spool
exactly and are never overwritten silently. Incomplete acquisitions are explicitly
marked in the NWB system event table. Legacy schema-v1 spools remain recoverable, but
the resulting invalid-time record identifies the per-frame host receipt and commanded
intensity provenance that those older spools never persisted.

The storage estimate uses configured duration, total frame rate, image dimensions,
raw-retention policy, enabled ROI count, and a 64 MiB reserve. CI correctness tests
exercise non-realtime simulator sessions at 300 frames/s trace-only and 60 frames/s
with raw 32 × 24 frames for one ROI. These small-fixture bounds test queue/data
integrity; they are not throughput claims for the CS505MU or any physical rig.

Run the release gates:

```bash
python -m pytest --cov=driftless_photometry --cov-report=term-missing
python -m ruff check .
python -m ruff format --check .
python -m build
python -m twine check --strict dist/*
```

The complete release procedure and one-time Trusted Publisher setup are in
[the release guide](docs/releasing.md).

See [the architecture](docs/architecture.md) and the synchronized agent contracts
in `AGENTS.md` and `CLAUDE.md`. The planned hardware wire contract is documented in
[the controller protocol](docs/controller-protocol.md). Development priorities,
dependencies, and phase exit criteria are maintained in [the roadmap](roadmap.md).
