Metadata-Version: 2.4
Name: carwatch
Version: 1.0.0
Summary: Registration-aware processing of CARWatch sampling logs and saliva data.
Project-URL: Homepage, https://github.com/carwatch-tools/carwatch-python
Project-URL: Documentation, https://github.com/carwatch-tools/carwatch-python/tree/main/docs
Project-URL: Repository, https://github.com/carwatch-tools/carwatch-python
Project-URL: Issues, https://github.com/carwatch-tools/carwatch-python/issues
Project-URL: Changelog, https://github.com/carwatch-tools/carwatch-python/blob/main/CHANGELOG.md
Author-email: Robert Richer <robert.richer@fau.de>
License-Expression: MIT
License-File: LICENSE
Keywords: ambulatory assessment,cortisol,saliva,sampling compliance
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Requires-Python: <4.0,>=3.10
Requires-Dist: ipympl>=0.10.0
Requires-Dist: matplotlib<4,>=3.10
Requires-Dist: numpy<3,>=1.26
Requires-Dist: pandas<3,>=2
Requires-Dist: pingouin<1,>=0.5
Requires-Dist: scipy<2,>=1
Requires-Dist: seaborn<1,>=0.13
Provides-Extra: interactive
Requires-Dist: ipydatagrid<2,>=1.4; extra == 'interactive'
Requires-Dist: ipywidgets<9,>=8; extra == 'interactive'
Description-Content-Type: text/markdown

# CARWatch — Python tools for processing CARWatch and saliva data

<img src="docs/_static/brand/logo.svg" align="right" width="120" alt="CARWatch logo">

[![PyPI](https://img.shields.io/pypi/v/carwatch)](https://pypi.org/project/carwatch/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/license/mit/)
[![Documentation Status](https://readthedocs.org/projects/carwatch-python/badge/?version=latest)](https://carwatch-python.readthedocs.io/en/latest/?badge=latest)
[![Test and Lint](https://github.com/carwatch-tools/carwatch-python/actions/workflows/test-and-lint.yml/badge.svg)](https://github.com/carwatch-tools/carwatch-python/actions/workflows/test-and-lint.yml)
[![codecov](https://codecov.io/gh/carwatch-tools/carwatch-python/branch/main/graph/badge.svg?token=IK0QBHQKCO)](https://codecov.io/gh/carwatch-tools/carwatch-python)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![PyPI downloads](https://img.shields.io/pypi/dm/carwatch)](https://pypi.org/project/carwatch/)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/carwatch-tools/carwatch-python)


_CARWatch_ supports the processing of app-recorded sampling logs and their
integration with saliva biomarkers. It is designed for ambulatory sampling
studies in which researchers need auditable sampling times, protocol deviations,
manual diary fallbacks, and biomarker features.

- Import CARWatch logs from CSV files, ZIP archives, and participant folders.
- Reconstruct registration-aware study days and sampling positions.
- Review conversion anomalies in a structured, editable two-pass issue report.
- Patch missing timestamps from a wide-format manual measurement diary.
- Load Study Manager exports and simplified study results.
- Merge saliva measurements by physical sample ID or scheduled sample position.
- Correct documented tube swaps and compute cortisol response features.

**Documentation:** [User guides and API reference](https://github.com/carwatch-tools/carwatch-python/tree/main/docs)

**Working with coding agents:** [`AGENTS.md`](AGENTS.md) and the
[repository-local skills](skills) provide task-specific guidance for raw-log
processing, saliva analysis, and package development. See the
[agent-assisted workflow guide](docs/guides/agent_assisted_workflows.rst).

## What CARWatch produces

CARWatch makes sampling adherence visible at the participant-day level. The
timeline shows the recorded sampling times, protocol targets, app-updated
targets, timing deviations, and the source of each timestamp.

![Sampling timeline with app-updated targets and timing compliance](docs/images/sampling_timeline.png)

Conversion anomalies can be resolved in a spreadsheet or in an optional,
interactive notebook editor. The editor limits decisions to actions valid for
the selected issue and refreshes the queue after upstream corrections.

![Interactive conversion issue editor in Jupyter](docs/images/conversion_issue_editor.png)

## Installation

CARWatch requires Python 3.10 or newer.

Using [uv](https://docs.astral.sh/uv/) in an existing Python project:

```bash
uv add carwatch
```

Using `pip` in an existing virtual environment:

```bash
pip install carwatch
```

### Installing the latest version from GitHub

The current development version can be installed directly from GitHub:

```bash
uv add "carwatch @ git+https://github.com/carwatch-tools/carwatch-python.git"
```

The `main` branch can contain unreleased or unstable changes. To work on the
source code itself:

```bash
git clone https://github.com/carwatch-tools/carwatch-python.git
cd carwatch-python
uv sync
```

## Not familiar with Python? Don't worry!

The [beginner setup tutorial](https://github.com/carwatch-tools/carwatch-python/blob/main/docs/guides/python_setup.md)
explains how to install everything you need to use CARWatch on macOS, Linux, and Windows, including installing `uv`, creating an isolated
CARWatch analysis environment, installing the package, and running Jupyter notebooks.
No existing Python installation or manual environment activation is required.

## Typical workflow

The complete workflow separates raw-log reconstruction, researcher decisions,
and downstream saliva analysis. The examples below build on the variables
created in the preceding subsection.

### 1. Load raw CARWatch logs

For the common layout with one folder per participant, map the study-specific
folder names to the participant IDs used in the CARWatch filenames:

```python
from pathlib import Path

import carwatch as cw

participant_folders = {
    "vp01": Path("data/carwatch/vp01"),
    "vp02": Path("data/carwatch/vp02"),
}

raw_logs, source_audit = cw.io.load_raw_logs_from_participant_folders(
    participant_folders,
    create_report=True,
)
```

`source_audit` lists every selected or excluded CSV or ZIP source, the reason
for that choice, and the raw-event count for each selected source. If the
relevant files are already known, load them
directly instead:

```python
raw_logs = cw.io.load_raw_logs(
    ["data/carwatch/vp01.csv", "data/carwatch/vp02.zip"],
)
```

### 2. Reconstruct the study and create an issue report

The first conversion pass reconstructs registrations, canonical study days,
awakening times, and scheduled samples. In warning mode it returns usable
results while reporting unresolved anomalies:

```python
initial_results, conversion_report = (
    cw.logs.convert_raw_logs_to_study_manager_summary(
        raw_logs,
        errors="warn",
        create_report=True,
    )
)

conversion_report["issues"].to_csv("conversion_issues.csv")
print(conversion_report["summary"])
```

The exported report is a resolution queue. Its predefined `accept` decisions
are proposals only and are not applied during this first pass.

### 3. Review and apply issue decisions

Review `conversion_issues.csv` in a spreadsheet editor. Keep `accept` to execute
the proposed action, select another documented decision, or clear the cell to
leave the issue unresolved.

If missing timestamps should be taken from a manual measurement diary, load the
wide diary before the second conversion:

```python
decisions = cw.logs.load_conversion_issue_report("conversion_issues.csv")
manual_diary = cw.io.load_manual_diary("manual_diary.csv")
checker = cw.compliance.SamplingComplianceChecker(
    awakening_delay_tolerance_min=5,
    sampling_delay_tolerance_min=5,
    absolute_time_tolerance_min=15,
)

study_results, final_report = (
    cw.logs.convert_raw_logs_to_study_manager_summary(
        raw_logs,
        errors="raise",
        create_report=True,
        issue_decisions=decisions,
        manual_diary=manual_diary,
        compliance_checker=checker,
    )
)
```

Omit `manual_diary` when no accepted decision uses it. With `errors="raise"`,
the second pass stops if a submitted decision is invalid or an issue remains
unresolved. Sampling compliance is checked by default. Relative samples are
checked against awakening or the preceding sample; fixed-time samples are
checked against their registered clock time. Supply `sampling_schedule` when
the registration contains no complete timing schedule, or set
`check_compliance=False` to skip this assessment.

### 4. Inspect and save canonical study results

Create focused day-level and sample-level tables for quality control:

```python
study_days = cw.logs.extract_day_summary_from_summary(study_results)
study_samples = cw.logs.extract_sample_events_from_summary(study_results)
analysis_results = cw.compliance.drop_non_compliant_samples(study_results)

cw.io.save_study_results(study_results, "study_results.csv")
```

`study_days` contains dates, awakening information, registration context, and
day-level compliance.
`study_samples` contains actual sampling times, minutes since awakening,
scheduled and recorded sample IDs, sample positions, timing deviations,
sample-level compliance, and mismatch indicators.
`drop_non_compliant_samples()` preserves the canonical wide results format. It
clears the complete participant-day when any sample failed. Pass
`drop_entire_day=False` to clear only failed sample observations, or
`drop_unassessed=True` to also clear unassessed data.

Visual inspection uses the same final long tables. The timeline is most useful
when reviewing an individual record; the other plots provide cohort-level
quality control and cortisol-response checks:

```python
fig, ax = cw.plotting.plot_sampling_timeline(
    study_results,
    participant="vp01",
    day="D1",
)

# In Jupyter, inspect any participant-day interactively.
# Requires: uv add "carwatch[interactive]"
from IPython.display import display

timeline_widget = cw.plotting.interactive_sampling_timeline(study_results)
display(timeline_widget)

compliance_summary = cw.compliance.summarize_compliance(study_results)
fig, ax = cw.plotting.plot_compliance_overview(study_results)
fig, ax = cw.plotting.plot_timing_deviation(study_results)
```

### Resolve conversion issues interactively

The structured conversion report can be reviewed directly in Jupyter instead
of exporting it to a spreadsheet. Install the interactive extra, create the
initial report, and use **Refresh remaining issues** after changing decisions.
The editor retains accepted upstream decisions while showing only issues that
still require review.

```python
_, conversion_report = cw.logs.convert_raw_logs_to_study_manager_summary(
    raw_logs, errors="warn", create_report=True
)
editor = cw.logs.interactive_conversion_issue_report(
    raw_logs,
    conversion_report["issues"],
    manual_diary=manual_diary,
)
display(editor.widget)

# After resolving and refreshing the issue queue:
study_results = cw.logs.convert_raw_logs_to_study_manager_summary(
    raw_logs,
    errors="raise",
    issue_decisions=editor.decisions,
    manual_diary=manual_diary,
)
```

CSV export and `cw.logs.load_conversion_issue_report()` remain available when
decisions need to be reviewed outside Jupyter.

The cohort-level protocol and, when necessary, the detailed registration
schedule can optionally be inspected with:

```python
protocol = cw.logs.summarize_protocol(raw_logs)
registration_schedule = (
    cw.logs.extract_registration_schedule_from_raw_logs(raw_logs)
)
```

### 5. Restore results in a later analysis

Reload package-generated results:

```python
study_results = cw.io.load_study_results(
    "study_results.csv",
)
```

Use `simple=True` only for display or quick inspection. It is intentionally
rejected by merge, quality-control, plotting, and feature functions; load the
complete result for those operations. If the starting point is a flat export
from the CARWatch Study Manager rather than package-generated results, use:

```python
study_results = cw.io.load_study_manager_export(
    "study_manager_export.csv",
)
```

### 6. Load or prepare laboratory saliva data

When the laboratory export contains the physical tube IDs registered in
CARWatch, use the standard long format
`participant,sample,cortisol`:

```python
saliva = cw.io.load_saliva("cortisol.csv")
```

When the laboratory export identifies samples by study day and sampling
position, perform the study-specific column renaming first and create the
required index:

```python
import pandas as pd

saliva = (
    pd.read_csv("cortisol_by_position.csv")
    .rename(columns={"vp_nr": "participant"})
    .set_index(["participant", "day", "sample_position"])
)
```

All non-index measurement columns must be numeric. Additional metadata such as
`condition` can be retained as an additional named index level.

### 7. Merge sampling and laboratory data

Merge by physical tube ID:

```python
merged_results = cw.merge.merge_saliva(
    study_results,
    saliva,
    match_on="sample",
    correct_swaps=True,
)
```

For position-based laboratory data, use:

```python
merged_results = cw.merge.merge_saliva(
    study_results,
    saliva,
    match_on="position",
    correct_swaps=True,
)
```

The result remains canonical wide Study Results. It retains CARWatch timing,
laboratory values at sample level, day-level metadata such as condition, and
explicit laboratory availability and tube-swap flags. Save it with
`cw.io.save_study_results()` and load it with `cw.io.load_study_results()`.

Plot the aligned cortisol response using actual CARWatch sampling times:

```python
fig, ax = cw.plotting.plot_saliva_curve(merged_results, value="cortisol")
```

### 8. Compute saliva response features

The CARWatch adapter groups the merged samples by participant and day, orders
them by `sample_position`, and uses the actual `time_min` values:

```python
cortisol_features = cw.saliva.compute_features_from_carwatch(
    merged_results,
    saliva_type="cortisol",
)
```

The output contains AUCg, AUCi, initial value, maximum value, maximum increase,
and slope features. For generic long-format saliva data that did not originate
from `cw.merge.merge_saliva`, use `cw.saliva.compute_features()` and specify its grouping
and sample levels when required.

See the
[raw-log processing guide](https://github.com/carwatch-tools/carwatch-python/blob/main/docs/guides/raw_log_processing.rst)
for the complete two-pass workflow, manual diary integration, and validation
views.

## Citation

Report the CARWatch package version used in the analysis. For research using
the CARWatch framework, cite:

> Richer, R., Abel, L., Küderle, A., Eskofier, B. M., & Rohleder, N. (2023).
> CARWatch — A smartphone application for improving the accuracy of cortisol
> awakening response sampling. *Psychoneuroendocrinology, 151*, 106073.
> https://doi.org/10.1016/j.psyneuen.2023.106073

The installed package version is available as:

```python
import carwatch

print(carwatch.__version__)
```

## Contributing

Bug reports, feature requests, and reproducible examples belong in the
[GitHub issue tracker](https://github.com/carwatch-tools/carwatch-python/issues).
Changes should include tests and documentation for the affected research
workflow.

## License

CARWatch is published under the
[MIT License](https://opensource.org/license/mit/).

## For developers

Install [uv](https://docs.astral.sh/uv/getting-started/installation/), clone the
repository, and synchronize the project environment:

```bash
git clone https://github.com/carwatch-tools/carwatch-python.git
cd carwatch-python
uv sync
```

The main development commands are:

```bash
uv run poe format      # Format and automatically fix source files.
uv run poe ci_check    # Check formatting and linting.
uv run poe test        # Run the test suite with coverage.
uv run poe docs        # Build the Sphinx documentation.
uv run poe docs_preview
```

The package dependencies and development dependencies are managed through
`pyproject.toml`. `uv sync` resolves and installs the versions recorded by the
project environment.
