Skip to content

Known-truth benchmark traces

circadian_workbench.benchmark_cases constructs continuous test traces for evaluating analysis choices. It returns the normal Workbench Result, including case/trace tables, a reproducible script and the completed run record. The same operation is available as the benchmark_cases action through cw.call.

These are simulations under declared assumptions. They do not identify real biological negatives or establish that a measurement should have a daily rhythm. Generation does not call a period estimator or rhythmicity test.

import circadian_workbench as cw

design = {
    "replicates": 10,
    "truth_policy": {
        "min_observations": 24,
        "min_cycles": 3,
        "period_min_hours": 2,
        "period_max_hours": 48,
        "relative_tolerance": 0.10,
        "absolute_tolerance_hours": 0,
        "target": "all",
        "extra_components": "penalize",
    },
    "scenarios": [
        {"id": "triangle-12", "components": [
            {"id": "oscillator", "waveform": "triangle", "period_hours": 12,
             "amplitude": 2, "phase_hours": 0}],
         "noise": {"kind": "white", "sd": 0.5}},
        {"id": "constructed-negative", "components": [],
         "noise": {"kind": "exponential", "sd": 0.5, "correlation_hours": 2},
         "drift": {"linear_per_hour": 0.01},
         "disturbances": [{"kind": "pulse", "start_hours": 20,
                           "duration_hours": 1, "amplitude": 3}]},
    ],
}
profiles = [{"id": "recording-profile", "hours": list(range(96)),
             "missing": [20 <= hour < 25 for hour in range(96)],
             "metadata": {"source": "declared example; no biological replicates"}}]
development = cw.benchmark_cases(design, profiles, partition="development", seed=321)
# Request this only after choices and decision rules have been frozen:
confirmation = cw.benchmark_cases(design, profiles, partition="confirmation", seed=321)
cases = development.tables["cases"]
traces = development.tables["traces"]

The numerical values above illustrate an explicit design; they are not universal scientific tolerances. Freeze them before comparing candidates. A complete audit should declare additional periods, waveform/component combinations, disturbance and noise levels, and recording profiles appropriate to the intended analysis.

Input contract

The design requires scenarios, a positive integer replicates, and every truth_policy key shown above. target is all or dominant; extra_components is penalize or ignore. These matching instructions are saved for the scorer; the generator never scores a candidate or chooses a winner. The tolerance for a target is the larger of the absolute tolerance and period times relative tolerance.

Each scenario requires a unique id and components. An empty component list is a constructed negative. Components require unique ids, distinct positive periods, positive amplitudes and a waveform: cosine, triangle, sawtooth or square. Square waves optionally specify a duty fraction strictly between zero and one. phase_hours defaults to zero and locates the waveform origin, not a common biological phase: the cosine origin is a peak, triangle/sawtooth origin is a trough, and square origin begins the positive interval. Non-cosine waveforms contain harmonics; these are not labelled as extra independent ground-truth oscillators. The piecewise waveforms are not band-limited, so sparse sampling can alias them. This limitation follows the SciPy waveform definitions.

Optional scenario settings:

  • baseline defaults to zero. drift.linear_per_hour and drift.quadratic_per_hour2 default to zero and act on time from the profile origin.
  • noise.kind is none (default), white or exponential. Stochastic noise requires positive sd in measurement units; exponential noise also requires a positive correlation_hours. Unused parameters are rejected rather than ignored.
  • Disturbances are step or pulse with start time and signed amplitude. A pulse also requires positive duration; its interval includes the start and excludes the end. These times are relative to the profile origin.
  • retain_observations optionally retains the first N originally present samples and masks the rest, producing a deliberately short case without compressing time.
  • weight defaults to one and must be positive. It is a scenario weight for later scoring, not a number of independent biological samples.

Profiles require unique ids and sorted unique finite hours. The optional missing list has exactly one boolean per timestamp; true means unavailable. Missing values remain null in observed traces, with source/limited-observation reasons. Omitted time points remain gaps, without being reindexed or zero-filled. origin_hours defaults to zero on the provided time axis. An empty profile yields an explicitly insufficient case. Arbitrary finite-JSON metadata preserves the source movie/cell/measurement, units and confirmed sample assignment if supplied.

Outputs are continuous measurements and may be signed, matching Workbench's trace interface. They are not nonnegative activity-count recordings and are not clipped, rounded or transformed into counts. One call refuses more than 10,000 cases or 2,000,000 observation positions; batch an already frozen large design if needed.

Randomness and truth

White noise is zero-mean independent Gaussian noise with the declared standard deviation. Exponential noise is a stationary Gaussian Ornstein–Uhlenbeck process with covariance sd**2 * exp(-abs(time_difference)/correlation_hours). Its exact transition on an interval Δt uses correlation exp(-Δt/τ) and innovation standard deviation sd * sqrt(1-exp(-2*Δt/τ)); the initial value has stationary variance sd**2. Generation uses actual time differences and applies missingness after generating the latent series. This is the exact-time transition, rather than an Euler approximation. Gillespie, Physical Review E 54, 2084 (1996).

Each profile/scenario/replicate/partition has a content-keyed PCG64 stream derived with SeedSequence from a SHA256 digest and the explicit root seed. Case order or adding another candidate/case does not change an existing realization. The full seed entropy, source hash and NumPy/SciPy/Workbench versions are retained; numerical results are reproducible with those recorded implementations. The stream construction uses NumPy's supported sequence-of-integers seeding.

Development and confirmation use distinct streams. Deterministic cases may be generated in either partition, but fresh_confirmation_eligible is false: a copied deterministic waveform is not independent confirmation. A fully missing case, or noise too small to alter represented observations, also cannot supply fresh evidence. A downstream audit must enforce the separation of selection and confirmation; the generator itself is a scientific primitive, not an access-control or recommendation system.

Truth eligibility uses the frozen minimum observations, observed span/cycles and period bounds. It is not a guarantee of identifiability under severe gaps or aliasing. Component eligibility and reasons are preserved even when the whole case is insufficient. Equal largest amplitudes make a dominant-only target ambiguous. Constructed positives with insufficient support never become negative controls.

Saved result structure

  • cases: one record per case, with case/profile/scenario/partition/replicate identity, source metadata, all components, target eligibility/reasons/tolerances, observed support, weight, seed entropy, confirmation eligibility and trace hashes.
  • traces: original hours, observed values/masks/rejection reasons, latent rhythm, individual components, baseline/drift, disturbances and noise, keyed by case id.
  • design, profiles, design_id, partition, seed: the complete resolved generating specification. Explicit defaults survive the round-trip.
  • provenance: schema/version, generator source hash, numerical-library versions and stream derivation. The surrounding Workbench result also carries its normal completed run record and executable replay script.

No source recording is changed. No statistical claim is inferred from a generated trace until a separate candidate evaluator and frozen scoring policy are applied.