Metadata-Version: 2.4
Name: design-research-experiments
Version: 0.4.0
Summary: Study design and orchestration for the CMU Design Research Collective ecosystem
Author: The Design Research Collective
Maintainer-email: "Christopher C. McComb" <ccm@cmu.edu>
License-Expression: MIT
Project-URL: Homepage, https://cmudrc.github.io/design-research-experiments/
Project-URL: Documentation, https://cmudrc.github.io/design-research-experiments/
Project-URL: Repository, https://github.com/cmudrc/design-research-experiments
Project-URL: Issues, https://github.com/cmudrc/design-research-experiments/issues
Keywords: design,research,experiments,orchestration,doe
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML<7,>=6.0
Requires-Dist: tqdm<5,>=4.66
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: mypy<2,>=1.10; extra == "dev"
Requires-Dist: pydata-sphinx-theme<1,>=0.16; extra == "dev"
Requires-Dist: pre-commit<5,>=3.7; extra == "dev"
Requires-Dist: pytest<9,>=8.2; extra == "dev"
Requires-Dist: pytest-cov<8,>=7.0; extra == "dev"
Requires-Dist: ruff<1,>=0.6.0; extra == "dev"
Requires-Dist: sphinx<9,>=7.4; extra == "dev"
Requires-Dist: sphinx-copybutton<1,>=0.5; extra == "dev"
Requires-Dist: twine<7,>=5.1; extra == "dev"
Requires-Dist: types-PyYAML<7,>=6.0; extra == "dev"
Requires-Dist: types-tqdm<5,>=4.66; extra == "dev"
Provides-Extra: doe
Requires-Dist: pyDOE3<2,>=1.0; extra == "doe"
Requires-Dist: scipy<2,>=1.11; extra == "doe"
Dynamic: license-file

# design-research-experiments
[![CI](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml)
[![Coverage](https://raw.githubusercontent.com/cmudrc/design-research-experiments/HEAD/.github/badges/coverage.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml)
[![Examples Passing](https://raw.githubusercontent.com/cmudrc/design-research-experiments/HEAD/.github/badges/examples-passing.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/examples.yml)
[![API in Examples](https://raw.githubusercontent.com/cmudrc/design-research-experiments/HEAD/.github/badges/examples-api-coverage.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/examples.yml)
[![Docs](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml)
[![PyPI Version](https://img.shields.io/pypi/v/design-research-experiments.svg)](https://pypi.org/project/design-research-experiments/)
[![Python Versions](https://img.shields.io/pypi/pyversions/design-research-experiments.svg)](https://pypi.org/project/design-research-experiments/)

`design-research-experiments` is the study-design and orchestration layer
in the CMU Design Research Collective design-research ecosystem. It makes
hypotheses, factors, conditions, replications, and artifact flows explicit
before execution.

It composes sibling libraries rather than reimplementing them:

- `design-research-agents` for executable agent behavior, workflows, and traces
- `design-research-problems` for problem catalogs, registries, and evaluators
- `design-research-analysis` for downstream unified-table analysis and reporting

## Quality Signals

- **Coverage** reports total line coverage for the default deterministic test suite; CI requires at least 95%.
- **Examples Passing** reports checked-in example scripts that execute successfully in the examples workflow.
- **API in Examples** reports curated top-level `__all__` exports referenced by runnable examples. `N/N` means every supported top-level export appears in at least one example, and CI requires 100%.

Run `make coverage`, `make examples-test`, and `make examples-coverage` to reproduce these checks locally.

## Overview

This package centers on reproducible experiment structure and execution:

- typed schemas for studies, factors, blocks, hypotheses, outcomes, and analysis plans
- design-of-experiments materialization (full/constrained factorial, randomized block,
  repeated measures, latin square, custom matrices)
- run orchestration with deterministic seeding, always-on per-run evidence,
  optional checkpointing, resume support, and interactive `tqdm` progress
- explicit, disabled-by-default paper-support aggregation and paper-draft
  assembly with provenance, curated references, visible TODOs, and author-review boundaries
- standalone condition callbacks for simulations that do not require synthetic
  problem or agent bindings
- canonical artifact exports (`study.yaml`, `manifest.json`, `conditions.csv`,
  `runs.csv`, `events.csv`, `evaluations.csv`, and machine-readable hypothesis/plan files)
- documented artifact contracts that downstream analysis can ingest directory-first
- documented composition seams for user code:
  `design_research_problems.integration`, the stable
  `design_research_agents.study` facade, and the top-level
  `design_research_analysis` artifact API

`design-research-experiments` itself is the orchestration surface. There is
intentionally no separate `design_research_experiments.integration` module.
Internally, the package adapters consume
`design_research_problems.integration` and
`design_research_agents.integration`; the latter is a compatibility seam, not
the recommended authoring API. New user code should use
`design_research_agents.study`.

## Quickstart

Requires Python 3.12+.
Maintainer workflows target Python `3.12` (`.python-version`).
For a VS Code path that starts from PyPI and then shows the repository example
workflow, see
[VS Code Start](docs/vscode_start.rst).

Install the base package from PyPI:

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install design-research-experiments
```

The base install is enough to define, validate, and materialize a study:

```python
from design_research_experiments import build_design, build_prompt_framing_study

study = build_prompt_framing_study()
conditions = build_design(study)
print(study.study_id, len(conditions))
```

For contributor workflows from a source checkout:

```bash
python -m venv .venv
source .venv/bin/activate
make dev
make test
```

Run the repository's basic example:

```bash
make run-example
```

This repo maintains a hard 95% total line-coverage floor in CI via
`make coverage`. The repo-specific rule tracks the family-wide coverage policy
in [cmudrc/design-research#4](https://github.com/cmudrc/design-research/issues/4).

## CLI

The package installs a `drexp` CLI:

```bash
drexp validate-study path/to/study.yaml
drexp materialize-design path/to/study.yaml
drexp generate-doe --kind lhs --factors-json '{"x": [0, 1], "y": [10, 20]}' --n-samples 12 --out artifacts/doe.csv
drexp run-study path/to/study.yaml
drexp resume-study path/to/study.yaml
drexp export-analysis path/to/study.yaml
drexp draft-paper path/to/output_dir --require-complete
drexp bundle-results path/to/output_dir
```

Install `design-research-experiments[doe]` to enable optional SciPy/QMC and
pyDOE3 backends for `generate-doe --backend ...`; the stdlib backend remains
the default.

## Examples

See [examples/README.md](https://github.com/cmudrc/design-research-experiments/blob/HEAD/examples/README.md)
for runnable scripts, including end-to-end recipe executions.

## Docs

See the [published documentation](https://cmudrc.github.io/design-research-experiments/)
for guides, the [artifact contract](https://cmudrc.github.io/design-research-experiments/artifact_contract.html),
and API reference.

For whole-stack orientation and canonical cross-package examples, use the
[`design-research` umbrella documentation](https://cmudrc.github.io/design-research/).

Build docs locally with:

```bash
make docs-check
make docs-build
```

## Public API

Selected primary entry points include:

- `Study`, `Factor`, `Level`, `Constraint`, `Condition`, `Block`, `DesignSpec`, `DesignKind`
- `RecipeStudyConfig`, `ComparisonStudyConfig`, and recipe-specific typed config classes
- `Hypothesis`, `OutcomeSpec`, `AnalysisPlan`
- `RunSpec`, `RunResult`, `RunOutput`, `ConditionRunner`, `BenchmarkBundle`
- `build_design`, `generate_doe`, `materialize_conditions`
- `build_univariate_comparison_study`, `build_bivariate_comparison_study`,
  `build_strategy_comparison_study`, and other recipe builders
- `run_study`, `resume_study`, `load_run_evidence_records`
- `collect_paper_support`, `export_paper_support`, `export_paper_draft`,
  `PaperDraftIncompleteError`
- `export_analysis_tables`, `validate_study`

## Contributing

Contribution workflow and quality gates are documented in
[CONTRIBUTING.md](https://github.com/cmudrc/design-research-experiments/blob/HEAD/CONTRIBUTING.md).
