Metadata-Version: 2.4
Name: synthetic-surveys
Version: 0.1.0
Summary: Generate surveys, simulate synthetic respondents, and produce exploratory reports
Author: Christian Arthur
License-Expression: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.6
Requires-Dist: openai>=1.40.0
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scipy>=1.10
Requires-Dist: scikit-learn>=1.3
Requires-Dist: requests>=2.31
Requires-Dist: Pillow>=10.0
Requires-Dist: matplotlib>=3.7
Requires-Dist: structlog>=23.1
Requires-Dist: keyring>=25.6
Requires-Dist: prompt-toolkit<4,>=3.0.52
Provides-Extra: executable
Requires-Dist: pyinstaller<7,>=6.16; extra == "executable"
Requires-Dist: pdfplumber>=0.10; extra == "executable"
Provides-Extra: ocr
Requires-Dist: pdfplumber>=0.10; extra == "ocr"
Requires-Dist: pytesseract>=0.3.10; extra == "ocr"
Provides-Extra: nlp
Requires-Dist: sentence-transformers>=2.2.2; extra == "nlp"
Provides-Extra: report
Requires-Dist: reportlab>=4.0; extra == "report"
Provides-Extra: gui
Requires-Dist: customtkinter>=5.2; extra == "gui"
Provides-Extra: desktop
Requires-Dist: customtkinter>=5.2; extra == "desktop"
Requires-Dist: pyinstaller>=6.0; extra == "desktop"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: ty>=0.0.21; extra == "dev"
Requires-Dist: ruff>=0.5.0; extra == "dev"
Requires-Dist: pre-commit>=3.7; extra == "dev"
Requires-Dist: black>=23.12; extra == "dev"
Requires-Dist: types-requests; extra == "dev"
Dynamic: license-file

# Synthetic Surveys

A CLI for exploring policy and program questions with synthetic survey respondents. Generate a ten-question survey, review or edit it, simulate a panel, and save a Markdown report with results and an interpretation.

These are model-generated responses, not measured public opinion. Use them to explore questions and hypotheses before research with people.

## Install

The macOS/Linux MVP will use a single-command installer powered by uv. The installer downloads Python 3.12 as needed, installs the Python package and dependencies, then starts the CLI. Public download URLs are not configured yet; see [distribution preparation](docs/DISTRIBUTION.md).

Self-contained downloads are built for Windows x64, Linux x64, and both Apple Silicon and Intel Macs. They include Python and dependencies. Extract the archive, then launch `synthetic-surveys.exe` on Windows or `Start Synthetic Surveys.command` on macOS. On Linux, run `./synthetic-surveys` in a terminal. Preview builds are unsigned and may require operating-system approval. Downloads remain private until public distribution is arranged.

Each archive includes `START-HERE.txt`. See [executable builds](docs/RELEASE.md#executable-builds) for build and verification instructions.

### Python package alternative

Requires Python 3.10 or later. Once the public wheel is published, download it from the release location linked on the website, then install it in an isolated environment:

```bash
python -m venv .venv
# macOS/Linux:
source .venv/bin/activate
# Windows PowerShell:
# .venv\Scripts\Activate.ps1
python -m pip install /path/to/synthetic_surveys-0.1.0-py3-none-any.whl
synthetic-surveys
```

Until a release is available, install from a source checkout with `python -m pip install .` inside the environment. Developers can use the workflow below. This guide does not assume a PyPI publication.

## First run

The default provider is **OpenRouter**. One `OPENROUTER_API_KEY` covers survey generation, respondent simulation, interpretation, and Pro-mode embeddings. Missing credentials trigger a connection menu with browser authorization and hidden API-key entry. The CLI validates OpenRouter/OpenAI keys using an account/models request before offering to save them.

- API usage is billed to your provider account. Start with two or three respondents; Pro mode and ablation make additional requests. Key validation does not generate model tokens and does not guarantee access to every model.
- Saving a key is optional. Credentials use macOS Keychain or the Linux desktop Secret Service through Python keyring. If no secure store is available, the app keeps the key only for the current process. Non-secret preferences remain in `~/.synthetic_surveys/settings.json`.
- Environment variables override saved settings. A bad exported key must be changed or unset; changing the saved key alone will not override it on the next launch.
- On first use, choose browser login (default), paste an API key, or authorize on another device and paste a one-time code. Browser login opens OpenRouter and returns authorization to a temporary localhost listener. Ctrl+C or a timeout returns to setup options; no separate OpenRouter CLI is required.

To replace a key or select a provider:

```bash
synthetic-surveys configure
synthetic-surveys configure --provider openai
```

Direct OpenAI configuration sets OpenAI as both the LLM and embedding provider. Remove any old OpenRouter model overrides when switching providers.

## Your first report

```bash
synthetic-surveys run --mode normal --sample-size 3
```

1. Type or paste your question into the text box. Enter adds a new line; use the arrow keys to edit. Press Tab to select **Continue**, then Enter to submit (or click **Continue**).
2. Optionally add survey-design guidance and supporting documents.
3. Choose a sampling profile, review the draft cost, and approve generation. You can change the draft model or question before paying.
4. Review the generated survey; accept it, edit questions for free, or approve a paid revision.
5. Choose the respondent model and count, then review an itemized estimate including the final interpretation and any Pro/ablation work. Change settings or cancel before starting.
6. Read the results, reported charges, and saved report path.

The question and survey-design context boxes preserve pasted paragraphs and blank lines. A character count shows the limit; if you exceed it, you can edit your text without starting over. Shift+Tab moves back to the text box, and **Cancel** or Ctrl+C exits. Redirected input uses ordinary line prompts; scripts can pass a multiline question with `--question`.

Reports go into `~/synthetic-surveys/reports/YYYY-MM-DD/`, with a readable survey-title-and-timestamp filename. The title comes from the generated survey, so saving does not require another model request. Choose another reports root with `--output-dir /path/to/reports` (`--dev-dir` remains an alias); date folders are created beneath that root. The CLI is interactive even when you supply a question and sample size. `generate-survey` and `sample-personas` are separate utility commands.

```bash
synthetic-surveys --help
synthetic-surveys generate-survey "Should the city pedestrianize Main Street?"
synthetic-surveys sample-personas 3
```

Normal mode directly selects Likert answers. **Pro mode** generates free-text responses and maps them through embeddings to Likert labels; it needs additional requests and fails clearly if embedding requests fail. `--ablation` adds a second control run without demographic/personality signals and reports distribution similarity.

### Cost transparency

The interactive survey flow reads OpenRouter's public model and embedding catalogs without sending your survey text. Estimates use current list prices, approximately three UTF-8 bytes per input token (including system prompts and JSON schemas), and explicit output allowances: 2,200 tokens for a draft, 800/1,000 per Normal/Pro respondent, 1,600 per anchor-generation call, and 400 per interpretation. It also shows costs at configured output limits; these are **not spending caps**. Allowances are planning assumptions, not measured usage averages.

Pro adds one anchor-generation request per survey item, then six embedding requests per item per respondent. The estimate includes repeated anchors and answers (40 tokens per anchor statement and 80 per answer). Its displayed Pro/Normal ratio depends on your models and panel size, includes the remaining report work, and excludes the already-generated draft. There is no fixed or empirically validated multiplier.

Actual charges can differ because of reasoning, output length, caching, routing, retries, or price changes. Native image/file processing, direct-provider prices, unknown models and embedding fallback costs may be unavailable; incomplete estimates are labeled explicitly. Credit-purchase fees and taxes are excluded. No price lookup makes an inference request.

OpenRouter response-reported charges and token counts are shown after the session and saved in completed reports alongside approved estimates. Missing costs are marked as incomplete, including embeddings if their response omits cost. Failed requests without usage data may still incur charges: your provider's billing activity remains authoritative. Canceling stops future steps, not charges already incurred. These checkpoints apply to the interactive CLI; utility commands and the desktop preview do not use this review flow.

## Weighting and IPUMS

| Profile | Included with the app? | Setup |
| --- | --- | --- |
| General demographic sampling (menu label: No Demographics Weights) | Built-in sampling proportions | None |
| Harm reduction | Built-in normalized overdose-rate weights | None |
| Massachusetts IPUMS | No; individual-record extract | Your own IPUMS USA access and a one-time download |

The profiles affect which personas are sampled, not post-hoc weighting of completed responses. Even the general profile has built-in demographic sampling proportions. Harm-reduction values are an approximation, not a population-adjusted epidemiologic model; see [methodology](docs/METHODOLOGY.md).

Selecting IPUMS without local data offers setup/download or a return to the profile menu. You can also prepare it explicitly:

```bash
synthetic-surveys configure --provider ipums
synthetic-surveys download-ipums-ma
```

Use your own [IPUMS USA account](https://account.ipums.org/usa/) and API key. The app requests sample `us2024a`, filters `STATEFIP=25`, and uses `PERWT` to sample Massachusetts records. Preparation may take several minutes. Later runs use the local file without an IPUMS API key.

The extract is saved as `~/.synthetic_surveys/data/ipums/massachusetts.csv.gz`. A successful replacement is atomic; failed downloads preserve the old extract. For a compatible existing local extract, copy it to that path. Old development checkouts may have `data/ipums/massachusetts_us2024a_3.csv.gz`; copy that file to the new location to reuse it.

**Do not commit or redistribute IPUMS microdata.** Redistribution requires IPUMS permission outside the stated publication exception. Follow [IPUMS USA terms](https://usa.ipums.org/usa/terms.shtml), and retain the extract's citation for reports/publications. The downloader specification is in `synthetic_surveys/ipums_api.py`; no IPUMS data is bundled.

## Documents and limits

Executable downloads include text and PDF extraction. Text-like files work with the base Python installation. Install `.[ocr]` from a checkout for PDF extraction and image OCR; image OCR also requires the Tesseract executable. Model-native attachment support varies by provider and model.

Limits: four attachments, 4 MB per file, 12 MB total, a 4,000-character question, and 2,000 characters of regeneration feedback. Questions, document content, and generated personas are sent to the selected providers. Reports contain research inputs and run metadata, so choose their storage location accordingly.

## Environment variables

All are optional unless needed by the selected feature. Environment values take precedence over saved settings.

| Variable | Purpose/default |
| --- | --- |
| `OPENROUTER_API_KEY` | Default LLM and embedding credential |
| `OPENAI_API_KEY` | Direct OpenAI credential; optional embedding fallback |
| `GEMINI_API_KEY` | Only for direct Gemini embeddings |
| `IPUMS_API_KEY` | Only for downloading an IPUMS extract |
| `SYNTHETIC_SURVEYS_HOME` | App settings/data/cache root; defaults to `~/.synthetic_surveys` |
| `SYNTHETIC_SURVEYS_LLM_PROVIDER` | `openrouter` (default), `openai`, or offline `stub` |
| `SYNTHETIC_SURVEYS_EMBEDDING_PROVIDER` | `openrouter`, `openai`, or `gemini`; CLI defaults to the selected LLM provider |
| `SYNTHETIC_SURVEYS_MODEL` | Override all LLM stages |
| `SYNTHETIC_SURVEYS_SURVEY_MODEL` | Override survey generation model |
| `SYNTHETIC_SURVEYS_SIMULATION_MODEL` | Override simulation model |
| `SYNTHETIC_SURVEYS_SUMMARY_MODEL` | Override interpretation model |
| `SYNTHETIC_SURVEYS_OPENROUTER_EMBEDDING_MODEL` | `openai/text-embedding-3-small` |
| `SYNTHETIC_SURVEYS_OPENAI_EMBEDDING_MODEL` | `text-embedding-3-small` |
| `SYNTHETIC_SURVEYS_EMBEDDING_MODEL` | Direct Gemini embedding model |
| `SYNTHETIC_SURVEYS_IPUMS_SAMPLE` | Download sample override; default `us2024a` |
| `SYNTHETIC_SURVEYS_SSR_EPSILON` | SSR bias offset; default `0.0` |
| `SYNTHETIC_SURVEYS_SSR_TEMPERATURE` | SSR temperature; default `1.0` |

The checked-in defaults use `openai/gpt-5.5` for OpenRouter survey/summary stages and `openai/gpt-5.4-mini` for simulation. Provider availability can change; use model overrides if necessary.

## Local development

```bash
uv sync --extra dev
cp .env.example .env
# Edit .env locally to add your key. Never commit it.
```

The application does **not** load `.env` itself. If you use direnv, create `.envrc` containing:

```bash
dotenv_if_exists .env
if [[ -d .venv/bin ]]; then
  PATH_add .venv/bin
fi
```

Then run `direnv allow` with direnv installed and hooked into your shell. Alternatively export the variables in your shell. Both files are gitignored.

```bash
uv run synthetic-surveys
uv run ruff check .
uv run ty check
uv run pytest -q
```

For an offline demo with placeholder responses, use `uv run synthetic-surveys --provider stub --mode normal`. This is not a real simulation. Live OpenAI tests require `SYNTHETIC_SURVEYS_RUN_OPENAI_TESTS=1` and `OPENAI_API_KEY`.

See [release verification](docs/RELEASE.md) and [methodology](docs/METHODOLOGY.md). License: Apache-2.0.
