Metadata-Version: 2.3
Name: ark-audio-agent
Version: 0.1.9
Summary: Autonomous text-to-music composer driven by a multi-agent ReAct harness (Claude API, with an offline deterministic fallback).
Keywords: music,generative,agents,react,claude,midi,synthesizer
Author: Immanuel R
Author-email: Immanuel R <raj@immanuel.co>
License: MIT
Requires-Dist: numpy>=1.24
Requires-Dist: mido>=1.3
Requires-Dist: rich>=13.7
Requires-Dist: anthropic>=0.107.1 ; extra == 'llm'
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/ir-dev/ark-audio-agent
Project-URL: Repository, https://github.com/ir-dev/ark-audio-agent
Project-URL: Issues, https://github.com/ir-dev/ark-audio-agent/issues
Provides-Extra: llm
Description-Content-Type: text/markdown

# ark-audio-agent

**Autonomous text → music.** Give it a sentence; it reasons about the feeling,
composes a structured piece, and renders it to MIDI + audio — with no human in
the loop.

The interesting part is the *harness*: composition is driven by a team of
specialist agents running a **ReAct loop** (reason → act → observe) over a shared
musical blackboard. It runs on the **Claude API** when a key is available, and on
a **fully-offline deterministic music-theory brain** otherwise, so it always
produces a result.

```bash
ark-audio "a rainy night in a neon city, lonely but hopeful"
# → renders/20261011-132450_a-rainy-night-in-a-neon-city-lonely-but/song.mp3  (+ .wav, .mid, report.md, trace.jsonl)
```

Add a **direction** to say how the piece must be built, in plain words. The text
still sets the mood:

```bash
ark-audio "The Lord is my shepherd; I shall not want." \
  --direction "a few secs of intro and a bit of rising notes before the main melody, \
  structured, fewer long leaps, strongly leaning toward one chord and tone; after the \
  climax a falling melody, then a few sec of outro to end it"
```

Name **reference music** to draw on: a piece from the built-in library, a style, or your
own MIDI file. The piece borrows its traits (chords, melodic shape, palette, groove), not
its notes:

```bash
ark-audio "a rainy afternoon in an empty flat" -r satie --borrow harmony,melody,tempo
ark-audio "a summer night drive" -r synthwave-night-drive
ark-audio "develop my sketch" -r ./idea.mid --strength 0.8
```

> **Building an agent or app on top of this?** [docs/AI_SPEC.md](docs/AI_SPEC.md) is the
> full contract: every parameter, nuance and conflict. [docs/PROMPTS.md](docs/PROMPTS.md)
> has a ready-made system prompt and Claude, OpenAI-compatible, MCP and CLI wiring.
> `ark_audio_agent.spec` ships the tool definitions and a `run_tool` dispatcher.

---

## Why it's built this way

Rather than ask one model to emit raw notes (which drifts out of key and is hard
to steer), the system separates **musical intent** from **musical realization**:

- **Agents decide intent** — key, mode, tempo, song form, chord progressions, a
  melody's contour/register/density, instrument roles, drum and bass styles.
- **Deterministic engines realize it** — pure music-theory functions turn that
  intent into in-key notes, which a `mido` writer turns into MIDI and a
  from-scratch `numpy` synthesizer turns into audio.

Because every decision an agent can make is a **tool**, both brains (Claude and
the offline planner) drive the exact same action surface and produce the exact
same trace format.

### Architecture

```
            text prompt
                │
     ┌──────────▼───────────────────────────────────────────────┐
     │  Orchestrator (the harness)                                │
     │                                                            │
     │   Interpreter → Composer → Melodist → Arranger             │  each agent runs a
     │                               │                            │  ReAct loop:
     │                               ▼                            │   reason (thinking)
     │                            Critic ──reject──┐              │   → act  (tool call)
     │                               │approve      │ one targeted │   → observe (result)
     │                               │        revision (bounded)  │
     └───────────────────────────────┼────────────┘              │
                                      │                           │
          ┌───────────────────────────▼───────────────────────┐  │
          │  Tools  (the ReAct action space)                   │  │
          │  set_brief · set_direction · get_theory ·          │  │
          │  find_references · choose_references · set_global ·│  │
          │  set_structure · set_progression · design_melody · │  │
          │  set_instruments · set_bass · set_drums ·          │  │
          │  analyze_score · finish / approve / request_revision│ │
          └───────────────────────────┬───────────────────────┘  │
                                       ▼                          │
                            Score blackboard (shared state)       │
     └──────────────────────────────┬─────────────────────────────┘
                                     ▼
          performance.py  →  { melody, harmony, bass, drums } notes
                                     │
                 ┌───────────────────┴────────────────────┐
                 ▼                                         ▼
        midi_render.py (mido)                   synth.py (numpy software synth)
                 │                                         │
              song.mid                        song.wav ──(ffmpeg)──► song.mp3
```

### The agents

| Agent | Decides | Tools |
|---|---|---|
| **Interpreter** | mood, imagery, genre, key/mode/tempo, emotional arc; reviews the direction's plan; picks reference music | `set_brief`, `set_title`, `set_direction`, `find_references`, `choose_references` |
| **Composer** | time signature, section form + dynamic arc, chord progressions | `get_theory`, `set_global`, `set_structure`, `set_progression` |
| **Melodist** | per-section melody contour, register, density, syncopation | `design_melody`, `get_theory` |
| **Arranger** | instruments, per-section bass & drum styles | `set_instruments`, `set_bass`, `set_drums` |
| **Critic** | judges the whole piece; approves or bounces back one revision | `analyze_score`, `approve`, `request_revision` |

Every agent also has `analyze_score` (a sensing action) and a terminal `finish`.

---

## Install

Uses [uv](https://docs.astral.sh/uv/). Runs on Windows, macOS and Linux, and CI
tests all three.

```bash
pip install "ark-audio-agent[llm]"   # from PyPI, with the anthropic SDK for the Claude brain
pip install ark-audio-agent          # or offline only
```

From a checkout, `uv sync` installs everything, the anthropic SDK included (see
[Development](#development)).

`ffmpeg` is optional — if present, an MP3 is exported alongside the WAV
(`brew install ffmpeg`, `sudo apt install ffmpeg`, or `winget install Gyan.FFmpeg`).

---

## Usage

### Run with Claude in two steps

```bash
cp .env.example .env     # then fill in ONE provider: ANTHROPIC_API_KEY, or the ANTHROPIC_FOUNDRY_* pair
uv run ark-audio "triumphant sunrise over the mountains"
```

The CLI loads `./.env` on its own, so there is nothing to `export` or `source`.
Check the start-up panel: it should read `brain: claude multi-agent (...)` and
`env: .env`. If it says `deterministic (offline — ...)`, the text after the dash
names what's missing. Use `--env-file path/to/file` to load a different file;
a variable already set in your shell overrides the file.

### CLI

```bash
ark-audio "triumphant sunrise over the mountains"
ark-audio "a quiet dawn over still water" --offline --seed 7
ark-audio "lonely synthwave drive at 2am" --out renders/drive --no-mp3
echo "a storm breaking over the sea" | ark-audio      # reads stdin
```

Flags: `--direction`/`-d` (how to build it, see below), `--direction-file`,
`--reference`/`-r` (music to draw on, repeatable), `--borrow`, `--strength`,
`--no-reference`, `--list-references`, `--text-file`, `--offline` (force the
deterministic brain), `--env-file` (default `./.env`), `--seed`, `--out`, `--model`,
`--effort low|medium|high|xhigh|max`, `--max-revisions`, `--reverb`, `--no-mp3`,
`--spec` (print the JSON tool spec for AI agents). Every flag, its default and how it
interacts with the others is in [docs/AI_SPEC.md](docs/AI_SPEC.md).

### Library

```python
from ark_audio_agent import compose

result = compose("a melancholy rainy afternoon", out_dir="renders/rain", offline=True)
print(result.outputs["wav"], "-", result.score.title)

# text for the mood, direction for the form and constraints
result = compose("For God so loved the world", out_dir="renders/bgm",
                 direction="short intro, rising notes into the verse, a climax, "
                           "then a falling melody and a few seconds of outro")
for check in result.direction_checks:
    print("✓" if check["ok"] else "✗", check["requirement"], "-", check["detail"])

# reference music: a library id/title or a .mid path; borrow picks what to take
result = compose("a rainy afternoon", references=["satie-gymnopedie-1"],
                 borrow=["harmony", "melody"], strength=0.6, offline=True)
print(result.inspiration_checks)   # how closely each borrowed aspect was followed
```

The library doesn't read `.env` (the CLI does). Load it with
`ark_audio_agent.envfile.load_env_file(".env")`, and check `result.brain_kind` to see which
brain ran.

### Musical direction

Two inputs, with separate jobs:

- **The text** (the positional prompt) sets the **mood**: key, mode, tempo, genre,
  instruments.
- **The direction** (`--direction`) sets **how the piece is built**. The engine
  treats it as requirements and lets the agents be creative inside them.

A rule-based reader turns the direction into a plan the agents can see, so it
works offline as well. With the Claude brain, the Interpreter reviews that plan
against your words and corrects it with `set_direction`. It understands:

| You write | It becomes |
|---|---|
| "a few secs of intro", "a 6 second intro", "intro of 2 bars", "no intro" | an `intro` section of ~4 s / 6 s / 2 bars, or none |
| "rising notes", "build up", "crescendo" | a `rise`: an ascending run with a crescendo, leading in |
| "main part of the melody", "verse", "chorus", "bridge" | the main sections, in that order |
| "climax", "peak" | a `climax` that holds the piece's highest note |
| "falling melody", "wind down" | a `fall`: a descending line that resolves to the tonic |
| "outro", "end it", "fade out", "long outro" | an `outro` that settles on the tonic and fades |
| "fewer long leaps", "smooth" / "stepwise", "no leaps" | no interval wider than a fourth / a third |
| "leaning toward one chord and tone", "drone", "grounded" | a tonal anchor: tonic-heavy harmony, a tonic pedal bass, a melody drawn to the tonic |
| "structured", "memorable phrases", "4-bar phrases" | phrase-structured melody: motifs repeated as A A′ B A′ |
| "bgm", "background music for …" | a lighter arrangement (fewer notes, soft or no drums) |
| "in D minor", "90 bpm", "slow", "45 seconds long" | key, tempo and length |

Durations given in seconds are converted to whole bars at the chosen tempo.

**How the engine enforces it.** The melody engine guarantees the hard limits:

- No leap is wider than the limit, and that includes the step from one section
  into the next.
- Rises and falls move one way.
- The climax reaches above everything before it, and nothing after it goes higher.
- Cadences resolve to the tonic.

Then `analyze_score` measures the realized piece against each requirement. The
Critic sends any unmet one back to the agent responsible, and the CLI and
`report.md` show the result:

```
direction: 11/11 requirements met
  ✓ form follows the plan        sections: intro -> rise -> verse -> climax -> fall -> outro
  ✓ intro lasts ~4s              3.3s (1 bars at 73 BPM)
  ✓ rise rises                   G4 -> D5, trend +5.0 semitones
  ✓ climax holds the highest note
  ✓ fall falls                   C6 -> C5, trend -10.5 semitones
  ✓ no leap larger than 3 scale steps
  ✓ harmony leans on the tonic chord   tonic on 58% of bars (need 45%)
  ✓ melody ends on the tonic     last note C5
  ...
```

### Reference music

A third input, with its own job: **music to draw on**. A reference is a set of
traits, not notes:

- a tonality, a tempo and meter, and a form;
- chord progressions as Roman numerals, so they move to any key;
- melodic traits plus the interval shape of a motif;
- an instrument palette and a groove.

The library ships 23 references:

- **Public-domain works:** Pachelbel's Canon, Satie's Gymnopédie No. 1, Bach's
  Prelude in C and Toccata in D minor, Beethoven's Moonlight Sonata, Ode to Joy and
  Für Elise, Debussy's Clair de lune, Grieg's Mountain King and Morning Mood, Amazing
  Grace, Greensleeves, Scarborough Fair and the Dies Irae.
- **Genre archetypes**, which describe a style rather than any one song: pop anthem,
  12-bar blues, synthwave, lo-fi, epic trailer, ambient drone, flamenco, celtic reel
  and worship ballad.

```bash
ark-audio --list-references            # the whole library
ark-audio --list-references rainy piano   # ranked for those words
```

Choose what to take with `--borrow`, and how closely with `--strength` (0..1):

| Aspect | Takes |
|---|---|
| `tonality` | key and mode |
| `tempo` | tempo and meter (how to get 3/4 or 6/8) |
| `form` | the section layout and its intensity arc |
| `harmony` | the progressions per section, moved into the piece's key and mode (I–V–vi–IV becomes i–v–VI–iv in minor) |
| `melody` | contour, density, phrase length, and the motif's interval shape, which the engine re-rhythms and answers with its inversion |
| `instruments` | the palette |
| `groove` | bass and drum styles |

By default a reference you name lends form, harmony, melody, instruments and groove,
but not tonality or tempo, because the text sets the mood. What you set is locked
against the agents.

Without `-r`, the Interpreter may pick a reference itself when one clearly fits the
text. Offline it pins the best match at strength 0.5; `--no-reference` turns this off.

Point `-r` at a `.mid` file and it is read into the same traits: key, tempo, meter,
one chord per bar, melodic traits and a motif, palette, bass and drum styles. The
reader only knows major and minor, so a modal piece comes back as the major or minor
key with the same notes.

The precedence is **direction > text > references**. A direction's sections replace
a reference's form, and a tonal anchor keeps the harmony tonic-heavy. `analyze_score`
and `report.md` show an *echo* per borrowed aspect: how closely the piece follows its
reference, from 0 to 1. It is a measure, not a target.

### Outputs

Each run gets its own folder, `renders/<YYYYMMDD-HHMMSS>_<slug of the text>`, so
runs sort by time and never overwrite each other (`--out` or `out_dir=` picks
a folder yourself). It contains:

| File | What |
|---|---|
| `song.mid` | Standard MIDI file (open in any DAW / soundfont player) |
| `song.wav` | 44.1 kHz stereo audio from the built-in synth |
| `song.mp3` | 192 kbps MP3 (if `ffmpeg` is installed) |
| `score.json` | the full declarative Score (reload / tweak / re-render) |
| `brief.json` | compact human/LLM summary of the piece |
| `trace.jsonl` | every agent step: thought · action · observation |
| `report.md` | a readable write-up: which steps an LLM made and which were deterministic, every setting used (brain, model, effort, thinking, tokens, seed, revisions, references, reverb, versions), the brief, the direction and how it was met, the references and what was borrowed, the structure, and the agent trace |

---

## The two brains

**Claude multi-agent** (default when an API key is set). Each agent runs
the canonical Messages-API tool-use loop with adaptive thinking; the model
reasons, calls a tool, reads the observation, and continues until it finishes.
Defaults to `claude-opus-5-5`. The API call degrades gracefully across
SDK/account differences (server-side refusal fallback → plain streaming → drop
effort → drop thinking), so the run never hard-fails on a capability mismatch.

```bash
# .env
ANTHROPIC_API_KEY=sk-ant-...
```
```bash
ark-audio "a cathedral of light at the edge of the world" --effort high
```

**Claude on Microsoft Foundry (Azure).** Point the brain at a Foundry resource
instead and it uses the SDK's `AnthropicFoundry` client with API-key auth. A
Foundry endpoint takes precedence over `ANTHROPIC_API_KEY`:

```bash
# .env
ANTHROPIC_FOUNDRY_API_KEY=...
ANTHROPIC_FOUNDRY_RESOURCE=my-resource   # or ANTHROPIC_FOUNDRY_BASE_URL=https://my-resource.services.ai.azure.com/anthropic/
ARK_MODEL=claude-opus-5-5                # your Foundry deployment name
```

If both `ANTHROPIC_FOUNDRY_RESOURCE` and `ANTHROPIC_FOUNDRY_BASE_URL` are set,
the base URL is used.

Server-side refusal fallback is a first-party API feature, so it is skipped on
Foundry. Adaptive thinking and effort still apply.

**Deterministic** (offline, always available). A rule-based planner maps the
text to a feeling via a lexicon, then applies music-theory heuristics to make the
same tool calls an LLM would. Reproducible given a `--seed`. This is what runs in
CI and what powers `--offline`.

> If no key is found (and `--offline` isn't set), the CLI automatically uses the
> deterministic brain and says so.

---

## How the music is realized

- **Theory** (`theory.py`): scales/modes, diatonic chords, Roman-numeral
  resolution, scale-degree → MIDI, equal-temperament frequencies.
- **Blackboard** (`blackboard.py`): the declarative `Score` the agents edit.
- **Direction** (`direction.py`): reads a direction prompt into a section plan
  and constraints, and checks a realized piece against it.
- **References** (`references.py`, `midi_analysis.py`): the reference library,
  its search and name resolution, the measure of how closely a piece echoes its
  references, and the reader that turns a MIDI file into reference traits.
- **Spec** (`spec.py`): the library as tools for another model, with the
  definitions, a `run_tool` dispatcher, the recommended system prompt and the full
  machine-readable contract.
- **Performance** (`performance.py`): realizes the Score into notes:
  - a contour-driven melody that snaps to chord tones on strong beats, or a
    phrase-structured melody when a direction (or the Melodist) asks for one.
    That melody is built from motifs, keeps leaps within a limit, leans on the
    tonic, flows across sections and ends on a cadence;
  - intensity ramps within a section;
  - chord comping or sustained pads;
  - several bass styles (roots, root-fifth, pulse, walking, arp, tonic pedal);
  - synthesized drum patterns (four-on-floor, backbeat, breakbeat, a riser, …).

  All of it is seeded and reproducible.
- **Synth** (`synth.py`): additive (band-limited) oscillator voices with ADSR,
  per-instrument patches, synthesized percussion, an FFT-convolution reverb, and
  a soft-clipping master bus.

---

## Development

### First-time setup (virtual environment)

You need **Python 3.10+** (the repo pins 3.12 in `.python-version`). On macOS
the system `python3` is 3.9, which is too old, so use the Python that uv
installs or one from Homebrew.

**Option A: uv (recommended)**

```bash
git clone https://github.com/ir-dev/ark-audio-agent.git
cd ark-audio-agent
uv sync                      # creates .venv: the package (editable), anthropic, pytest
source .venv/bin/activate    # Windows: .venv\Scripts\activate
```

If Python 3.12 is missing, uv downloads it. `uv sync` also installs the `dev`
group (pytest and the anthropic SDK) by default, so a later `uv sync` never
drops the Claude brain.

**Option B: plain `venv` + pip**

```bash
git clone https://github.com/ir-dev/ark-audio-agent.git
cd ark-audio-agent
python3.12 -m venv .venv     # any Python 3.10+
source .venv/bin/activate    # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
pip install -e ".[llm]" pytest
```

`-e` is an editable install, so changes under `src/` take effect without
reinstalling.

### Check it works

```bash
pytest -q                                                 # theory + pipeline tests (offline, no key needed)
ark-audio "a quiet dawn over still water" --offline --seed 7
```

With the venv activated, `ark-audio` and `pytest` run directly. Without
activating it, prefix them with `uv run`.

### API key (optional)

A key is only needed for the Claude brain. Copy the template and fill in **one**
provider:

```bash
cp .env.example .env         # .env is git-ignored; never commit it
```

The CLI reads `./.env` itself, on every OS, so just run it from the repo root:

```bash
uv run ark-audio "a storm breaking over the sea"
```

## License

MIT.
