Metadata-Version: 2.3
Name: ark-audio-agent
Version: 0.1.7
Summary: Autonomous text-to-music composer driven by a multi-agent ReAct harness (Claude API, with an offline deterministic fallback).
Keywords: music,generative,agents,react,claude,midi,synthesizer
Author: Immanuel R
Author-email: Immanuel R <raj@immanuel.co>
License: MIT
Requires-Dist: numpy>=1.24
Requires-Dist: mido>=1.3
Requires-Dist: rich>=13.7
Requires-Dist: anthropic>=0.107.1 ; extra == 'llm'
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/ir-dev/ark-audio-agent
Project-URL: Repository, https://github.com/ir-dev/ark-audio-agent
Project-URL: Issues, https://github.com/ir-dev/ark-audio-agent/issues
Provides-Extra: llm
Description-Content-Type: text/markdown

# ark-audio-agent

**Autonomous text → music.** Give it a sentence; it reasons about the feeling,
composes a structured piece, and renders it to MIDI + audio — with no human in
the loop.

The interesting part is the *harness*: composition is driven by a team of
specialist agents running a **ReAct loop** (reason → act → observe) over a shared
musical blackboard. It runs on the **Claude API** when a key is available, and on
a **fully-offline deterministic music-theory brain** otherwise, so it always
produces a result.

```bash
ark-audio "a rainy night in a neon city, lonely but hopeful"
# → renders/a-rainy-night-in-a-neon-city-lonely-but-hopeful/song.mp3  (+ .wav, .mid, report.md, trace.jsonl)
```

Add a **direction** to say how the piece must be built, in plain words. The text
still sets the mood:

```bash
ark-audio "The Lord is my shepherd; I shall not want." \
  --direction "a few secs of intro and a bit of rising notes before the main melody, \
  structured, fewer long leaps, strongly leaning toward one chord and tone; after the \
  climax a falling melody, then a few sec of outro to end it"
```

---

## Why it's built this way

Rather than ask one model to emit raw notes (which drifts out of key and is hard
to steer), the system separates **musical intent** from **musical realization**:

- **Agents decide intent** — key, mode, tempo, song form, chord progressions, a
  melody's contour/register/density, instrument roles, drum and bass styles.
- **Deterministic engines realize it** — pure music-theory functions turn that
  intent into in-key notes, which a `mido` writer turns into MIDI and a
  from-scratch `numpy` synthesizer turns into audio.

Because every decision an agent can make is a **tool**, both brains (Claude and
the offline planner) drive the exact same action surface and produce the exact
same trace format.

### Architecture

```
            text prompt
                │
     ┌──────────▼───────────────────────────────────────────────┐
     │  Orchestrator (the harness)                                │
     │                                                            │
     │   Interpreter → Composer → Melodist → Arranger             │  each agent runs a
     │                               │                            │  ReAct loop:
     │                               ▼                            │   reason (thinking)
     │                            Critic ──reject──┐              │   → act  (tool call)
     │                               │approve      │ one targeted │   → observe (result)
     │                               │        revision (bounded)  │
     └───────────────────────────────┼────────────┘              │
                                      │                           │
          ┌───────────────────────────▼───────────────────────┐  │
          │  Tools  (the ReAct action space)                   │  │
          │  set_brief · set_direction · get_theory ·          │  │
          │  set_global ·                                      │  │
          │  set_structure · set_progression · design_melody · │  │
          │  set_instruments · set_bass · set_drums ·          │  │
          │  analyze_score · finish / approve / request_revision│ │
          └───────────────────────────┬───────────────────────┘  │
                                       ▼                          │
                            Score blackboard (shared state)       │
     └──────────────────────────────┬─────────────────────────────┘
                                     ▼
          performance.py  →  { melody, harmony, bass, drums } notes
                                     │
                 ┌───────────────────┴────────────────────┐
                 ▼                                         ▼
        midi_render.py (mido)                   synth.py (numpy software synth)
                 │                                         │
              song.mid                        song.wav ──(ffmpeg)──► song.mp3
```

### The agents

| Agent | Decides | Tools |
|---|---|---|
| **Interpreter** | mood, imagery, genre, key/mode/tempo, emotional arc; reviews the direction's plan | `set_brief`, `set_title`, `set_direction` |
| **Composer** | time signature, section form + dynamic arc, chord progressions | `get_theory`, `set_global`, `set_structure`, `set_progression` |
| **Melodist** | per-section melody contour, register, density, syncopation | `design_melody`, `get_theory` |
| **Arranger** | instruments, per-section bass & drum styles | `set_instruments`, `set_bass`, `set_drums` |
| **Critic** | judges the whole piece; approves or bounces back one revision | `analyze_score`, `approve`, `request_revision` |

Every agent also has `analyze_score` (a sensing action) and a terminal `finish`.

---

## Install

Uses [uv](https://docs.astral.sh/uv/). Runs on Windows, macOS and Linux, and CI
tests all three.

```bash
uv sync --extra llm      # installs the optional anthropic SDK for the Claude brain
# or, without the Claude brain (offline only):
uv sync
```

`ffmpeg` is optional — if present, an MP3 is exported alongside the WAV
(`brew install ffmpeg`, `sudo apt install ffmpeg`, or `winget install Gyan.FFmpeg`).

---

## Usage

### CLI

```bash
ark-audio "triumphant sunrise over the mountains"
ark-audio "a quiet dawn over still water" --offline --seed 7
ark-audio "lonely synthwave drive at 2am" --out renders/drive --no-mp3
echo "a storm breaking over the sea" | ark-audio      # reads stdin
```

Flags: `--direction`/`-d` (how to build it, see below), `--direction-file`,
`--text-file`, `--offline` (force the deterministic brain), `--seed`, `--out`,
`--model`, `--effort low|medium|high|xhigh|max`, `--max-revisions`, `--reverb`,
`--no-mp3`.

### Library

```python
from ark_audio_agent import compose

result = compose("a melancholy rainy afternoon", out_dir="renders/rain", offline=True)
print(result.outputs["wav"], "-", result.score.title)

# text for the mood, direction for the form and constraints
result = compose("For God so loved the world", out_dir="renders/bgm",
                 direction="short intro, rising notes into the verse, a climax, "
                           "then a falling melody and a few seconds of outro")
for check in result.direction_checks:
    print("✓" if check["ok"] else "✗", check["requirement"], "-", check["detail"])
```

### Musical direction

Two inputs, with separate jobs:

- **The text** (the positional prompt) sets the **mood**: key, mode, tempo, genre,
  instruments.
- **The direction** (`--direction`) sets **how the piece is built**. The engine
  treats it as requirements and lets the agents be creative inside them.

A rule-based reader turns the direction into a plan the agents can see, so it
works offline as well. With the Claude brain, the Interpreter reviews that plan
against your words and corrects it with `set_direction`. It understands:

| You write | It becomes |
|---|---|
| "a few secs of intro", "a 6 second intro", "intro of 2 bars", "no intro" | an `intro` section of ~4 s / 6 s / 2 bars, or none |
| "rising notes", "build up", "crescendo" | a `rise`: an ascending run with a crescendo, leading in |
| "main part of the melody", "verse", "chorus", "bridge" | the main sections, in that order |
| "climax", "peak" | a `climax` that holds the piece's highest note |
| "falling melody", "wind down" | a `fall`: a descending line that resolves to the tonic |
| "outro", "end it", "fade out", "long outro" | an `outro` that settles on the tonic and fades |
| "fewer long leaps", "smooth" / "stepwise", "no leaps" | no interval wider than a fourth / a third |
| "leaning toward one chord and tone", "drone", "grounded" | a tonal anchor: tonic-heavy harmony, a tonic pedal bass, a melody drawn to the tonic |
| "structured", "memorable phrases", "4-bar phrases" | phrase-structured melody: motifs repeated as A A′ B A′ |
| "bgm", "background music for …" | a lighter arrangement (fewer notes, soft or no drums) |
| "in D minor", "90 bpm", "slow", "45 seconds long" | key, tempo and length |

Durations given in seconds are converted to whole bars at the chosen tempo.

**How the engine enforces it.** The melody engine guarantees the hard limits:

- No leap is wider than the limit, and that includes the step from one section
  into the next.
- Rises and falls move one way.
- The climax reaches above everything before it, and nothing after it goes higher.
- Cadences resolve to the tonic.

Then `analyze_score` measures the realized piece against each requirement. The
Critic sends any unmet one back to the agent responsible, and the CLI and
`report.md` show the result:

```
direction: 11/11 requirements met
  ✓ form follows the plan        sections: intro -> rise -> verse -> climax -> fall -> outro
  ✓ intro lasts ~4s              3.3s (1 bars at 73 BPM)
  ✓ rise rises                   G4 -> D5, trend +5.0 semitones
  ✓ climax holds the highest note
  ✓ fall falls                   C6 -> C5, trend -10.5 semitones
  ✓ no leap larger than 3 scale steps
  ✓ harmony leans on the tonic chord   tonic on 58% of bars (need 45%)
  ✓ melody ends on the tonic     last note C5
  ...
```

### Outputs

Each run writes, into the output directory:

| File | What |
|---|---|
| `song.mid` | Standard MIDI file (open in any DAW / soundfont player) |
| `song.wav` | 44.1 kHz stereo audio from the built-in synth |
| `song.mp3` | 192 kbps MP3 (if `ffmpeg` is installed) |
| `score.json` | the full declarative Score (reload / tweak / re-render) |
| `brief.json` | compact human/LLM summary of the piece |
| `trace.jsonl` | every agent step: thought · action · observation |
| `report.md` | a readable write-up of the brief, the direction and how it was met, the structure, and the agent trace |

---

## The two brains

**Claude multi-agent** (default when an API key is set). Each agent runs
the canonical Messages-API tool-use loop with adaptive thinking; the model
reasons, calls a tool, reads the observation, and continues until it finishes.
Defaults to `claude-opus-5-5`. The API call degrades gracefully across
SDK/account differences (server-side refusal fallback → plain streaming → drop
effort → drop thinking), so the run never hard-fails on a capability mismatch.

```bash
export ANTHROPIC_API_KEY=sk-ant-...
ark-audio "a cathedral of light at the edge of the world" --effort high
```

**Claude on Microsoft Foundry (Azure).** Point the brain at a Foundry resource
instead and it uses the SDK's `AnthropicFoundry` client with API-key auth. A
Foundry endpoint takes precedence over `ANTHROPIC_API_KEY`:

```bash
export ANTHROPIC_FOUNDRY_API_KEY=...
export ANTHROPIC_FOUNDRY_RESOURCE=my-resource   # or ANTHROPIC_FOUNDRY_BASE_URL=https://my-resource.services.ai.azure.com/anthropic/
export ARK_MODEL=claude-opus-5-5                # your Foundry deployment name
ark-audio "a cathedral of light at the edge of the world"
```

Server-side refusal fallback is a first-party API feature, so it is skipped on
Foundry. Adaptive thinking and effort still apply.

**Deterministic** (offline, always available). A rule-based planner maps the
text to a feeling via a lexicon, then applies music-theory heuristics to make the
same tool calls an LLM would. Reproducible given a `--seed`. This is what runs in
CI and what powers `--offline`.

> If no key is found (and `--offline` isn't set), the CLI automatically uses the
> deterministic brain and says so.

---

## How the music is realized

- **Theory** (`theory.py`): scales/modes, diatonic chords, Roman-numeral
  resolution, scale-degree → MIDI, equal-temperament frequencies.
- **Blackboard** (`blackboard.py`): the declarative `Score` the agents edit.
- **Direction** (`direction.py`): reads a direction prompt into a section plan
  and constraints, and checks a realized piece against it.
- **Performance** (`performance.py`): realizes the Score into notes:
  - a contour-driven melody that snaps to chord tones on strong beats, or a
    phrase-structured melody when a direction (or the Melodist) asks for one.
    That melody is built from motifs, keeps leaps within a limit, leans on the
    tonic, flows across sections and ends on a cadence;
  - intensity ramps within a section;
  - chord comping or sustained pads;
  - several bass styles (roots, root-fifth, pulse, walking, arp, tonic pedal);
  - synthesized drum patterns (four-on-floor, backbeat, breakbeat, a riser, …).

  All of it is seeded and reproducible.
- **Synth** (`synth.py`): additive (band-limited) oscillator voices with ADSR,
  per-instrument patches, synthesized percussion, an FFT-convolution reverb, and
  a soft-clipping master bus.

---

## Development

### First-time setup (virtual environment)

You need **Python 3.10+** (the repo pins 3.12 in `.python-version`). On macOS
the system `python3` is 3.9, which is too old, so use the Python that uv
installs or one from Homebrew.

**Option A: uv (recommended)**

```bash
git clone https://github.com/ir-dev/ark-audio-agent.git
cd ark-audio-agent
uv sync --extra llm          # creates .venv: the package (editable), anthropic, pytest
source .venv/bin/activate    # Windows: .venv\Scripts\activate
```

If Python 3.12 is missing, uv downloads it. `uv sync` also installs the `dev`
group (pytest) by default.

**Option B: plain `venv` + pip**

```bash
git clone https://github.com/ir-dev/ark-audio-agent.git
cd ark-audio-agent
python3.12 -m venv .venv     # any Python 3.10+
source .venv/bin/activate    # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
pip install -e ".[llm]" pytest
```

`-e` is an editable install, so changes under `src/` take effect without
reinstalling.

### Check it works

```bash
pytest -q                                                 # theory + pipeline tests (offline, no key needed)
ark-audio "a quiet dawn over still water" --offline --seed 7
```

With the venv activated, `ark-audio` and `pytest` run directly. Without
activating it, prefix them with `uv run`.

### API key (optional)

A key is only needed for the Claude brain. Copy the template and fill in **one**
provider:

```bash
cp .env.example .env         # .env is git-ignored; never commit it
```

The CLI does not read `.env` by itself, so load it when you run:

```bash
uv run --env-file .env ark-audio "a storm breaking over the sea"     # any OS
# or export it into the current shell (bash/zsh; works with either option):
set -a; source .env; set +a
ark-audio "a storm breaking over the sea"
```

## License

MIT.
