Metadata-Version: 2.4
Name: solace-researcher
Version: 0.1.1
Summary: Autonomous multi-agent research system with persistent per-agent state and cross-agent corroboration.
License: Proprietary
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Requires-Dist: openai>=1.0
Requires-Dist: tavily-python>=0.5
Requires-Dist: python-dotenv>=1.0
Requires-Dist: requests>=2.28
Requires-Dist: flask>=3.0
Provides-Extra: faiss-cpu
Requires-Dist: faiss-cpu; extra == "faiss-cpu"
Provides-Extra: faiss-gpu
Requires-Dist: faiss-gpu-cu12; extra == "faiss-gpu"

# Solace Researcher

A continuous, tick-driven research agent. No chat turns — it runs as a daemon, persists its own state to disk, works through a task queue, proposes its own next tasks when the queue empties, and periodically reviews its own recent self-directed decisions to steer future ones.

## Quickstart

```
pip install .
export GROQ_API_KEY=your_groq_key      # console.groq.com
export TAVILY_API_KEY=your_tavily_key  # tavily.com
solace-dashboard
```

`pip install .` puts `solace-dashboard` (and `solace-researcher`, the terminal-only version) on your PATH — run either from anywhere afterward, no need to stay in this folder. `solace-dashboard` opens a local web page at `http://127.0.0.1:5055`: pick agent count (1–5), topics, and duration in the form, hit Start, and watch tick/claim/verification counts update live per agent. Ctrl+C in the terminal stops the server; running it again against the same output directory resumes from saved state instead of repeating research.

*(First install creates a throwaway `build/` folder and `*.egg-info` folder next to the source — that's normal pip/setuptools output, not part of the project. Safe to delete; already git-ignored.)*

That's everything needed to run it. Everything below this line is internals — useful for extending the code, not required to use it.

---

## Other ways to run it

### Terminal-only version (no dashboard)

```
solace-researcher
```

Same underlying run, `input()` prompts instead of a web form.

### As a library

`RunConfig`/`run`/`run_single` are exported at the package's top level — nothing in `agent.py`/`hub.py`/`llm_functions.py` needs to be read or touched:

```python
from solace_researcher import RunConfig, run

config = RunConfig(
    groq_api_key="gsk_...",
    tavily_api_key="tvly-...",
    topics=["stateful agent architectures", "vector database benchmarks"],
    num_agents=4,             # defaults to len(topics) if omitted -- one shard per topic
    duration_minutes=10,      # wall-clock budget per shard
)
hub, results = run(config)

print(hub.summary())
for claim, shards in hub.corroborated_claims(min_shards=2):
    print(f"[{claim.confidence:.2f}] ({', '.join(shards)}) {claim.text}")
```

`run_single(config)` is the same idea for one agent instead of a shard pool. See `RunConfig` in `sdk.py` for every field (feature toggles included — vector memory, code repair, iteration limits).

This is a config layer over fully open code, not a way of hiding it — everything `sdk.py` calls is the same public code the rest of this README documents. If you want the daemon (runs forever, resumable, no time budget) instead of a bounded run, see below.

### As a daemon (continuous, resumable, no time budget)

```
export GROQ_API_KEY=your_groq_key
export TAVILY_API_KEY=your_tavily_key
python3 -m solace_researcher.main
```

**Security:** never commit real keys to this repo. Both are read from environment variables only. If a key was ever pasted anywhere outside your own terminal/environment (a chat, a screenshot, a doc), rotate it.

Runs forever. Ctrl+C shuts down gracefully and force-saves state first.

State persists in `agent_state.json`. Notes accumulate in `research_notes/research_log.md` (human-readable) and `research_notes/research_log.jsonl` (structured).

### Offline demo (no API keys, no network, mocked calls)

```
python3 -m solace_researcher.agent
```

Self-direction, claim extraction, and self-reflection are all off by default in the demo (their `_fn` params default to `None` → old, simpler behavior, documented per-feature below). Useful for checking the plumbing before spending API credits.

## How it works (internals)

### What's actually autonomous


- **Search grounding**: `real_reasoning` and `real_self_direct` search via Tavily, then have Groq synthesize an answer from the results — plain search-API + inference, not a separate browsing-agent product. Every sentence in the synthesized answer is tagged inline as it's generated: `[SOURCE: <url>]` if it's a fact taken directly from a retrieved result, `[ESTIMATE]` if it's the model's own inference. Those tags are what `real_extract_claims` (below) reads.
- **Structured claim extraction**: task results used to collapse into one claim at a flat 0.7/0.3 confidence. `real_extract_claims` now splits a result into multiple individual claims, each carrying `verified: bool` and `source_url: str | None` — a claim backed by a real `[SOURCE: url]` tag is stored differently from one that's an `[ESTIMATE]`, instead of both landing in the same bucket. No extractor wired → falls back cleanly to the old single-claim behavior.
- **Self-direction**: when the task queue empties, the agent calls `real_self_direct`, which proposes its own next research task from its open hypotheses' unknowns and recent findings — no human has to queue the next thing.
- **Self-reflection, first order**: every few ticks, `_self_reflect` reviews `reflection_log` — a record of the agent's own recent self-proposed tasks and how they turned out (success, verified-claim ratio) — and produces a `reflection_bias` via `real_self_reflect`. That bias is fed back into the context for the *next* self-directed proposal. So the loop is: agent proposes a task → task outcome gets logged → agent reviews its own log → that review shapes the next proposal. Verified end-to-end in testing (a later self-direction call demonstrably receives the prior reflection's assessment as context).

- **Self-reflection, second order (calibration)**: first-order reflection only judges raw task outcomes. `_meta_reflect` goes one level further — it reviews `reflection_history` (a trace of the agent's own *past reflections*, not raw tasks) against what actually happened afterward, and judges whether those past self-assessments were well-calibrated. The result, `meta_confidence`, then *scales* how much a new first-order reflection is allowed to move `reflection_bias` — `effective_delta = raw_delta * meta_confidence`. A track record of badly-calibrated self-judgment measurably damps the agent's own future self-judgment instead of letting it compound unchecked. Tested end to end: a run with deliberately miscalibrated mock reflections drove `meta_confidence` down and visibly shrank later `reflection_bias` moves compared to earlier ones.

  **Precision on both layers**, because it matters if you're reading the code and not just this file: `real_self_reflect` and `real_meta_reflect` are structurally identical to `real_judge` or `real_support_judge` — format some text, send it to Groq, parse the JSON that comes back. The *feedback loops* around them are real and tested (output of past decisions measurably changes future decisions, and a calibration check measurably changes how much a new judgment is trusted). What this is not is introspective self-awareness or reasoning about its own reasoning process in the phenomenal sense — there's no representation of *why* the agent believes something that it can inspect independent of these scored logs. This is the same family of technique as Reflexion-style agents (verbal self-critique with memory feeding into future decisions) — a real, literature-grounded pattern, not a claim about subjective experience. Call it a two-level self-directed feedback loop, not self-recursive thinking.

- **Everything already covered**: task → oracle review → code-gate → (genesis+evaluator | reasoning) → claim(s) recorded in knowledge store → evidence applied to hypotheses per-claim → staggered contradiction sweep → staggered self-reflection → notes written — all still true, all still tested.

### State vector (`state.py`, 13 dimensions)

Two derived scalars come off this vector each tick: `binding` (coherence — is the vector's norm stable or extreme) and `self_model` (surprise — distance between current state and the running memory mean, decayed). All 13 raw dimensions are wired to something real; none are placeholders.

| # | Field | What it actually measures |
|---|---|---|
| 0 | `backlog_pressure` | pending tasks / capacity |
| 1 | `recent_failure_rate` | failures in last N tasks / N |
| 2 | `vitality` | `1 - recent_failure_rate` |
| 3 | `idle_ticks_norm` | ticks since last task, normalized |
| 4 | `novelty` | embedding distance from recent task history |
| 5 | `cumulative_success` | lifetime success rate |
| 6 | `consecutive_failures_norm` | current failure streak, normalized |
| 7 | `avg_duration_norm` | execution time trend |
| 8 | `self_direction_ratio` | of recent finished tasks, fraction the agent proposed itself vs. was assigned |
| 9 | `verified_claim_ratio` | of recent claims, fraction that came back `verified=True` (real `source_url`) vs. estimate |
| 10 | `contradiction_pressure` | of concepts touched in the last sweep, fraction with an active conflict |
| 11 | `hypothesis_revision_rate` | how often a hypothesis's status actually *flipped* (not just accumulated evidence) in the recent window, per hypothesis tracked |
| 12 | `meta_reflection_delta` | size of the last change to `reflection_bias` from `_self_reflect` |

Slots 8–12 depend on `extract_claims_fn` and `self_reflect_fn` being wired (they are, in `main.py`; they aren't in the offline demo) — without them those slots stay at 0, same fallback pattern as everything else here.

### Token/cost tracking and benchmark graphs

Every LLM and search call is logged to `research_notes/token_usage.jsonl` as it happens (crash-safe, same append-immediately pattern as the research log itself) — tagged with which function made the call (`oracle`, `genesis`, `reasoning`, `judge`, `support_judge`, `self_direct`, `extract_claims`, `self_reflect`, `meta_reflect`), tick number, prompt/completion tokens, latency, and cost in USD.

```
pip install matplotlib   # not in requirements.txt -- benchmarking tool, not a runtime dep
python3 analyze_metrics.py                        # defaults to ./research_notes/token_usage.jsonl
python3 analyze_metrics.py path/to/token_usage.jsonl
```

Produces a per-call-site summary table, a CSV (`token_usage_summary.csv`), and three PNGs: cumulative cost over the run, tokens by call site, and latency distribution by call site — the actual artifacts for a benchmark writeup, not just numbers in a terminal.

Cost math uses `GROQ_PRICING` in `metrics.py`, currently openai/gpt-oss-120b at $0.15/M input, $0.60/M output tokens. Verify against [groq.com/pricing](https://groq.com/pricing) before trusting the dollar figures for anything investor- or reviewer-facing — pricing pages change, this was current as of when the file was written, not guaranteed current when you read it. Tavily search cost is a rough paid-tier estimate for the same reason; verify at tavily.com/pricing.

### Vector memory (semantic search over claims)

`knowledge.py`'s `SemanticStore` buckets claims by concept for exact lookups — that's cheap and still how contradiction checking works. On top of that, `vector_memory.py` adds real cross-concept semantic retrieval: `store.semantic_search("some query", k=5)` returns claims ranked by similarity to the query regardless of which concept they got filed under, backed by an actual similarity index instead of a dict scan.

**Backend, auto-selected in this order:**
1. **FAISS + GPU** — if `faiss` is installed and a GPU is visible (`faiss.get_num_gpus() > 0`). `IndexFlatIP` moved onto the GPU via `index_cpu_to_gpu` — real CUDA-accelerated search.
2. **FAISS on CPU** — if `faiss` is installed but no GPU.
3. **Pure NumPy** — if `faiss` isn't installed at all. Brute-force cosine similarity. Mathematically identical results to the FAISS backends at this scale (thousands of claims), just not index-accelerated.

Nothing else in the codebase needs to know or care which backend is active — same `search()` call, same return shape. Check `agent.vector_memory.backend` at runtime to see which one you're actually running on.

Wired into `_self_seed`: when the agent proposes its own next task, it now pulls claims that are *semantically* relevant to its open unknowns, not just the most recent claims within the same concept bucket — a claim filed under a different concept can still surface if it's actually relevant.

**On embedding quality, honestly:** `embeddings.py`'s `embed()` is a hashed bag-of-words (the "hashing trick") — a lexical/token-overlap proxy for similarity, not a trained semantic embedding model. It catches shared vocabulary, not shared meaning; two sentences about the same thing using different words won't necessarily land close together. Good enough to keep near-duplicate tasks from colliding and to give `semantic_search` real signal over exact-bucket lookup — not a substitute for a real sentence-embedding model if you need genuine paraphrase-level recall. Swapping in something like `sentence-transformers` locally is a drop-in replacement for just the `embed()` function body; nothing downstream needs to change.

**A real bug found and fixed while building this:** `embed()` originally used Python's built-in `hash()` on tokens, which is randomized per-process by default (`PYTHONHASHSEED`, a security feature — `hash("foo")` returns a *different* number in two separate `python3` runs). That's invisible within a single run, but this system persists embeddings to disk and reloads them in a new process on every restart. With the built-in `hash()`, a freshly-embedded query after a restart would land in effectively random buckets relative to vectors indexed in the previous process — silently corrupting every similarity comparison across restarts, including the pre-existing novelty-scoring embedding history in `state.py`. Fixed by switching to `hashlib.md5` (stable across processes/machines/Python versions — used for stability, not any cryptographic property) and verified with a same-process-vs-fresh-process test before and after.

### Multi-agent (hub.py)

Runs N independent `SolaceResearcher` shards, each with its own full state (own `agent_state.json`, own tick loop, own reflection history), pooling their claims into one `SharedKnowledgeHub`. Threading, not multiprocessing — shard work is I/O-bound (waiting on Groq/Tavily), so Python releases the GIL during the actual network wait and threads can share the hub's memory directly.

```python
from hub import SharedKnowledgeHub, run_shards

def make_factory(shard_id, topic):
    def factory():
        agent = build_agent()  # from main.py, or your own construction
        agent.notes_dir = f"./shards/{shard_id}"
        agent.queue.add_task(f"Investigate {topic}", source="assigned", concept=topic)
        return agent
    return factory

hub = SharedKnowledgeHub(path="./hub_state")
factories = {"a": make_factory("a", "topic X"), "b": make_factory("b", "topic Y")}
results = run_shards(factories, hub, max_ticks=20)
print(hub.summary())
```

**Corroboration is the actual point**, not just pooling: when two *different* shards independently produce a claim that's semantically close (via `vector_memory.py`'s `semantic_search`, not exact-string matching), that's real signal — independent convergence from separate research paths is stronger evidence than one shard saying something once. `hub.corroborated_claims(min_shards=2)` returns exactly that subset, and a corroborated claim's confidence gets a small bump (capped at 0.95) each time another shard independently reaches it. Tested end to end: three shards, two converging on the same finding from different task text, one unrelated — the hub correctly pooled to 2 distinct claims, flagged exactly one as corroborated, and credited the right two shard IDs.

**A real threading bug found and fixed while building this:** `metrics.py`'s token tracker is a module-level singleton — harmless with one agent, but once multiple shard threads run in the same process, each shard's constructor sets `tracker.path` to its own `notes_dir`. As a plain instance attribute, every shard's write would race to set the same shared value — whichever shard finished constructing last would silently steal every other shard's token logging into its own file, and `current_tick` would reflect whatever thread last set it regardless of which shard actually made the call. Fixed with `threading.local()` for both `path` and `current_tick`, verified with a direct concurrent-write test (5 threads, each asserting it kept its own path/tick throughout).

**On "multiverse"** (the sim-file concept this is loosely inspired by): there's no actual parallel-reality mechanic here, and there wasn't one worth porting — it maps cleanly onto an ensemble of *run configs* instead: different shards given different topic focus, different starting `reflection_bias`, different search depth, then compared via `hub.corroborated_claims()` and `hub.summary()`. That's the honest, useful version of the idea.

### Tests

`tests/` holds standalone regression checks (plain `assert` + `print`, not pytest) for issues that were found and fixed by running this against real usage:

```
python3 tests/run_all.py
```

| Test | What it guards against |
|---|---|
| `test_fix1_hypothesis_linkage.py` | Self-directed tasks used to accumulate claims under `self_directed_research` with no hypothesis attached — their findings never counted as evidence for anything. Now linked to the first open tracked hypothesis. |
| `test_fix2_source_verification.py` | `verify_source_fn` independently checks a claim's `source_url` once, at creation time — downgrades a claim to estimate-tier if the page doesn't actually support it, leaves it alone if verification itself is inconclusive (a fetch failure isn't evidence the claim is wrong). |
| `test_fix3_retry_backoff.py` | `judge_fn` calls in the contradiction sweep now retry with exponential backoff on transient errors, and a pair is only cached as "checked" after a real verdict — not on failure, which used to mean a pair that hit a rate limit was silently skipped forever. |
| `test_fix4_persistence_atomic.py` | `Persistence.save()` writes to a `.tmp` file and atomically replaces the real state file — a crash mid-write can no longer corrupt `agent_state.json` in place. |
| `test_fix5_task_injection.py` | `external_tasks_path` lets another process drop JSON-line tasks into a *running* daemon (`echo '{"objective": "..."}' >> external_tasks.jsonl`) — consumed and truncated each tick, malformed lines skipped without blocking the rest of the batch. |
| `test_fix6_genesis_live_data.py` | `needs_live_data()` detects objectives needing current information and routes a `reasoning_fn` fetch into `genesis_fn`'s context first, instead of `genesis_fn` writing code that tries to hit the network from `evaluator.py`'s bare subprocess (which has none). |
| `test_fix7_code_repair.py` | `evaluator.run_with_repair()`: a missing dependency gets installed and the same code retried; any other error gets handed to `genesis_fn` for a real fix; an unfixable error stops exactly at `max_iterations` instead of looping forever. |

**On fix 2 specifically**, since it caused a real problem worth naming: verification must run exactly ONCE per claim, at the moment the claim is created — never as a recurring sweep over already-verified claims. A sweep-style design re-checks every claim ever produced on every pass, with no per-claim memory of "already handled" — call volume only grows, tick over tick, for the life of the run. `verify_source_fn` is deliberately wired into `_record_outcome` (creation time) rather than alongside the staggered contradiction sweep for exactly this reason.

### Self-healing code execution

`evaluator.py`'s `run_with_repair()` (used by default — `enable_code_repair=True`) iterates on a failed genesis run instead of just logging the failure:

- **Missing dependency** → the module name is extracted from the real `ModuleNotFoundError`, regex-validated to rule out anything that isn't a plain package name, installed via a separate `pip install` subprocess, and the *same* code is retried — no wasted `genesis_fn` call on an error that's already fixed.
- **Any other failure** → the error is handed back to `genesis_fn` along with the original objective, and the revised code is tried instead.
- Bounded by `code_repair_max_iterations` (default 3) either way — won't loop forever on something unfixable.

**Trust boundary, stated plainly:** the generated code itself still runs in a bare subprocess with no network access — that's unchanged. The only new network-capable action is `pip install <name>`, run as its own separate subprocess, and only for a name Python's own import machinery already asked for. Sandboxed execution and unsandboxed dependency resolution are different guarantees; this keeps the first and deliberately relaxes the second. Set `enable_code_repair=False` if you want the original single-attempt behavior back.

### Files

- `sdk.py` — config-driven entry point (`RunConfig`, `run()`, `run_single()`) — set keys/topics/agent count/duration, no internals reading required.
- `quickstart.py` — runnable demo of `sdk.py`, works out of the box once API keys are set.
- `main.py` — the daemon entrypoint (`python3 main.py`, runs forever, resumable). Continuous loop, graceful shutdown, self-direction, claim extraction, first- and second-order self-reflection, source verification, and external task injection all wired in.
- `llm_functions.py` — real Groq + Tavily functions: oracle, genesis, reasoning, judge, support_judge, self_direct, extract_claims, self_reflect, meta_reflect, verify_source.
- `metrics.py` — token/cost/latency tracking, one JSONL row per LLM/search call, tagged by call site and tick. Thread-safe (thread-local path/tick) for multi-agent runs.
- `analyze_metrics.py` — reads `token_usage.jsonl`, produces a summary table, CSV, and benchmark graphs (cumulative cost, tokens by call site, latency by call site).
- `vector_memory.py` — cross-concept semantic search over claims, GPU via FAISS if available, correct numpy fallback if not.
- `hub.py` — `SharedKnowledgeHub` + `run_shards`: multi-agent pooling with semantic dedup and cross-shard corroboration.
- `embeddings.py` — content-based vectorizer (hashed bag-of-words) for novelty scoring and vector memory. Stable across process restarts (hashlib.md5, not Python's randomized built-in hash).
- `agent.py` — the tick loop itself (`SolaceResearcher` class) plus the mocked demo under `__main__`.
- `state.py` — telemetry → binding (coherence) → self_model (surprise/confidence); 13-dimensional, all slots wired, see table above.
- `knowledge.py` — semantic store: concept → claims, each with confidence/verified/source_url.
- `hypotheses.py` — persistent research questions, updated in place across ticks.
- `contradictions.py` — scoped, staggered contradiction detection, with retry/backoff.
- `tasks.py` — priority queue, concept/hypothesis linkage.
- `evaluator.py` — sandboxed execution + real pass/fail verification for generated code, with self-healing iteration (missing-package install, genesis-driven repair) — see below.
- `persistence.py` — atomic save/load of everything above.
- `notes.py` — the only output surface. No chat replies, no return-to-caller.
- `tests/` — regression checks for the fixes in the table above; `tests/run_all.py` runs them all.

### What still needs your input

- `real_genesis` doesn't currently have code-execution-time internet access (it writes code, but that code runs in a bare subprocess with no network) — `needs_live_data()` (see tests table above) covers the common case by fetching first and handing genesis the data as context, but it's a keyword gate, not exhaustive. Widen `LIVE_DATA_KEYWORDS` in `agent.py` if you hit a phrasing it misses.
- Tavily's free tier is 1,000 searches/month — `real_reasoning` and `real_self_direct` both consume it on every grounded call. Fine for testing/demo use; budget for it before running this unattended for long stretches.
- `_self_seed`'s hypothesis linkage picks the first open hypothesis found, not the most relevant one — fine with one active research question at a time, worth revisiting if you're tracking several unrelated hypotheses concurrently.

