# TASK: Build "AutoResearcher", a pip-installable multi-agent research assistant (Python package)

You are building a production-quality open-source Python package called `autoresearcher`.
Users supply their own API keys (Anthropic/Claude, OpenAI, Google Gemini, plus a search key).
Given a research question (e.g. "Chennai trip for 3 days"), the system plans the research,
runs parallel sub-agents that use tools (via MCP servers), verifies findings, and produces a
detailed, cited report as a .docx file.

Work in phases (see "PHASES"). After each phase: run tests, run ruff, fix failures, and
commit-style summarize what was done. Do not skip ahead until the current phase passes.
If something is ambiguous, choose a sensible default, document it in README under "Design decisions", and continue.

## 1. TECH STACK
- Python >= 3.10, `src/` layout, packaging with `pyproject.toml` (hatchling), console script `autoresearcher`.
- Orchestration: LangGraph (use `Send` API for parallel fan-out; reducers via `Annotated[list, operator.add]`).
- LLM providers: `langchain-anthropic`, `langchain-openai`, `langchain-google-genai`.
- Tools: MCP servers written with FastMCP (`mcp` package), consumed via `langchain-mcp-adapters` (`MultiServerMCPClient`).
- Data models & settings: Pydantic v2, `pydantic-settings`.
- DOCX: `python-docx` only (no LLM-written raw docx).
- HTTP: `httpx` (async). Search: Tavily (primary), DuckDuckGo fallback (no key).
- Vector store: pluggable. Pinecone (optional extra) with local Chroma fallback. Define a `VectorStore` protocol.
- API: FastAPI + `sse-starlette`. CLI: Typer + Rich.
- Dev: pytest, pytest-asyncio, respx, ruff, mypy, pre-commit, GitHub Actions CI.
- Optional extras in pyproject: `[pinecone]`, `[chroma]`, `[api]`, `[dev]`, `[all]`.

## 2. PUBLIC API (must work exactly like this)
```python
from autoresearcher import AutoResearcher

ar = AutoResearcher(
    anthropic_api_key="...",   # any subset of the three LLM keys is fine
    openai_api_key="...",
    gemini_api_key="...",
    search_api_key="...",      # Tavily; optional (falls back to DuckDuckGo)
    models={"planner": "claude", "worker": "gemini", "writer": "openai"},  # optional
    max_subagents=8,
    max_critic_loops=2,
)
result = ar.run("Chennai trip for 3 days", output="chennai_trip.docx")   # sync
result = await ar.arun("Chennai trip for 3 days", output="chennai_trip.docx")  # async
# result: RunResult(docx_path, report_spec, plan, sources, token_usage, warnings)
```
Also support streaming progress: `for event in ar.stream(question): ...` yielding typed
events (`PlanCreated`, `TaskStarted`, `TaskFinished`, `CriticVerdict`, `ReportWritten`).

Keys resolve in this order: constructor args > environment variables (ANTHROPIC_API_KEY,
OPENAI_API_KEY, GEMINI_API_KEY/GOOGLE_API_KEY, TAVILY_API_KEY, PINECONE_API_KEY) > `.env` file.
NEVER log, print, or persist API keys. Redact them in repr/exceptions.

## 3. PROJECT STRUCTURE
```
autoresearcher/
├── pyproject.toml, README.md, LICENSE (MIT), CHANGELOG.md, .env.example, .gitignore
├── .github/workflows/ci.yml
├── src/autoresearcher/
│   ├── __init__.py            # exports AutoResearcher, RunResult
│   ├── config.py              # Settings (pydantic-settings), key resolution
│   ├── errors.py              # custom exceptions
│   ├── llm/factory.py         # get_llm(role) with provider abstraction + fallback
│   ├── graph/
│   │   ├── state.py           # ResearchState TypedDict + reducers
│   │   ├── clarifier.py       # infer days/budget/interests or use defaults
│   │   ├── planner.py         # ResearchPlan + todo list + template selection
│   │   ├── workers.py         # sub-agent runner (ReAct-style w/ tools)
│   │   ├── critic.py          # gap/contradiction/uncited-claim detection
│   │   ├── synthesizer.py     # -> ReportSpec (structured output)
│   │   └── build.py           # assembles + compiles the LangGraph
│   ├── mcp_servers/
│   │   ├── search_server.py
│   │   ├── travel_server.py
│   │   ├── knowledge_server.py
│   │   └── docx_server.py     # thin wrapper exposing render_docx
│   ├── mcp_client.py          # MultiServerMCPClient setup + third-party server config
│   ├── tools/                 # plain-Python implementations (MCP servers wrap these)
│   ├── memory/{base.py, pinecone_store.py, chroma_store.py, cache.py}
│   ├── report/
│   │   ├── schema.py          # ReportSpec and block models
│   │   ├── docx_renderer.py   # deterministic renderer
│   │   └── templates/{base.py, trip.py, generic.py, market_research.py}
│   ├── api/app.py             # FastAPI
│   └── cli.py
├── tests/  (unit + integration, all offline)
└── examples/{chennai_trip.py, generic_research.py}
```

## 4. LLM LAYER
- `get_llm(role, settings)` returns a LangChain chat model. Roles: `planner`, `worker`, `critic`, `writer`.
- Default model per provider/role in a config table (easy to override): strong model for planner/writer/critic,
  cheaper/faster for worker. Do not hardcode model IDs in multiple places; centralize them and let users override
  with `models={"planner": "claude:<model-id>"}`.
- If the requested provider's key is missing, fall back to the next available provider and add a warning to the result.
  If no LLM key exists at all, raise a clear `MissingAPIKeyError` listing what to set.
- Add retry with exponential backoff (tenacity) for rate limits/timeouts. Track token usage per role.
- All structured outputs use `.with_structured_output(PydanticModel)` with one repair-retry on validation failure.

## 5. GRAPH / AGENT DESIGN
Flow: START -> clarifier -> planner -> [fan-out via Send: one worker per task, max `max_subagents`]
-> fan-in -> critic -> (if gaps and loops < max_critic_loops: back to planner with critique to add tasks)
-> synthesizer -> docx renderer -> END.

State fields: question, assumptions, plan (ResearchPlan), findings (append-reducer), sources (dedup by URL),
critique, loop_count, report_spec, docx_path, warnings, token_usage.

Models:
- `Task`: id, title, worker_type (web | places | weather | budget | transport | events | retrieval), query, day (optional), priority.
- `ResearchPlan`: goal, assumptions, template ("trip" | "market_research" | "generic"), todo_list (list[str]), tasks.
- `Source`: id (e.g. "S1"), url, title, snippet, retrieved_at, credibility_score.
- `Finding`: task_id, claims: list[Claim], raw_notes. `Claim`: text, source_ids (REQUIRED, min 1).
- `Critique`: verdict (pass | revise), gaps, contradictions, uncited_claims, new_tasks.

CRITICAL QUALITY RULES:
1. Every factual claim in the final report must reference >=1 `source_id`. The critic rejects uncited claims.
   Claims that cannot be sourced must be dropped or explicitly labeled "assumption/estimate".
2. Sub-agents may only call tools; they must not invent URLs. Source ids come from tool results only.
3. Hard limits: max sub-agents per run, max tool calls per sub-agent (default 6), per-run token budget
   (default configurable), per-task timeout. On limit hit, degrade gracefully and record a warning.
4. One failing sub-agent must not crash the run. Capture the error as a warning and continue.

## 6. TOOLS (implement plain-Python functions first, then expose via MCP)
Priority set (Phase 2-3): web_search, fetch_page, search_places, get_weather_forecast (Open-Meteo, no key),
estimate_travel_time, estimate_budget, embed_and_store, retrieve_similar, build_itinerary, render_docx.
Later set (Phase 6, optional, behind feature flags): news_search, wikipedia_lookup, extract_facts,
find_transport_options, get_events, get_currency_rate, cache_lookup, dedupe_sources, decompose_question,
build_todo_list, verify_claim, detect_gaps, score_source_credibility, add_table/add_checklist/add_toc helpers, build_citations.

Tool requirements: typed signatures, docstrings (used as tool descriptions), Pydantic-validated inputs,
timeouts, clear error messages, no secrets in outputs. `build_itinerary` must cluster places by geographic
proximity so each day is coherent (simple k-means/greedy on lat/lon is fine) and respect opening hours if available.

MCP: each server runs over stdio by default (`python -m autoresearcher.mcp_servers.search_server`).
`mcp_client.py` builds `MultiServerMCPClient` from defaults + user-supplied config so users can add third-party
MCP servers (e.g. Google Maps, Brave). If MCP startup fails, fall back to calling the plain-Python tools directly.

## 7. DOCX RENDERER (python-docx, deterministic)
`ReportSpec` = title, subtitle, metadata, list of `Section`s, each containing typed blocks:
heading, paragraph, bullets, numbered, table (header + rows), checklist, callout (info/tip/warning), day_plan, page_break.
Every block that states facts carries `source_ids`. Sources are rendered as numbered footnote-style markers [1] and a final "Sources" section.

`DocxRenderer` must produce:
- Title page (title, subtitle, date, assumptions summary)
- Table of contents (Word TOC field, updates on open)
- Consistent styles (fonts, heading colors, spacing) defined in one place
- Header/footer with page numbers ("Page X of Y")
- Real Word tables with header row shading and column widths
- Checklist with "☐" items
- Callout boxes as shaded single-cell tables
- Optional images from local path (skip gracefully if missing)
- Clickable hyperlinks in the Sources section
- Output must open cleanly in Microsoft Word and LibreOffice (validate in a test by re-opening with python-docx and,
  if LibreOffice is available, converting to PDF).

Templates (`report/templates/`): the planner chooses one; each template defines required sections and synthesizer prompt guidance.
`trip` template sections: Trip summary & assumptions; Pre-trip to-do checklist; Best time & weather; Day-by-day plan
(one time-blocked table per day: morning / afternoon / evening / food / transport / cost);
Where to stay (by area & budget tier); Food guide; Getting around; Budget estimate table; Packing list & safety tips;
Optional day trips; Sources. Also implement `generic` (Executive summary, Background, Findings by theme, Analysis,
Recommendations, Risks, Sources) and a minimal `market_research`.

## 8. FASTAPI + CLI
API:
- `POST /research` (body: question, options; keys via headers `X-Anthropic-Key`, `X-OpenAI-Key`, `X-Gemini-Key`, `X-Search-Key`) -> `{job_id}`
- `GET /research/{job_id}/stream` -> SSE progress events
- `GET /research/{job_id}` -> status/result JSON
- `GET /research/{job_id}/download` -> the .docx
- Keys are per-request, held only in memory for the job, never written to disk or logs. In-memory job store with TTL cleanup.
CLI: `autoresearcher "Chennai trip for 3 days" -o chennai_trip.docx [--models planner=claude,worker=gemini] [--max-subagents 8]`
plus `autoresearcher serve` to start the API, and `autoresearcher doctor` to verify keys/connectivity (without printing keys).

## 9. TESTING (CI must never spend money or need network)
- Use LangChain fake chat models (`GenericFakeChatModel`/custom) and recorded/mocked HTTP (respx) for search & weather.
- Unit tests: config/key resolution, LLM factory fallback, each tool, state reducers, itinerary clustering,
  critic rejecting uncited claims, ReportSpec validation, DocxRenderer (open output and assert headings/tables/TOC/sources exist).
- Integration test: full graph run on "Chennai trip for 3 days" with fakes -> valid .docx produced.
- API tests with FastAPI TestClient incl. SSE.
- Mark real-provider tests with `@pytest.mark.live` and skip by default.
- CI: ruff, mypy, pytest on Python 3.10-3.12.

## 10. DOCS
README with: pitch, architecture diagram (mermaid), install, quickstart, key config, model routing, MCP servers and how to add
third-party ones, the tools table, template system, API usage, cost controls, limitations, contributing, and roadmap.
Include `.env.example` and a sample output description for the Chennai example.

## PHASES
1. Scaffold + config + LLM factory + schemas + minimal single-agent flow (planner -> web_search -> simple docx). Tests pass.
2. LangGraph with parallel fan-out, full state, clarifier, critic loop, and the priority tools as plain Python.
3. MCP servers + client (with direct-call fallback) + vector memory (Chroma default, Pinecone optional) + cache.
4. Full DocxRenderer + trip/generic templates + citations enforcement + itinerary builder.
5. FastAPI + SSE + CLI (`run`, `serve`, `doctor`).
6. Remaining tools, docs, examples, CI, packaging checks (`python -m build`, `twine check`).

## DEFINITION OF DONE
- `pip install -e .[all,dev]` works; `pytest` passes offline; `ruff` and `mypy` are clean.
- `examples/chennai_trip.py` produces a well-formatted multi-page .docx (with fake LLMs in tests, real ones when keys are present).
- Every claim in the sample report has a source; uncited claims are blocked.
- No API keys appear in logs, tracebacks, or files.
- README quickstart works copy-paste.

Start with Phase 1 now. Show the file tree you'll create, then implement it.