# agentprobe-toolkit (AgentProbe)

> AgentProbe (`pip install agentprobe-toolkit`, `import agentprobe`) is an MIT-licensed Python library for fault-injection testing of tool-using LLM agents in pytest. It wraps an agent's tools, injects deterministic failures (timeouts, rate limits, API errors) or substituted return values, records every tool call, and fails the test unless every declared fault actually fired and every required assertion passed. Works with plain Python functions, LangChain tools and LangGraph agents. No model or API key is needed for scripted agents.

Core API: `from agentprobe.testing import probe`; `with probe() as p:`; `tools = p.wrap(TOOLS)`; `p.on("tool").fail_first(1)` / `always_fail(with_="timeout")` / `returns(value)` / `slow(ms=...)`; `result = p.run(agent, input, timeout=...)`; `result.assert_no_crash()`, `assert_answer_contains(...)`, `assert_not_called(...)`.

## Docs

- [Getting started](https://agentprobe-toolkit.pages.dev/getting-started/): install and a first fault test
- [Test that your agent retries](https://agentprobe-toolkit.pages.dev/guides/retries/)
- [Simulate timeouts and rate limits](https://agentprobe-toolkit.pages.dev/guides/timeouts-and-rate-limits/)
- [Test wrong or stale tool data](https://agentprobe-toolkit.pages.dev/guides/wrong-data/)
- [LangGraph tool errors](https://agentprobe-toolkit.pages.dev/guides/langgraph-tool-errors/): ToolNode re-raises tool exceptions by default
- [Inject the exceptions your code handles](https://agentprobe-toolkit.pages.dev/guides/real-exceptions/)
- [Run fault tests in CI](https://agentprobe-toolkit.pages.dev/guides/ci/)

## Reference

- [Scenarios](https://agentprobe-toolkit.pages.dev/reference/scenarios/)
- [Assertions and results](https://agentprobe-toolkit.pages.dev/reference/assertions/)
- [How probing works](https://agentprobe-toolkit.pages.dev/instrumentation/): guarantees about wrapping, concurrency, deadlines and evidence

## Optional

- [Source on GitHub](https://github.com/chhayankjain/agentprobe-toolkit)
- [Case study: a third-party LangGraph agent](https://agentprobe-toolkit.pages.dev/case-studies/agents-from-scratch/)
- [Research: benchmark and paper](https://agentprobe-toolkit.pages.dev/research/)
