Metadata-Version: 2.4
Name: inspect_ai
Version: 0.3.284
Summary: Framework for large language model evaluations
Author: UK AI Security Institute
License: MIT License
Project-URL: Documentation, https://inspect.aisi.org.uk/
Project-URL: Source Code, https://github.com/UKGovernmentBEIS/inspect_ai
Project-URL: Issue Tracker, https://github.com/UKGovernmentBEIS/inspect_ai/issues
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: agent-client-protocol>=0.12.0
Requires-Dist: aiobotocore>=2.18.0
Requires-Dist: anyio>=4.14.2
Requires-Dist: beautifulsoup4>=4.10.0
Requires-Dist: boto3
Requires-Dist: click!=8.2.0,!=8.2.2,!=8.3.0,!=8.3.1,>=8.1.3
Requires-Dist: debugpy
Requires-Dist: docstring-parser>=0.16
Requires-Dist: fastapi>=0.119.0
Requires-Dist: fsspec<=2026.6.0,>=2023.1.0
Requires-Dist: httpx
Requires-Dist: ijson>=3.2.0
Requires-Dist: jsonlines>=3.0.0
Requires-Dist: jsonpatch>=1.32
Requires-Dist: jsonpath-ng>=1.8.0
Requires-Dist: jsonref>=1.1.0
Requires-Dist: jsonschema>3.1.1
Requires-Dist: markdown-it-py>=2.2.0
Requires-Dist: mmh3>3.1.0
Requires-Dist: nest_asyncio2>=1.7.4
Requires-Dist: numpy
Requires-Dist: platformdirs>=2.3.0
Requires-Dist: psutil
Requires-Dist: pydantic>=2.13.0
Requires-Dist: python-dotenv>=0.16.0
Requires-Dist: pyyaml
Requires-Dist: rich!=14.0.0,>=13.3.3
Requires-Dist: s3fs>=2023
Requires-Dist: semver>=3.0.0
Requires-Dist: shortuuid
Requires-Dist: sniffio
Requires-Dist: tenacity
Requires-Dist: textual>=6.2.0
Requires-Dist: tiktoken>=0.12.0
Requires-Dist: typing_extensions>=4.14.0
Requires-Dist: universal-pathlib>=0.2.6
Requires-Dist: uvicorn
Requires-Dist: zipfile-zstd>=0.0.4; python_version < "3.14"
Requires-Dist: zipp>=3.19.1
Requires-Dist: zstandard>=0.20.0
Provides-Extra: dev
Requires-Dist: adlfs>=2025.8.0; extra == "dev"
Requires-Dist: anthropic>=2.0.0; extra == "dev"
Requires-Dist: antlr4-python3-runtime!=4.10.*,!=4.12.*,!=4.13.0,!=4.13.1,<=4.13.2,>=4.9.3; extra == "dev"
Requires-Dist: apprise; extra == "dev"
Requires-Dist: aws-bedrock-token-generator; extra == "dev"
Requires-Dist: azure-ai-inference; extra == "dev"
Requires-Dist: azure-identity; extra == "dev"
Requires-Dist: brotli; extra == "dev"
Requires-Dist: google-genai>=2.14.0; extra == "dev"
Requires-Dist: griffe>=2; extra == "dev"
Requires-Dist: groq>=0.28.0; extra == "dev"
Requires-Dist: huggingface_hub>=1.6.0; extra == "dev"
Requires-Dist: inspect-scout>=0.5.5; extra == "dev"
Requires-Dist: inspect-sentinel>=0.2.0; extra == "dev"
Requires-Dist: ipython; extra == "dev"
Requires-Dist: jsonpath-ng; extra == "dev"
Requires-Dist: latex2sympy2-extended==1.11.0; extra == "dev"
Requires-Dist: markdown; extra == "dev"
Requires-Dist: mcp>=1.10.0; extra == "dev"
Requires-Dist: mistralai>=3.0.0; extra == "dev"
Requires-Dist: moto[server]>=5.1.5; extra == "dev"
Requires-Dist: mypy>=2.3.0; extra == "dev"
Requires-Dist: nbformat; extra == "dev"
Requires-Dist: openai>=3.4.0; extra == "dev"
Requires-Dist: pandas>=2.0.0; extra == "dev"
Requires-Dist: pandas-stubs; extra == "dev"
Requires-Dist: panflute; extra == "dev"
Requires-Dist: pip; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pyarrow>=10.0.1; extra == "dev"
Requires-Dist: pyarrow-stubs; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-dotenv; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-reportlog; extra == "dev"
Requires-Dist: pytest-textual-snapshot; extra == "dev"
Requires-Dist: pytest-timeout; extra == "dev"
Requires-Dist: pytest-watcher; extra == "dev"
Requires-Dist: pytest-xdist; extra == "dev"
Requires-Dist: ruff==0.9.6; extra == "dev"
Requires-Dist: textual-dev>=0.86.2; extra == "dev"
Requires-Dist: together>=2.0.0; extra == "dev"
Requires-Dist: trio; extra == "dev"
Requires-Dist: trustme; extra == "dev"
Requires-Dist: types-aiobotocore; extra == "dev"
Requires-Dist: types-beautifulsoup4; extra == "dev"
Requires-Dist: types-boto3; extra == "dev"
Requires-Dist: types-botocore; extra == "dev"
Requires-Dist: types-grpcio; extra == "dev"
Requires-Dist: types-jsonpatch; extra == "dev"
Requires-Dist: types-jsonschema; extra == "dev"
Requires-Dist: types-Markdown; extra == "dev"
Requires-Dist: types-protobuf; extra == "dev"
Requires-Dist: types-psutil; extra == "dev"
Requires-Dist: types-python-dateutil; extra == "dev"
Requires-Dist: types-PyYAML; extra == "dev"
Requires-Dist: types-regex; extra == "dev"
Requires-Dist: xai_sdk>=1.18.0; extra == "dev"
Requires-Dist: zipfile-zstd>=0.0.4; python_version < "3.14" and extra == "dev"
Provides-Extra: doc
Requires-Dist: click>=8.2.0; extra == "doc"
Requires-Dist: griffe>=2; extra == "doc"
Requires-Dist: jupyter; extra == "doc"
Requires-Dist: markdown; extra == "doc"
Requires-Dist: nbclient; extra == "doc"
Requires-Dist: panflute; extra == "doc"
Requires-Dist: pyyaml; extra == "doc"
Requires-Dist: quarto-cli>=1.9.36; extra == "doc"
Requires-Dist: types-PyYAML; extra == "doc"
Provides-Extra: dist
Requires-Dist: build; extra == "dist"
Requires-Dist: twine; extra == "dist"
Provides-Extra: math
Requires-Dist: antlr4-python3-runtime!=4.10.*,!=4.12.*,!=4.13.0,!=4.13.1,<=4.13.2,>=4.9.3; extra == "math"
Requires-Dist: latex2sympy2-extended==1.11.0; extra == "math"
Dynamic: license-file

[<img width="295" src="https://inspect.aisi.org.uk/images/aisi-logo.svg" />](https://aisi.gov.uk/)

Welcome to Inspect, a framework for large language model evaluations created by the [UK AI Security Institute](https://aisi.gov.uk/).

Inspect provides many built-in components, including facilities for prompt engineering, tool usage, multi-turn dialog, and model graded evaluations. Extensions to Inspect (e.g. to support new elicitation and scoring techniques) can be provided by other Python packages.

To get started with Inspect, please see the documentation at <https://inspect.aisi.org.uk/>.

Inspect also includes a collection of over 200 pre-built evaluations ready to run on any model (learn more at <https://inspect.aisi.org.uk/evals/>).

> Coding agents: a structured index of the docs is published at <https://inspect.aisi.org.uk/llms.txt>. The user guide is concatenated as Markdown at <https://inspect.aisi.org.uk/llms-guide.txt>, and <https://inspect.aisi.org.uk/llms-full.txt> additionally bundles the API and CLI reference. Individual pages are available as Markdown by appending `.md` to the `.html` path (e.g. `/extensions/index.html.md`).

***

To work on development of Inspect, install [uv](https://docs.astral.sh/uv/) (0.11.8 or later), clone the repository, and sync the development environment from the checked-in lockfile:

```bash
git clone https://github.com/UKGovernmentBEIS/inspect_ai.git
cd inspect_ai
uv sync
```

This installs Inspect in editable mode with its development dependencies at the versions recorded in `uv.lock`. Project dependencies are declared in `requirements*.txt` and exposed through `pyproject.toml`. When changing dependencies, update the appropriate requirements file and run `uv lock` rather than relying on `uv add`. uv only resolves releases that are at least seven days old, except for model provider SDKs and Inspect's own packages; see [CONTRIBUTING.md](CONTRIBUTING.md#working-on-your-issue) for how to adopt a newer release.

Optionally install pre-commit hooks via

```bash
make hooks
```

Run linting, formatting, and tests via

```bash
make check
make test
```

Prefix those commands with `uv run` (for example, `uv run make check`), or activate the environment in `.venv` first.

If you use VS Code, you should be sure to have installed the recommended extensions (Python, Ruff, and MyPy). Note that you'll be prompted to install these when you open the project in VS Code.

### Frontend development (TypeScript)

The web UI lives in a git submodule at `src/inspect_ai/_view/ts-mono/`. **These steps are only needed if you plan to work on the TypeScript/React frontend** — Python-only contributors can skip this entirely.

Initialize the submodule and install dependencies — see the [one-time setup guide](https://github.com/meridianlabs-ai/ts-mono/blob/main/docs/submodule-guide.md#one-time-setup).

### Documentation

To work on the Inspect documentation, sync the optional `doc` dependencies and build the docs:

```
uv sync --extra doc
cd docs
uv run quarto render # or 'uv run quarto preview'
```

If you intend to work on the docs iteratively, you'll want to install the Quarto extension in VS Code.
