Metadata-Version: 2.4
Name: mcp-api-test
Version: 0.1.1
Summary: MCP server for testing REST API with built-in assertions
Author: Ryan Febriansyah
Keywords: testing,mcp,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp
Requires-Dist: httpx
Requires-Dist: python-dotenv
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: respx; extra == "dev"
Dynamic: license-file

# mcp-api-test

![image](https://gitlab.com/ryaneatfood/mcp-api-testing/badges/master/pipeline.svg)

An AI-native API testing engine that enables LLM agents to execute, validate, and reason about REST APIs using natural language or cURL inputs. Designed for autonomous QA, backend validation, CI automation, and agentic software engineering workflows.

Unlike traditional testing frameworks, `mcp-api-test` exposes API testing as a first-class capability for AI agents rather than a developer-facing tool. LLM-powered clients can invoke the `test_api` tool directly, making API validation a building block for autonomous coding agents, not a separate human-driven workflow. It complements (rather than replaces) established tools like pytest, Postman, Playwright and such.

## Where it fits

### QA and test engineering

Instead of manually crafting requests in Postman or writing custom scripts, QA engineers can prompt an agent with natural language:

> "Can you test this API and give me the result?"

> "Run these 12 endpoints from the spec and tell me which ones are failing"

The agent invokes `test_api` for each call, validates against the expected contract, and reports back in a structured format. No context switching, no UI, just conversation-driven testing.

### Post-deployment regression

After a deploy, ask your agent to smoke-test critical paths:

> "Deploy just finished, run the regression suite against staging"

The agent executes each endpoint, checks status codes and key response fields, and flags any regression. Because the tool returns deterministic structured results, the agent can chain multiple validations and summarize failures without parsing raw HTTP output.

### Quick validation during development

During a coding session, validate an endpoint you just built or modified:

> "Does the new `/api/v2/orders` endpoint return 201 with an `order_id`?"

No need to open a separate tool, write a test file, or keep Postman collections in sync. The agent tests it inline, in the same conversation where you're writing code.

### CI/CD automation

Agents integrated into CI pipelines can validate deployments autonomously such as: verifying API contracts, checking health endpoints, or running smoke tests. Without maintaining separate test suites or glue scripts. The tool is a primitive the agent can combine with other capabilities (git, issue tracking, notification) to build end-to-end automation.

> **Token efficiency.** Without this tool, an agent would issue raw `curl` commands via a shell, parse unstructured terminal output, and pattern-match against expected values, consuming significantly more tokens per validation. With `mcp-api-test`, the agent gets deterministic, structured results in a single tool call. Fewer tokens, fewer hallucinations, more reliable decision-making at scale.

## Prerequisites

- Python >= 3.10
- pip or uv for package installation

## Install

### From PyPI

```bash
pip install mcp-api-test
```

### From source

```bash
git clone git@gitlab.com:ryaneatfood/mcp-api-testing.git

# change the directory
cd mcp-api-testing

# install from source
pip install -e .
```

This registers the `mcp-api-test` CLI command on your PATH. Verify with:

```bash
which mcp-api-test
```

## MCP Client Setup

### OpenCode

Edit `opencode.json` (already configured at project root):

```json
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "mcp-api-test": {
      "type": "local",
      "command": ["mcp-api-test"],
      "enabled": true
    }
  }
}
```

### Claude Desktop

Edit `claude_desktop_config.json` (usually at `~/Library/Application Support/Claude/claude_desktop_config.json` on macOS):

```json
{
  "mcpServers": {
    "mcp-api-test": {
      "command": "mcp-api-test"
    }
  }
}
```

Restart Claude Desktop after saving.

## Troubleshooting

### Claude Desktop fails to connect: Cargo lock file version mismatch

If Claude Desktop logs show an error like:

```
lock file version `4` was found, but this version of Cargo does not
understand this lock file, perhaps Cargo needs to be updated?
```

The `mcp[cli]` dependency pulls in `cryptography` which builds from source and requires a recent version of Cargo (Rust's build tool). Your system's Cargo is too old to understand the lock file format generated by newer Rust toolchains.

To fix this, you need to update Rust and Cargo via [rustup](https://rustup.rs/):

```bash
rustup update
```

After updating, restart Claude Desktop and the MCP server should connect normally.

### Server not found / command not found

If the MCP client reports that `mcp-api-test` is not found, verify the CLI command is on your PATH:

```bash
which mcp-api-test
```

If nothing resolves, reinstall:

```bash
pip install -e .
```

and ensure your Python `bin` directory (e.g. `~/.local/bin` or your virtualenv's `bin`) is in your `$PATH`.

## Usage

AI agents interact with the `test_api` tool in two modes:

### cURL mode

Pass a raw cURL command — the server parses method, URL, headers, and payload automatically. Example prompt an agent might receive:

> "Please test this API using this cURL: `curl -X POST https://api.example.com/v1/users -H 'Authorization: Bearer xxx' -d '{"name":"test"}'`, ensure status code is 201 and response contains 'id'"

### Structured mode

Provide individual fields instead of a cURL command. Example prompt:

> "Please test this API at https://api.example.com/v1/users with method GET, and the request header is {"Content-Type":"application/json"}, ensure status code is 200 and response body contains 'data'" attribute

### Assertions

At the moment, this library provides built-in assertions that proved useful for testing the API, such as:

| Parameter | Type | Description |
|-----------|------|-------------|
| `required_status_code` | `int` | Assert exact status code |
| `required_status_code_range` | `str` | Assert status code is within a range (e.g. `"2xx"`, `"200-299"`) |
| `required_response_fields` | `list[str]` | Assert fields exist in JSON response |
| `required_response_contains` | `str` | Assert response body contains a substring |

When assertions are omitted, the tool simply executes the request and reports the response, providing raw data for the agent to reason about independently.

## Response

The tool returns a structured, machine and human-readable result with box-drawing formatting, pass/fail indicators, and a summary:

### Passing (with assertions)

```
┌──────────────────────────────────────────────┐
│ Api Test Result                              │
└──────────────────────────────────────────────┘

API Name        : create-user
Method          : POST
URL             : https://api.example.com/v1/users
Status Code     : 201
Response Time   : 142.35 ms
Overall Result  : Passed

Assertions
--------------------------------------------------
✓ status_code
✓ response_field_exist

Summary
--------------------------------------------------
Total Assertions : 2
Passed           : 2
Failed           : 0
```

### Failing (assertion mismatch)

```
┌──────────────────────────────────────────────┐
│ Api Test Result                              │
└──────────────────────────────────────────────┘

API Name        : create-user
Method          : POST
URL             : https://api.example.com/v1/users
Status Code     : 500
Response Time   : 89.12 ms
Overall Result  : Failed

Assertions
--------------------------------------------------
✗ status_code

Summary
--------------------------------------------------
Total Assertions : 1
Passed           : 0
Failed           : 1
```

### Network / runtime error

```
┌──────────────────────────────────────────────┐
│ Api Test Result                              │
└──────────────────────────────────────────────┘

API Name        : check-service
Method          : GET
URL             : https://api.internal.example.com/v1/status
Overall Result  : Failed

Error
--------------------------------------------------
ConnectError: [Errno 8] nodename nor servname provided, or not known
```

## Logging

The server writes logs to `log/mcp-api-test.log` at the project root. Useful for debugging failed requests or tracing tool calls.

## Environment Variables

The server loads a `.env` file via `python-dotenv` on startup. Useful for storing API keys, base URLs, etc. without hardcoding them in prompts.

## Development

```bash
# Re-install after pulling changes
pip install -e .

# Run the MCP server directly
mcp-api-test
```

To run the test suite:

```bash
pip install -e ".[dev]"
pytest -v
```

## Testing with MCP Inspector

The [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector) provides a browser-based UI to verify your server connects and responds correctly before wiring it into an AI client.

Make sure `mcp-api-test` is installed on your PATH (see [Install](#install)), then run:

```bash
npx @modelcontextprotocol/inspector
```

This launches the inspector and opens `http://localhost:5173` in your browser. From the UI:

1. Set **Transport Type** to `STDIO`
2. Set **Command** to `mcp-api-test`
3. Leave **Arguments** empty (your server needs none)
4. Click **Connect**

When the connection succeeds, the status badge changes to **Connected** and the inspector lists the `test_api` tool under the Tools tab. You can click the tool to open a form and run test calls interactively. This is useful for rapid iteration without restarting your MCP client.

If the status shows "Failed to connect" double-check that `which mcp-api-test` resolves and that `pip install -e .` completed without errors

## Running Tests

Install test dependencies and run the suite:

```bash
# install test dependencies
pip install pytest pytest-asyncio respx

# run the whole test suite
pytest -v
```

All integration tests use `respx` to mock HTTP responses. No real network calls are made, so the suite runs fast and offline. The test suite also consisted of unit test and integration test
