Metadata-Version: 2.4
Name: query-agent-benchmarking
Version: 0.3
Summary: A Python library for benchmarking Weaviate's Query Agent!
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: dspy>=2.6.27
Requires-Dist: sentence-transformers>=5.0.0
Requires-Dist: weaviate-client[agents]>=4.15.4
Requires-Dist: weaviate-agents>=1.0.0
Requires-Dist: pandas>=2.3.1
Requires-Dist: datasets>=4.0.0
Requires-Dist: ir-datasets>=0.5.11
Requires-Dist: pip>=25.2
Requires-Dist: setuptools>=80.9.0
Requires-Dist: wheel>=0.45.1
Requires-Dist: twine>=6.2.0
Dynamic: license-file

# Query Agent Benchmarking

This repo contains a tool for benchmarking the performance of Weaviate's Query Agent.

Populate Weaviate with benchmark data:
```
uv run python query_agent_benchmarking/populate-db.py
```

Run eval:
```
uv run python scripts/run-benchmark.py
```

See `benchmarker/config.yml` to change the dataset populated in your Weaviate instance, as well as ablate `hybrid-search` or `query-agent-search-only`, as well as the number of samples and concurrency parameters.
