Metadata-Version: 2.4
Name: evidence-fetcher
Version: 0.3.0
Summary: Interactive evidence-search and relevance-feedback experiment scaffold.
Project-URL: Homepage, https://github.com/yeiichi/evidence-fetcher
Project-URL: Documentation, https://evidence-fetcher.readthedocs.io/
Project-URL: Repository, https://github.com/yeiichi/evidence-fetcher
Project-URL: Issues, https://github.com/yeiichi/evidence-fetcher/issues
Author: Eiichi YAMAMOTO
License-Expression: MIT
License-File: LICENSE
Keywords: evidence,relevance-feedback,search
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.12
Provides-Extra: ui
Requires-Dist: streamlit>=1.37; extra == 'ui'
Description-Content-Type: text/markdown

# evidence-fetcher

[![PyPI](https://img.shields.io/pypi/v/evidence-fetcher.svg)](https://pypi.org/project/evidence-fetcher/)
[![Python](https://img.shields.io/pypi/pyversions/evidence-fetcher.svg)](https://pypi.org/project/evidence-fetcher/)
[![License](https://img.shields.io/pypi/l/evidence-fetcher.svg)](https://github.com/yeiichi/evidence-fetcher/blob/main/LICENSE)

`evidence-fetcher` is an interactive evidence-search and relevance-feedback experiment.

The current vertical slice fetches candidate web evidence from Brave Search,
normalizes the provider response, removes duplicate URLs, applies a lightweight
local relevance ranker, and exposes the results through a Python API and a small
command-line interface.

The feedback-aware reranking is based on a Rocchio-style relevance feedback
algorithm over TF-IDF vectors.

Because Interactive Information Retrieval can amplify the feedback you provide,
it should be used with care. The goal is to condense useful evidence from search
results, not to cherry-pick sources that only support a preferred conclusion.
Reviewers should keep contrary, ambiguous, and low-confidence evidence visible
when it matters to the question.

## Installation

Install the core package:

```bash
pip install evidence-fetcher
```

## Configuration

Create a Brave Search API key and expose it as `BRAVE_SEARCH_API_KEY`:

```bash
export BRAVE_SEARCH_API_KEY="..."
```

## Python API

```python
from evidence_fetcher import fetch_evidence

results = fetch_evidence("retrieval augmented generation evaluation")

for evidence in results:
    print(evidence.title)
    print(evidence.url)
    print(evidence.snippet)
```

`fetch_evidence()` returns typed `Evidence` objects with normalized `title`,
`url`, `snippet`, `source`, and `rank` fields.

## CLI

Readable text output:

```bash
evidence-fetcher "retrieval augmented generation evaluation"
```

JSON output:

```bash
evidence-fetcher "retrieval augmented generation evaluation" --format json
```

Use `--limit` to control the maximum number of results.

## Streamlit UI

Install the optional UI extra:

```bash
pip install "evidence-fetcher[ui]"
```

Launch the Streamlit review app:

```bash
evidence-fetcher-ui
```

The UI uses the same Brave-backed retrieval, local reranking, relevance feedback,
and explicit expansion workflow as the CLI.

Interactive relevance feedback:

```bash
evidence-fetcher "retrieval augmented generation evaluation" --interactive
```

Interactive mode performs the same initial search, then lets you mark displayed
results, request a local rerank, or explicitly expand retrieval with a refined
query. Ordinary text and JSON commands are unchanged and remain non-interactive.
Interactive mode is text-only; combining `--interactive` with `--format json` is
rejected.

Supported interactive commands:

```text
more 1 3
less 4
irrelevant 5
unsure 2
show        # or rerank
expand
help        # or ?
done        # or quit, q, exit
```

Result numbers are presentation-only positions in the currently displayed list.
Feedback is stored against each evidence URL, and after `show` the same number
may refer to a different result.
The interactive prompt shows the number of displayed results and accumulated
feedback items, for example `[10 results, 2 feedback items] >`.

`show` reranks only the current candidate set using accumulated feedback.
`expand` builds a deterministic lexical query from the original query and
feedback-marked evidence, performs one additional Brave retrieval, merges any
new URLs into the session, preserves accumulated feedback, and reranks the
expanded candidate set. Expansion is always user-triggered; feedback commands do
not automatically run new searches.

Example session:

```text
$ evidence-fetcher "retrieval augmented generation evaluation" --interactive
Interactive relevance feedback. Type 'help' for commands.
1. Result title
   https://example.com/page
   Short snippet...
[10 results, 0 feedback items] > more 1
recorded more: 1 (1 feedback item total)
[10 results, 1 feedback item] > irrelevant 3
recorded irrelevant: 3 (2 feedback items total)
[10 results, 2 feedback items] > show
Updated ranking from accumulated feedback:
1. Result title
   https://example.com/page
   Short snippet...
[10 results, 2 feedback items] > expand
Searching with refined query:
retrieval augmented generation evaluation benchmark faithfulness
Added 4 new results.
Updated ranking:
1. Result title
   https://example.com/page
   Short snippet...
[14 results, 2 feedback items] > done
```

## Architecture

The public API is intentionally small: `fetch_evidence()` accepts a query and
returns normalized, deduplicated `Evidence` results. Provider-specific HTTP
requests, credential handling, and response parsing live behind an internal
Brave Search provider. The provider depends on a small HTTP client protocol, so
tests and future integrations can inject a mock client instead of using the
network.

Candidate retrieval and local relevance ranking are separate steps. Brave Search
retrieves candidate evidence from the web; a replaceable local ranker then orders
those candidates against the original query. The default ranker is a small
standard-library TF-IDF implementation over evidence titles and snippets. This
is the first foundation for Interactive Information Retrieval: the code now has
a boundary where a workflow can collect user relevance feedback and rerank
results.

Feedback-aware reranking is available through the Python/domain layer. A
`SearchSession` retains the original query, candidate evidence, current ranked
results, and accumulated `RelevanceFeedback` entries keyed by evidence URL. The
TF-IDF ranker applies a lightweight Rocchio-style query update: it keeps the
original query vector, moves it toward evidence marked `MORE` or `RELEVANT`,
moves it away from evidence marked `LESS` or `IRRELEVANT`, and records `UNSURE`
without changing ranking.

Feedback-informed retrieval expansion is separate from local reranking. The
default query refiner tokenizes the original query and feedback-marked evidence,
removes small stop words, scores candidate expansion terms lexically with TF-IDF,
avoids duplicating original query terms, suppresses terms associated mainly with
irrelevant evidence, and appends up to four useful terms. The refined query is
printed before each `expand` retrieval. Expansion does not persist a session
across invocations, run in the background, use an LLM, or send feedback anywhere
except the explicit refined Brave query.

Package-specific exceptions distinguish configuration errors, request failures,
and malformed provider responses.

Only Brave Search is implemented in this slice. No persistence or additional
search providers are included yet.

### IIR review caution

Interactive feedback can make evidence review faster, but it can also drift
toward cherry-picking if the reviewer marks only convenient sources as useful.
Treat reranking and expansion as prioritization aids. They do not replace
balanced judgment, source quality checks, or deliberate review of evidence that
complicates the initial hypothesis.

## Development

This project uses `uv` for environment and dependency management.

```bash
uv sync
uv run pytest
uv run ruff check .
uv run mypy src
uv build
uv run sphinx-build -b html docs/source docs/built
```

Common shortcuts are available through `make`:

```bash
make check
make docs
make clean
```

## Documentation

Documentation is built with Sphinx and Furo from `docs/source/` into `docs/built/`. Read the Docs should build the documentation through its normal repository integration.

## Releases

Releases are managed with Python Semantic Release and Conventional Commits. The GitHub Actions release workflow is intended to create the release commit, tag, GitHub release, wheel, sdist, and PyPI publication through trusted publishing.
