Metadata-Version: 2.4
Name: research-metadata-auditor
Version: 0.1.0
Summary: Audit Crossref DOI metadata for completeness and citation readiness.
Keywords: crossref,doi,metadata,research,data-quality
Author: Md Mehedi Hasan
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Dist: httpx>=0.28.1,<1
Requires-Dist: pydantic>=2.11,<3
Requires-Dist: python-dotenv>=1.1,<2
Requires-Dist: typer>=0.16,<1
Requires-Dist: pandas>=2.3,<4 ; extra == 'dashboard'
Requires-Dist: streamlit>=1.47,<2 ; extra == 'dashboard'
Requires-Python: >=3.11
Project-URL: Homepage, https://github.com/mehedi-hridoy99/research-metadata-auditor
Project-URL: Repository, https://github.com/mehedi-hridoy99/research-metadata-auditor
Project-URL: Issues, https://github.com/mehedi-hridoy99/research-metadata-auditor/issues
Provides-Extra: dashboard
Description-Content-Type: text/markdown

# Research Metadata Auditor

Research Metadata Auditor checks whether Crossref records contain the metadata
needed for common citation and research-data tasks. It accepts DOI values from a
CSV, TSV, plain-text, or BibTeX file, preserves complete API evidence, and writes
a smaller audit dataset for review.

The project is for graduate researchers and research teams who want a
repeatable check before using DOI records in a bibliography, literature review,
or dataset. The score measures field availability. It does not verify that
publisher-deposited metadata is factually correct.

## Fastest Path

Requirements:

- Python 3.11 or newer
- [uv](https://docs.astral.sh/uv/)
- Internet access for a new Crossref request

Clone the repository and restore the locked environment:

```bash
git clone https://github.com/mehedi-hridoy99/research-metadata-auditor.git
cd research-metadata-auditor
uv sync --all-extras --dev
```

Run the committed sample input:

```bash
uv run metadata-audit audit examples/sample-dois.csv
```

The command writes:

- `data/raw/crossref-responses.json`: complete response evidence and acquisition
  context
- `data/processed/metadata-audit.csv`: one stable audit row per unique DOI
- `data/cache/crossref.sqlite3`: local cache for successful responses

Open the interactive result:

```bash
uv run metadata-audit dashboard
```

A committed example of the raw and processed outputs is included so the project
can be inspected without contacting Crossref.

## Configuration and Responsible API Use

The [Crossref REST API](https://www.crossref.org/documentation/retrieve-metadata/rest-api/)
is public and does not require a key. Crossref recommends a contact email,
identifying user agent, and local caching. Copy the example file and replace the
placeholder locally:

```bash
cp .env.example .env
```

On Windows PowerShell:

```powershell
Copy-Item .env.example .env
```

```text
CROSSREF_MAILTO=your-email@example.com
```

`.env` is ignored by Git. A contact email is not an API secret, but it is still
personal information and should not be committed. The tool never prints the
email value.

Anonymous runs use one request at a time. Identified runs may request up to
three concurrent operations:

```bash
uv run metadata-audit audit examples/sample-dois.csv --concurrency 3
```

These limits follow the current
[Crossref access guidance](https://www.crossref.org/documentation/retrieve-metadata/rest-api/access-and-authentication/).

## Input Formats

CSV and TSV files must contain a case-insensitive `doi` column. Other columns
are allowed.

```csv
label,doi
example,10.1128/mbio.01735-25
```

Plain-text files contain one DOI per line. Blank lines and lines beginning with
`#` are ignored. BibTeX files are read from their `doi` fields.

Resolver URLs and a leading `doi:` label are normalized:

```text
https://doi.org/10.1038/s41586-020-2649-2
doi:10.1038/s41586-021-03819-2
```

If any input value is invalid, the command lists the bad items and exits before
contacting Crossref.

## Commands

Show command help:

```bash
uv run metadata-audit --help
uv run metadata-audit audit --help
```

Choose explicit output and cache locations:

```bash
uv run metadata-audit audit examples/sample-dois.csv \
  --raw-output data/raw/my-run.json \
  --processed-output data/processed/my-run.csv \
  --cache data/cache/metadata.sqlite3
```

Ignore the cache and request current records:

```bash
uv run metadata-audit audit examples/sample-dois.csv --refresh
```

Summarize an existing processed file without using the API:

```bash
uv run metadata-audit summary data/processed/metadata-audit.csv
```

Show setup status without displaying private configuration:

```bash
uv run metadata-audit project-info
```

Write operational logs:

```bash
uv run metadata-audit audit examples/sample-dois.csv \
  --log-level DEBUG \
  --log-file logs/metadata-audit.log
```

Logs describe operations. They do not include response payloads or the configured
contact email.

## Dashboard

Start the local Streamlit dashboard:

```bash
uv run metadata-audit dashboard
```

The dashboard loads `data/processed/metadata-audit.csv` by default. It can also
load another path or an uploaded processed CSV. It provides:

- record, ready, needs-attention, and average-score metrics
- status and score-range charts
- status and minimum-score filters
- a review table containing the citation fields and missing-field explanation

Install the released package and dashboard without cloning:

```bash
pip install "research-metadata-auditor[dashboard]"
metadata-audit project-info
```

Run `metadata-audit audit` on your own input before starting the dashboard.

## Scoring and Interpretation

The score is a 100-point field-availability check. Title and author availability
receive the most weight. Publication year, container title, publisher, work
type, landing URL, ORCID, and license data contribute the remaining points.

The processed status is one of:

- `ready`
- `review`
- `incomplete`
- `not_found`
- `api_error`
- `invalid_response`

See [Metadata Scoring Rules](docs/scoring.md) for exact weights and status rules.
Do not use the score to rank research quality or assume that available fields
are correct.

## Testing and Quality Checks

Automated tests use labeled synthetic fixtures, `httpx.MockTransport`, and a
fake client. They do not call the live API.

Run the tests:

```bash
uv run pytest
```

Run the same quality contract used by CI:

```bash
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run pytest \
  --cov=research_metadata_auditor \
  --cov-report=term-missing \
  --cov-fail-under=80
uv build --no-sources
```

GitHub Actions runs these checks on Python 3.11, 3.12, and 3.13 for every push
and pull request.

## Packaging and Release

Build the wheel and source distribution:

```bash
uv build --no-sources
```

The release workflow publishes through PyPI trusted publishing when a GitHub
release is published. No long-lived PyPI token belongs in the repository. The
repository owner must configure the `pypi` environment and the matching trusted
publisher in PyPI before the first release.

## Project Structure

```text
src/research_metadata_auditor/  package, API, validation, cache, CLI, dashboard
tests/                           offline tests and labeled synthetic fixtures
examples/sample-dois.csv         small real DOI input for the showcase run
data/raw/                         committed real Crossref response evidence
data/processed/                   committed processed audit dataset
docs/                             specification, architecture, scoring, dictionary
submission/                       reflection draft and release checklist
.github/workflows/                CI and PyPI release automation
pyproject.toml                    package metadata and tool configuration
uv.lock                           reproducible dependency resolution
```

## Documentation

- [Project specification](docs/specification.md)
- [Architecture](docs/architecture.md)
- [Metadata scoring rules](docs/scoring.md)
- [Data dictionary](docs/data-dictionary.md)
- [Learning note](docs/learning-note.md)
- [Contributor guidance](CONTRIBUTING.md)
- [MIT License](LICENSE)

## Common Failures

**Input contains invalid DOI values**

Correct the listed rows or lines. The tool does not make partial API requests
when DOI syntax is invalid.

**Crossref returns HTTP 429**

Reduce `--concurrency`, keep the cache enabled, and retry later. Do not increase
request frequency.

**A record is `invalid_response`**

Keep the raw evidence and inspect the error column. The response returned HTTP
200 but did not contain the fields required by the current Pydantic contract.

**The dashboard dependency is missing**

Run `uv sync --all-extras` in the repository or install
`research-metadata-auditor[dashboard]`.

**The dashboard has no data**

Run the audit command first or select a processed CSV in the dashboard sidebar.

## Known Limitations

- Crossref does not register every DOI. DataCite and other registration agencies
  are not queried in version 0.1.0.
- Crossref fields reflect deposited metadata and can be incomplete or incorrect.
- Citation counts change and should be interpreted as acquisition-time values.
- DOI validation supports the common Crossref syntax documented by this project;
  unusual valid suffixes outside that syntax may be rejected.
- The SQLite cache is local and is not designed for concurrent multi-user writes.
- The dashboard runs locally and is not deployed.

