Metadata-Version: 2.4
Name: llm-code-validator
Version: 0.1.0
Summary: CLI guardrail for catching stale Python APIs before runtime.
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Dynamic: license-file

# llm-code-validator

Python CLI for detecting stale or version-incompatible third-party API usage in Python source code.

## Installation

Install from the repository:

```bash
git clone https://github.com/mathew-felix/llm-code-validator
cd llm-code-validator
pip install -e .
```

Install with test dependencies:

```bash
pip install -e ".[dev]"
```

After the package is published to PyPI:

```bash
pip install llm-code-validator
```

## Usage

Check one file:

```bash
llm-code-validator check file.py
```

Check a directory:

```bash
llm-code-validator check src/
```

Check standard input:

```bash
llm-code-validator check - < snippet.py
```

Check staged Git files:

```bash
llm-code-validator check --staged
```

Use an explicit dependency file:

```bash
llm-code-validator check --requirements requirements.txt src/
```

Show low-confidence diagnostics:

```bash
llm-code-validator check --show-low-confidence src/
```

Exit codes:

- `0`: no diagnostics
- `1`: diagnostics found
- `2`: tool error

## Output

Text output:

```bash
llm-code-validator check src/
```

JSON output:

```bash
llm-code-validator check src/ --format json
```

GitHub Actions annotation output:

```bash
llm-code-validator check src/ --format github
```

## Fixes

Preview fixes:

```bash
llm-code-validator fix file.py
```

Apply safe fixes:

```bash
llm-code-validator fix file.py --write
```

Only rules marked `safe_fix` are written. Rules marked `suggested_fix` or `no_fix` are reported but not changed.

Current rule safety counts:

- `safe_fix`: 15 rules
- `suggested_fix`: 51 rules
- `no_fix`: 2 rules

## Signature Database

Validate the rule database:

```bash
llm-code-validator validate-signatures
```

The source rule database is:

```text
data/library_signatures.json
```

The packaged rule database is:

```text
llm_code_validator/library_signatures.json
```

Current rule count:

- 68 API-drift rules

## Benchmarks

Run the CLI benchmark dataset:

```bash
python -m llm_code_validator.benchmark --dataset validation_dataset/cli_benchmark_cases.json
```

Run the AI-stack benchmark dataset:

```bash
python -m llm_code_validator.benchmark --dataset validation_dataset/ai_stack_benchmark_cases.json
```

Current saved benchmark results:

- CLI dataset: precision `1.0`, recall `1.0`, p50 `0.243ms`, p95 `6.199ms`
- AI-stack dataset: precision `1.0`, recall `1.0`, p50 `0.444ms`, p95 `4.939ms`

## Pre-Commit

`.pre-commit-hooks.yaml` is included:

```yaml
repos:
  - repo: https://github.com/mathew-felix/llm-code-validator
    rev: v0.1.0
    hooks:
      - id: llm-code-validator
```

## GitHub Actions

Example workflow:

```yaml
name: API Drift Check

on:
  pull_request:

jobs:
  api-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install llm-code-validator
      - run: llm-code-validator check . --format github
```

## Testing

Run the test suite:

```bash
pytest -q
```

Current local result:

```text
72 passed
```

## Documentation

- `docs/demo.md`: example check and fix workflow
- `docs/rules.md`: rule database notes
- `docs/release.md`: package release steps
- `PROJECT_REPORT.md`: project report
