Metadata-Version: 2.4
Name: isitright
Version: 0.1.0.dev0
Summary: Evidence-first stress testing for text-to-SQL evaluations
Project-URL: Repository, https://github.com/isitright-ai/isitright
Author: isitright contributors
License: MIT License
        
        Copyright (c) 2026 isitright contributors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
License-File: LICENSE
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Requires-Dist: pydantic<3,>=2.10
Requires-Dist: pyyaml<7,>=6.0.2
Requires-Dist: sqlglot<29,>=27
Requires-Dist: typer<1,>=0.15
Provides-Extra: dev
Requires-Dist: build<2,>=1.2.2; extra == 'dev'
Requires-Dist: mypy<2,>=1.14; extra == 'dev'
Requires-Dist: pytest-cov<8,>=6; extra == 'dev'
Requires-Dist: pytest<9,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.9; extra == 'dev'
Requires-Dist: twine<7,>=6.1; extra == 'dev'
Description-Content-Type: text/markdown

# isitright

`isitright` is an evidence-first tool for stress-testing text-to-SQL evaluations.
Given reference and candidate SQL, the eventual v0.1 engine will search for a valid
database state on which their outputs differ and emit a minimized, replayable
counterexample. A disagreement is evidence, not automatic attribution of fault.

## Current status: Stage 0 review gate

This repository currently contains package scaffolding and specification contracts
only. SQL execution, database mutation, constraint validation, search, comparison,
and minimization are deliberately not implemented. The proposed fixtures under
`tests/fixtures/proposals` require owner approval before Stage 1 begins.

## Development

Python 3.11 or newer is required. With `uv` installed:

```console
uv sync --extra dev
uv run ruff check .
uv run mypy src
uv run pytest
uv build
```

The package exposes the future command surface now:

```console
isitright --version
isitright check --help
isitright suite --help
```

Calling `check` or `suite` during Stage 0 exits with the internal/not-implemented
code because no analysis engine exists yet.

The Stage 0 preview is versioned `0.1.0.dev0`. It is intentionally a development
release and must not be presented as an implemented analysis engine.

## Contract highlights

- Result statuses keep observed equivalence, state validity, and attribution separate.
- Unordered query results will be compared as multisets, preserving duplicates.
- Automatic ordering will honor a top-level `ORDER BY`; otherwise results are unordered.
- SQLite values are represented with explicit storage classes.
- Reports use schema version `0.1`; package versioning remains pre-release until an
  engine is implemented.

See [the contract reference](docs/contracts.md) and
[the fixture proposal](tests/fixtures/proposals/README.md).

## License

MIT

