Metadata-Version: 2.4
Name: diffolars
Version: 0.3.0
Summary: Yet another data diff
Author: Kenneth O'Dell
Author-email: Kenneth O'Dell <kenneth.l.odell@gmail.com>
License-Expression: MIT
Requires-Dist: dotenv>=0.9.9
Requires-Dist: ipykernel>=7.3.0
Requires-Dist: jupyter>=1.1.1
Requires-Dist: pandas>=3.0.3
Requires-Dist: polars>=1.41.2
Requires-Dist: pyarrow>=24.0.0
Requires-Dist: pyodbc>=5.3.0
Requires-Dist: pytest>=9.1.1
Requires-Dist: ruff>=0.15.20
Requires-Dist: sqlalchemy>=2.0.50
Requires-Python: >=3.13
Description-Content-Type: text/markdown

# diffolars

A small [Polars](https://pola.rs)-based toolkit for
comparing two versions of a dataframe and generating randomized test data to
exercise that comparison.

Ideally used to compare dataloads in the day-to-day of a database analyst.

## Installation

```bash
uv add diffolars
```

## Generating test data

`diffolars.demo` generates a random dataframe and a mutated copy of it, useful
for testing diff logic without hand-crafting fixtures.

```python
from diffolars.demo import get_df_pair

pair = get_df_pair(
    n_rows=100,
    n_cols=10,
    n_new_rows=5,    # rows added in the mutated copy
    n_new_cols=2,    # columns added in the mutated copy
    coverage=0.1,    # fraction of existing cells randomly changed
    seed=42,
)

original = pair["original"]
mutated = pair["mutated"]
```

Every generated row gets a `record_id` UUID column, used to match rows
between the original and mutated dataframes. `get_random_data` and
`get_mutated_data` are also available individually if you want to generate or
mutate a dataframe on its own.

## Diffing

`diffolars.diff` provides the comparison API for two dataloads (e.g. a
previous load vs. the latest load), and is under active development.

Inputs to these functions can be a `list`, `polars.DataFrame`, or
`polars.LazyFrame`.

- `column_intercept` / `column_symmetric_diff` — shared vs. exclusive columns
  between the two tables
- `row_intercept` / `row_symmetric_diff` — shared vs. exclusive rows, based on
  a record ID column
- `prune_rows` — returns the rows exclusive to each table (i.e. dropped by the
  join), tagged with which table they came from
- `report_prune` — summarizes the pruned rows/columns as a single dict entry,
  suitable for logging
- `get_core` — joins the two tables down to their shared rows and columns, for
  doing the field-to-field comparison

Currently only the Windows build is available.

## License

MIT — see [LICENSE](LICENSE).
