Metadata-Version: 2.4
Name: tabular-profiler
Version: 0.1.0
Summary: Profile CSV and Parquet files from the command line: inferred types, suggested dtypes, null/unique counts, min/max, and data-quality checks.
Author: Rehan Mahmood
License: MIT
Project-URL: Homepage, https://github.com/Rxh4n/tabular-profiler
Project-URL: Repository, https://github.com/Rxh4n/tabular-profiler
Project-URL: Issues, https://github.com/Rxh4n/tabular-profiler/issues
Project-URL: Changelog, https://github.com/Rxh4n/tabular-profiler/blob/main/CHANGELOG.md
Keywords: data,profiling,csv,parquet,pandas,cli,data-quality,dtype
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: pandas>=1.5
Requires-Dist: pyarrow>=10.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# tabular-profiler

[![CI](https://github.com/Rxh4n/tabular-profiler/actions/workflows/ci.yml/badge.svg)](https://github.com/Rxh4n/tabular-profiler/actions/workflows/ci.yml)
[![Python](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

A small command-line tool that profiles a **CSV, TSV, or Parquet** file and
tells you what is actually in each column — the inferred type, a suggested
pandas dtype, null and unique counts, min/max — and flags common data problems
such as columns that mix types or dates stored as plain strings.

```console
$ profiler data.csv
```

Built with [Click](https://click.palletsprojects.com/),
[pandas](https://pandas.pydata.org/), and
[PyArrow](https://arrow.apache.org/docs/python/).

---

## Installation

Once published to PyPI:

```console
pip install tabular-profiler
```

This installs a `profiler` command on your `PATH`.

To install from a checkout of this repository:

```console
pip install .
# or, for development (editable install with test/build tooling):
pip install -e ".[dev]"
```

Requires Python 3.9+.

## Usage

Point it at a file:

```console
$ profiler examples/sample.csv
File:    examples/sample.csv
Format:  csv
Rows:    8
Columns: 6

column       inferred  stored  suggested       nulls    unique  min                  max
-----------  --------  ------  --------------  -------  ------  -------------------  -------------------
id           integer   int64   int8            0 (0%)   8       1                    8
signup_date  datetime  str     datetime64[ns]  0 (0%)   8       2021-01-15 00:00:00  2021-08-09 00:00:00
price        mixed     str     string          0 (0%)   8       15.00                oops
country      string    str     category        1 (12%)  3       CA                   US
notes        string    str     string          0 (0%)   8         welcome            regular
active       boolean   bool    bool            0 (0%)   2       False                True

Data quality findings:
  - [id] every value is unique; looks like an identifier
  - [signup_date] dates stored as text; suggest parsing to datetime64[ns]
  - [price] mixes multiple value types (float: 7, string: 1)
  - [notes] every value is unique; looks like an identifier
  - [notes] 4 value(s) have leading/trailing whitespace
```

### Options

| Option | Description |
| --- | --- |
| `-f`, `--format [table\|json]` | Output format (default: `table`). |
| `-n`, `--sample N` | Only read the first N rows — handy for very large files. |
| `-o`, `--output FILE` | Write the report to a file instead of stdout. |
| `--version` | Show the version and exit. |
| `-h`, `--help` | Show help and exit. |

### JSON output

`--format json` emits a machine-readable report you can feed into other tools:

```console
$ profiler data.csv --format json
{
  "path": "data.csv",
  "file_format": "csv",
  "n_rows": 8,
  "n_columns": 6,
  "columns": [
    {
      "name": "price",
      "stored_dtype": "str",
      "inferred_type": "mixed",
      "suggested_dtype": "string",
      "count": 8,
      "null_count": 0,
      "null_pct": 0.0,
      "unique_count": 8,
      "unique_pct": 100.0,
      "min": "15.00",
      "max": "oops",
      "issues": ["mixes multiple value types (float: 7, string: 1)"]
    }
  ],
  "dataset_issues": []
}
```

## What it reports

For every column:

- **inferred type** — the *logical* type worked out from the values
  (`integer`, `float`, `boolean`, `datetime`, `string`, `mixed`, or `empty`),
  which can differ from how the data was stored.
- **stored** — the dtype pandas used to load the column.
- **suggested** — a recommended dtype. Suggestions aim to be both correct and
  memory-efficient:
  - the narrowest signed integer width that fits (`int8` … `int64`);
  - `Int64` (nullable) for integer columns with missing values, and for float
    columns whose values are all whole numbers;
  - `datetime64[ns]` for dates that were stored as text;
  - `category` for low-cardinality string columns;
  - `string` otherwise.
- **nulls** — count and percentage of missing values.
- **unique** — number of distinct non-null values.
- **min / max** — numeric/chronological for numbers and dates, alphabetical for
  text.

## Data-quality checks

Per column:

- **Mixed types** — values that don't agree on a type (e.g. mostly numbers with
  a few stray strings). Integers and floats together are *not* flagged — that's
  just a float column.
- **Dates stored as text** — text columns whose values parse as dates.
- **Numbers stored as text** — text columns whose values are all numeric.
- **High null fraction** — at least half the values are missing (entirely empty
  columns are called out separately).
- **Constant column** — only one distinct value.
- **Likely identifier** — every value is unique.
- **Surrounding whitespace** — values with leading/trailing spaces.

Per dataset:

- **Duplicate rows** and **duplicate column names**.

## Development

```console
python -m venv .venv
.venv\Scripts\activate            # Windows
# source .venv/bin/activate       # macOS/Linux
pip install -e ".[dev]"
pytest
```

The package uses a `src/` layout:

```
src/data_profiler/
  cli.py          # Click entry point (the `profiler` command)
  io.py           # CSV/Parquet loading + format detection
  inference.py    # logical type inference from raw values
  checks.py       # data-quality checks
  profiler.py     # orchestration -> DataProfile
  report.py       # text and JSON rendering
```

You can also use it as a library:

```python
import pandas as pd
from data_profiler.profiler import profile_dataframe

profile = profile_dataframe(pd.read_csv("data.csv"), path="data.csv")
for column in profile.columns:
    print(column.name, column.inferred_type, column.issues)
```

## Publishing to PyPI

Releases are cut by pushing a version tag. The
[`publish`](.github/workflows/publish.yml) workflow then builds the sdist +
wheel and uploads them via PyPI
[Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (no API token
needed once the project is configured on PyPI).

```console
# 1. bump the version in src/data_profiler/__init__.py and update CHANGELOG.md
# 2. tag and push
git tag v0.1.0
git push origin v0.1.0
```

To build and upload manually instead:

```console
python -m build
twine upload dist/*
```

> **Note on the package name:** PyPI names must be globally unique. If
> `tabular-profiler` is taken, change `name` in `pyproject.toml` (the import
> package `data_profiler` and the `profiler` command can stay the same).

## License

[MIT](LICENSE)
