Metadata-Version: 2.4
Name: sortdx
Version: 0.2.0
Summary: Sort CSV, TSV, JSONL, and text files of any size, with gzip and zstandard support
Author: Okymi-X
License: MIT
Project-URL: Homepage, https://github.com/Okymi-X/sortdx
Project-URL: Repository, https://github.com/Okymi-X/sortdx
Project-URL: Issues, https://github.com/Okymi-X/sortdx/issues
Project-URL: Documentation, https://github.com/Okymi-X/sortdx#readme
Keywords: sorting,csv,tsv,jsonl,external-sort,natural-sort,cli,data-processing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Text Processing :: Filters
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: full
Requires-Dist: typer>=0.9.0; extra == "full"
Requires-Dist: rich>=13.0.0; extra == "full"
Requires-Dist: chardet>=5.0.0; extra == "full"
Requires-Dist: python-dateutil>=2.8.0; extra == "full"
Provides-Extra: zstd
Requires-Dist: zstandard>=0.20.0; extra == "zstd"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: isort>=5.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: types-python-dateutil>=2.8.0; extra == "dev"
Dynamic: license-file

# sortdx

[![CI](https://github.com/Okymi-X/sortdx/actions/workflows/ci.yml/badge.svg)](https://github.com/Okymi-X/sortdx/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/sortdx)](https://pypi.org/project/sortdx/)
[![Python versions](https://img.shields.io/pypi/pyversions/sortdx)](https://pypi.org/project/sortdx/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

A sorting library and command line tool for Python that handles CSV, TSV, JSONL, and text files, including gzip and zstandard compressed files. Large files are sorted on disk through chunking and a k-way merge, so memory usage stays bounded. The core library has no required dependencies and ships full type information.

## Installation

```bash
pip install sortdx
```

The core library has no required dependencies. The command line interface and extra features (encoding detection, flexible date parsing) need the optional dependencies:

```bash
pip install "sortdx[full]"
```

## Quick start

```python
import sortdx

# Sort a CSV by a numeric column
sortdx.sort_file("data.csv", "sorted.csv", keys=[sortdx.key("age", "num")])

# Sort a JSONL file by timestamp
sortdx.sort_file("logs.jsonl", "sorted.jsonl", keys=[sortdx.key("timestamp", "date")])

# Sort in-memory data
data = [{"name": "Bob", "age": 30}, {"name": "Alice", "age": 25}]
result = list(sortdx.sort_iter(data, keys=[sortdx.key("age", "num")]))
```

## Sort keys

Create sort keys with `sortdx.key()`:

```python
# String sorting (default, case-insensitive)
sortdx.key("name")

# Numeric sorting
sortdx.key("price", "num")

# Date sorting
sortdx.key("created_at", "date")

# Natural sorting: file2 before file10
sortdx.key("filename", "nat")

# Descending order for one key
sortdx.key("salary", "num", desc=True)

# Locale-aware string collation
sortdx.key("name", "str", locale_name="fr_FR.UTF-8")
```

Columns are addressed by name for dict rows (CSV, JSONL), by integer index for list rows, and `None` sorts plain text lines as whole items.

### Data types

| Type | Description |
|------|-------------|
| `str` | Case-insensitive text sorting (default) |
| `num` | Numeric sorting for integers and floats |
| `date` | Date and time sorting; ISO 8601 always works, other formats need `python-dateutil` |
| `nat` | Natural sorting where embedded numbers compare by value |

Empty and unparseable values sort first within their type.

## API

### `sortdx.sort_file(input_path, output_path, keys, ...)`

Sort a file into another file.

| Parameter | Description |
|-----------|-------------|
| `input_path` | Input file path |
| `output_path` | Output file path |
| `keys` | List of sort keys |
| `memory_limit` | Memory budget such as `"512M"` or `"2G"`; larger inputs are sorted on disk (default threshold: 100M) |
| `reverse` | Reverse the entire sort order |
| `unique` | Column whose values must be unique in the output |
| `stats` | Return a `SortStats` object |

```python
stats = sortdx.sort_file(
    "huge.jsonl.gz",
    "sorted.jsonl.gz",
    keys=[sortdx.key("timestamp", "date")],
    memory_limit="512M",
    stats=True,
)
print(f"Processed {stats.lines_processed} lines in {stats.processing_time:.2f}s")
```

### `sortdx.sort_iter(data, keys, ...)`

Sort an iterable in memory and return an iterator. Accepts the same `reverse` and `unique` arguments.

### Multi-key sorting

```python
sortdx.sort_file("employees.csv", "sorted.csv", keys=[
    sortdx.key("department"),
    sortdx.key("salary", "num", desc=True),
])
```

## Command line

```bash
# Sort a CSV by price, then name; write to a file
sortdx data.csv -o sorted.csv -k price:num -k name:str

# Sort a compressed JSONL file with a memory budget
sortdx logs.jsonl.gz -o sorted.jsonl.gz -k timestamp:date --memory-limit=512M

# Natural sort of a text file to stdout
sortdx filenames.txt --natural

# Keep one row per user id, newest first
sortdx users.jsonl -o unique.jsonl -k created_at:date --reverse --unique=user_id
```

Key specifications use the format `column:type[:option=value]`, for example `price:num`, `name:str:locale=fr_FR.UTF-8`, or `rank:num:desc=true`.

## Supported formats

| Format | Extensions |
|--------|-----------|
| CSV | `.csv` (delimiter auto-detected) |
| TSV | `.tsv`, `.tab` |
| JSON Lines | `.jsonl`, `.ndjson`, `.json` |
| Plain text | `.txt` and anything else |
| Compression | `.gz`, `.gzip`, `.zst`, `.zstd` (zstandard requires `pip install zstandard`) |

## Development

```bash
git clone https://github.com/Okymi-X/sortdx.git
cd sortdx
pip install -e ".[full,dev]"
pytest
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the full set of checks (formatting, lint, type checking) and the contribution guidelines.

## Changelog

Release history is documented in [CHANGELOG.md](CHANGELOG.md).

## License

MIT License. See the [LICENSE](LICENSE) file for details.

## Contributing

Contributions are welcome. Please open an issue or submit a pull request on [GitHub](https://github.com/Okymi-X/sortdx).
