Metadata-Version: 2.4
Name: yez
Version: 1.2.0
Summary: YEz maximum-ratio lossless compressor: match parsing + context mixing, CRC-verified, with a from-scratch C++17 core
Author: Ahmed Alhrrani
Maintainer: Ahmed Alhrrani
License-Expression: MIT
Project-URL: Homepage, https://t.me/maho_s9
Project-URL: Author, https://t.me/maho_s9
Project-URL: Changelog, https://t.me/maho_s9
Keywords: compression,lossless,archive,compressor,context-mixing,entropy-coding,lz77,cm
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: System :: Archiving :: Compression
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# yez — maximum-ratio lossless compression

**YEz** is a from-scratch, multi-stage lossless compressor (C++17 core, Python
API). It combines match parsing with a hash-chain/optimal parser, bitwise
context mixing with 12 context models, x86/ARM64/delta filters, and a
CRC32-verified container. No external compression library is used anywhere —
`pip install yez` compiles the core from source on your machine.

Every archive is **lossless and verified**: the original data's CRC32 is
stored and `decompress()` never returns anything that does not match
byte-for-byte.

Developed by **Ahmed Alhrrani** — [t.me/maho_s9](https://t.me/maho_s9).

## Install

```bash
pip install yez
```

The package needs a C++17 compiler at install time; it picks the first
available of `c++`, `g++`, `clang++` (or `$CXX` if set). No prebuilt binaries
are downloaded — the core is always compiled from the bundled sources:

```bash
# no system compiler? the ziglang pip package works too:
pip install ziglang
CXX="python -m ziglang c++" pip install yez
```

## Quick start

```python
import yez

z = yez.compress(b"hello " * 100000, level=9)   # -> bytes (a .yez archive)
data = yez.decompress(z)                         # -> exact original bytes

yez.compress_file("big.csv", "big.csv.yez", level=7)
yez.decompress_file("big.csv.yez", "big.csv")

info = yez.inspect(z)     # dict: version, mode, level, sizes, crc, blocks[]
ok = yez.check(z)         # structural validation without decompressing
yez.is_yez(z)             # magic-byte test
```

## Measured results

Mixed 6.3 MB corpus of real files (text, CSV, JSON, XML, HTML), compared with
Python's stdlib codecs on the same machine:

| Codec | Total bytes | % of original | vs `lzma -9e` |
|---|---:|---:|---:|
| gzip -9 | 1,115,692 | 17.7% | +62% |
| bzip2 -9 | 760,867 | 12.1% | +11% |
| lzma -9e (xz) | 688,240 | 10.9% | — |
| yez level 1 | 928,199 | 14.8% | +35% (−17% vs gzip -9) |
| yez level 5 | 696,144 | 11.1% | +1% |
| **yez level 9** | **447,126** | **7.1%** | **−35%** |

Per-file highlights (level 9 vs `lzma -9e`): text −25%, CSV −52%, JSON −28%,
XML −38%, HTML −21%.

## Levels

| Level | Mode | What runs | Use for |
|---|---|---|---|
| 1–3 | Fast | greedy/lazy/optimal match parsing | speed, still beats `gzip -9` |
| 4–6 | Balanced | optimal parsing + per-block tuning | everyday default (≈ `xz -9e`) |
| 7–9 | Maximum | parsing **+ bitwise context mixing** (12 models) | archival, maximum ratio |

Level 9 is slow by design (context mixing is symmetric: decompression does
the same modelling work, ≈0.1–0.2 MB/s each way). Level 5 decompresses at
~30 MB/s.

## API reference

```python
yez.compress(data, level=5, *, threads=None, block_size=0, block_crc=False) -> bytes
yez.decompress(data, *, threads=None) -> bytes
yez.compress_file(src, dst=None, level=5, *, threads=None, block_size=0, block_crc=False) -> int
yez.decompress_file(src, dst=None, *, threads=None) -> int
yez.inspect(data) -> dict
yez.check(data) -> bool
yez.is_yez(data) -> bool
yez.YezError                    # raised on corruption / bad input
yez.MAGIC                      # b"YEz"
yez.__version__
```

* Accepts `bytes`, `bytearray` and `memoryview`.
* `threads=None` picks automatically: all cores for levels 1–6, capped to 3
  concurrent blocks for 7–9 (each context-mixing block holds ~400 MB of model
  state — the cap bounds peak memory; pass an explicit number to override).
* `block_size` (bytes; 0 = level default 4/16/32 MiB) and `block_crc=True`
  (per-block CRC32 — pinpoints which block is damaged) are available for
  tuning.
* Calls release the GIL, so one worker per thread scales from Python.
* `yez.YezError` is raised for bad magic, unsupported version, truncation,
  trailing data, header checksum mismatch or payload corruption — corrupt
  archives never yield wrong output silently.

## Command line

```text
yez c [-c N] [-t N] [-x] input output.yez    compress
yez d [-t N] input.yez output                decompress
yez t input.yez                              test integrity
yez i input.yez                              archive information
```

The same interface works as `python -m yez …`.

## How it works

Input is split into blocks; each block is analysed (constancy, entropy,
text-likeness, record stride, x86/ARM64 filter scores), candidate transforms
are ranked with a cheap proxy parse, then the block is encoded with the best
of: raw / constant / **repstream** (match parsing with a context-modelled
binary range coder: greedy, lazy or price-optimal parsing, per-block lc/pb
tuning) / **ctxmix** (bitwise context mixing: 12 hashed context models —
orders 1–7, a UTF-8 aware word model, sparse, record-stride and an indirect
model — two gated mixers and two APM/SSE stages). The smallest actual output
wins; incompressible data never expands by more than ~25 bytes.

The binding is plain `ctypes` over a C ABI (`yez_compress`,
`yez_decompress`, `yez_check`, `yez_inspect_json`, `yez_free`,
`yez_last_error`), so the same core is usable from any FFI.

## License

MIT — see [LICENSE](LICENSE). Developed by **Ahmed Alhrrani**
([t.me/maho_s9](https://t.me/maho_s9)).
