Metadata-Version: 2.4
Name: z85base91
Version: 0.0.6a8
Summary: base91, z85b and z85p encodings
Author-email: jarbasAI <jarbasai@mailfence.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/JarbasHiveMind/z85base91
Keywords: hivemind,encoding,z85,base91
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

# z85base91

A binary-to-text encoding library. It provides three codecs: **Base91**, **Z85B**, and **Z85P**. Each codec turns binary data into printable ASCII text for transport over text-oriented channels, with a fast C implementation and a pure-Python fallback, so the library works on any platform. The HiveMind mesh uses it to pack binary payloads for text-mode transports.

## Why z85base91?

Base64 expands data by 33% (1.33x). Base91 expands data by only 23% (1.23x), and stays more compact than base32 (62% overhead). The Z85 variants sit between these: Z85B expands data by 1.25x, and Z85P by 1.28x. Both add character-safety guarantees useful for sending encrypted frames over websockets and other text transports.

### Comparison

| Codec | Expansion | Use Case |
|-------|-----------|----------|
| **Base91** | 1.23x (smallest) | Maximum compactness, printable ASCII |
| **Z85B** | 1.25x | Z85 padding scheme, per-group framing |
| **Z85P** | 1.28x | Z85 padding scheme, prepended size byte |
| base64 | 1.35x | Standard, less compact |
| base32 | 1.62x | Case-insensitive, largest |

## Installation

```bash
pip install z85base91
```

The package bundles precompiled C extensions for x86_64, i386, and aarch64. If the C library fails to load, the package falls back to pure Python and logs a warning.

## Quick Start

```python
from z85base91 import B91, Z85B, Z85P

# Base91 (most compact)
data = b"Hello, World!"
encoded = B91.encode(data)        # b'>OwJh>}AQ;r@@Y?F'
decoded = B91.decode(encoded)     # b'Hello, World!'

# Z85B (Z85 with independent groups)
encoded = Z85B.encode(data)
decoded = Z85B.decode(encoded)

# Z85P (Z85 with prepended padding byte)
encoded = Z85P.encode(data)
decoded = Z85P.decode(encoded)
```

All three codecs accept `str` or `bytes` input and return `bytes`.

## Public API

### Base91 (`B91`)

```python
from z85base91 import B91
```

**Encoding**

```python
B91.encode(data: Union[str, bytes], encoding: str = "utf-8") -> bytes
```

Encodes binary data with Base91. Base91 uses 91 printable ASCII characters (A-Z, a-z, 0-9, plus 27 symbols) and expands data by 1.23x.

**Arguments:**
- `data`: Input as `str` (encoded to UTF-8) or raw `bytes`.
- `encoding`: Character encoding to use if `data` is `str`. Default: `"utf-8"`.

**Returns:** Base91-encoded bytes.

**Example:**
```python
B91.encode(b"test")           # b'fPNKd'
B91.encode("test")            # b'fPNKd' (auto UTF-8)
B91.encode(b"\x00\x01\x02")   # b':CQA'
```

**Decoding**

```python
B91.decode(encoded_data: Union[str, bytes], encoding: str = "utf-8") -> bytes
```

Decodes Base91-encoded input back to raw bytes.

**Arguments:**
- `encoded_data`: Base91 string or bytes.
- `encoding`: Character encoding to use for string conversion. Default: `"utf-8"`.

**Returns:** Decoded bytes.

**Raises:** `ValueError` if the input has a character outside the 91-character alphabet.

**Example:**
```python
B91.decode(b'>OwJh>}AQ;r@@Y?F')  # b'Hello, World!'
B91.decode('>OwJh>}AQ;r@@Y?F')   # b'Hello, World!' (str input)
```

---

### Z85B (`Z85B`)

```python
from z85base91 import Z85B
```

**Encoding**

```python
Z85B.encode(data: Union[str, bytes]) -> bytes
```

Encodes binary data with Z85B, a Z85 variant that processes 4-byte chunks independently. It uses the 85-character Z85 alphabet: `0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.-:+=^!/*?&<>()[]{}@%$#`. Expansion is about 1.25x.

**Arguments:**
- `data`: Input as `str` (encoded to UTF-8) or raw `bytes`.

**Returns:** Z85B-encoded bytes.

**Example:**
```python
Z85B.encode(b"Hello")         # b'NV0&yq1' (5 bytes -> 7 bytes)
Z85B.encode("Hello")          # b'NV0&yq1' (str auto UTF-8)
```

**Decoding**

```python
Z85B.decode(encoded_data: Union[str, bytes]) -> bytes
```

Decodes Z85B-encoded input.

**Arguments:**
- `encoded_data`: Z85B-encoded string or bytes.

**Returns:** Decoded bytes.

**Raises:** `ValueError` if the input has a character outside the Z85 alphabet.

**Example:**
```python
original = b"Hello, World!"
encoded = Z85B.encode(original)
decoded = Z85B.decode(encoded)
assert decoded == original
```

---

### Z85P (`Z85P`)

```python
from z85base91 import Z85P
```

**Encoding**

```python
Z85P.encode(data: Union[str, bytes]) -> bytes
```

Encodes binary data with Z85P, a Z85 variant that prepends an explicit padding byte. The first byte of the output states how many padding bytes were added (0-3), so the decoder can strip them. Z85P uses the same 85-character Z85 alphabet as Z85B. Expansion is about 1.28x.

**Arguments:**
- `data`: Input as `str` (encoded to UTF-8) or raw `bytes`.

**Returns:** Z85P-encoded bytes (first byte is the padding indicator, 0-3).

**Example:**
```python
Z85P.encode(b"A")      # b'\x03k(Z(+' (1 byte + 3 padding = 4, encodes to 5)
Z85P.encode(b"AB")     # b'\x02k%+LK' (2 bytes + 2 padding = 4)
Z85P.encode(b"ABCD")   # b'\x00k%^}b' (4 bytes, no padding)
```

**Decoding**

```python
Z85P.decode(encoded_data: Union[str, bytes]) -> bytes
```

Decodes Z85P-encoded input and strips the padding based on the first byte.

**Arguments:**
- `encoded_data`: Z85P-encoded string or bytes (must include the padding indicator byte).

**Returns:** Decoded bytes with padding removed.

**Raises:** `ValueError` if the input length is invalid or has a character outside the Z85 alphabet.

**Example:**
```python
original = b"X"
encoded = Z85P.encode(original)
decoded = Z85P.decode(encoded)
assert decoded == original
```

---

## Performance Notes

### C vs. Python

The C implementations run substantially faster than the Python fallback (about 1.5x for decoding, up to 2x for encoding). The library selects the C path automatically. The Python fallback runs only if compilation failed or the C library failed to load.

The package ships a benchmark harness (`z85base91/bench.py`) that compares all codecs against stdlib `base64`/`base32` across input sizes. It needs `click`, `tabulate`, `pybase64`, and [`hivemind-bus-client`](https://github.com/JarbasHiveMind/hivemind-bus-client) (for the pure-Python reference codecs), so install those before you run it:

```bash
pip install click tabulate pybase64 hivemind-bus-client
python -m z85base91.bench
```

Expansion ratios are deterministic: Base91 is about 1.23x, Z85B about 1.25x, and Z85P about 1.25x for aligned payloads (up to 1.28x for short, heavily padded inputs), against base64's 1.33x and base32's 1.6x. Absolute throughput depends on hardware. The C path is consistently faster than stdlib `base64` for encoding, and faster than the pure Python fallback for both directions.

---

## Architecture

The library has three layers:

1. **Public API** (`z85base91/__init__.py`): exports the `B91`, `Z85B`, and `Z85P` classes. At import time, each class tries to load its architecture-specific C library (`.so`) with `ctypes.CDLL`. If loading fails, the name is rebound to the pure-Python implementation.

2. **C implementations** (`src/*.c`, prebuilt as `z85base91/lib*-{arch}.so`):
   - `libbase91-{x86_64,aarch64,i386}.so`: B91 codec
   - `libz85b-{x86_64,aarch64,i386}.so`: Z85B codec
   - `libz85p-{x86_64,aarch64,i386}.so`: Z85P codec

3. **Pure-Python fallbacks** (`z85base91/{b91,z85b,z85p}.py`): self-contained reference implementations.

Data flow: normalize input to bytes, wrap it in a `ctypes.c_ubyte` array, call the C function, and read the output with `ctypes.string_at`.

---

## Error Handling

**Invalid input characters**

Every decoder raises `ValueError` if the input has a character outside the allowed alphabet:

```python
try:
    Z85P.decode("Hello€World")
except ValueError as e:
    print(f"Invalid character: {e}")
```

**Missing C library**

If the C library fails to load, the package logs a `WARNING` and uses the Python fallback:

```
WARNING - Z85P C library not available: Library load error. Falling back to pure Python implementation.
```

No exception is raised. Encoding and decoding still work, just slower.

---

## HiveMind Integration

The HiveMind mesh uses this library to pack binary payloads (encrypted frames, keys, metadata) for transmission over websocket channels, where text-mode framing is required. The 23% overhead of Base91, against 33% for base64, saves bandwidth at scale.

Example: a 1 MB encrypted message expands to 1.23 MB with Base91, against 1.35 MB with base64, a saving of about 12 KB per message that adds up across thousands of mesh nodes.

---

## Requirements

**Runtime:** Python 3.8+ (no external dependencies, `ctypes` is part of the standard library)

**Build:** `python3-dev`, `swig` (to compile the C extensions)

**Tests:** `pytest~=7.1`, `pytest-cov~=4.1`

---

## Testing

```bash
pip install -e . --no-deps
pip install -r test/requirements.txt
pytest test/ -q
```

The test suite covers all codecs for:
- Round-trip encode/decode (identity)
- Edge cases (empty input, single byte, odd lengths)
- Invalid input handling
- Unicode string support
- Large payloads (1000+ bytes)

---

## Related Projects

- [hivemind-websocket-client](https://github.com/JarbasHiveMind/hivemind-websocket-client): the HiveMind protocol client. Its `encodings/` module is the origin of the Base91/Z85B/Z85P codecs this library ships as a standalone, C-accelerated package.
- [hivemind-bus-client](https://github.com/JarbasHiveMind/hivemind-bus-client): provides the pure-Python reference codecs used by the benchmark harness.

## License

Apache 2.0
