Metadata-Version: 2.4
Name: iokit
Version: 0.5.2
Summary: Input Output Kit
Author: Vladislav A. Proskurov
Author-email: Vladislav A. Proskurov <rilshok@pm.me>
License-Expression: MIT
License-File: LICENSE
Requires-Dist: humanize>=4.9.0
Requires-Dist: packaging>=23.0
Requires-Dist: typing-extensions>=4.8.0
Requires-Dist: xxhash>=3.4.1
Requires-Dist: iokit[lint,test,ultra] ; extra == 'dev'
Requires-Dist: mypy>=1.7.1 ; extra == 'lint'
Requires-Dist: ruff>=0.6.3 ; extra == 'lint'
Requires-Dist: types-python-dateutil>=2.8.19 ; extra == 'lint'
Requires-Dist: types-pyyaml>=6.0.12 ; extra == 'lint'
Requires-Dist: types-requests>=2.31.0 ; extra == 'lint'
Requires-Dist: moto[s3,server]>=5.0.0 ; extra == 'test'
Requires-Dist: pytest>=8.2.2 ; extra == 'test'
Requires-Dist: pytest-cov==6.0.0 ; extra == 'test'
Requires-Dist: pytest-xdist>=3.6.1 ; extra == 'test'
Requires-Dist: cryptography>=41.0.7 ; extra == 'ultra'
Requires-Dist: ipython>=8.0.0 ; extra == 'ultra'
Requires-Dist: jsonlines>=4.0.0 ; extra == 'ultra'
Requires-Dist: numpy>=1.21.1 ; extra == 'ultra'
Requires-Dist: pandas>=1.5.3 ; extra == 'ultra'
Requires-Dist: pillow>=10.4.0 ; extra == 'ultra'
Requires-Dist: python-dateutil>=2.8.2 ; extra == 'ultra'
Requires-Dist: python-dotenv>=1.0.1 ; extra == 'ultra'
Requires-Dist: pyyaml>=6.0.1 ; extra == 'ultra'
Requires-Dist: requests>=2.32.3 ; extra == 'ultra'
Requires-Dist: soundfile>=0.12.1 ; extra == 'ultra'
Requires-Dist: torchaudio>=2.0.0 ; extra == 'ultra'
Requires-Dist: boto3>=1.40.7 ; extra == 'ultra'
Requires-Dist: botocore>=1.43.67 ; extra == 'ultra'
Requires-Python: >=3.10
Project-URL: Homepage, https://github.com/rilshok/iokit
Project-URL: Repository, https://github.com/rilshok/iokit.git
Project-URL: Issues, https://github.com/rilshok/iokit/issues
Provides-Extra: dev
Provides-Extra: lint
Provides-Extra: test
Provides-Extra: ultra
Description-Content-Type: text/markdown

# iokit

Python library for serialization and file operations. Unifies multiple format codecs (JSON, YAML, Tar, etc.) with a single State interface that combines data, path, and timestamp.

## Installation

Base install covers txt, bin, dat, json, gz, zip, tar:

```bash
pip install iokit
```

For all formats and web download support:

```bash
pip install iokit[ultra]
```

Missing formats will tell you what to install:

```python
from iokit import Yaml
Yaml({"key": "value"}, "file")
```

```plain-text
ModuleNotFoundError: Missing required packages: PyYAML>=6.0.1. Install with: pip install PyYAML
```

## Quick Start

Work with data as States. Each State carries the data, a path, and a timestamp. Load and save without thinking about formats.

JSON file:

```python
from iokit import Json

state = Json({"key": "value"}, path="config.json")
print(state.path)      # config.json
print(state.size)      # 16
print(state.load())    # {'key': 'value'}
```

Text file:

```python
from iokit import Txt

state = Txt("Hello, World!", "message")
state.save("/tmp/data", parents=True)
```

Load any file from disk:

```python
from iokit import file

state = file("/path/to/file.txt")
content = state.load()
```

## Common Operations

Chain transformations:

```python
from iokit import Txt

state = Txt("Secret data", "notes")
encrypted = state.encrypt(password="secret")
compressed = encrypted.gzip()
compressed.save("/tmp")
```

Load compressed:

```python
loaded = encrypted.load(password="secret").load()
```

Archives:

```python
from iokit import Tar, Txt

file1 = Txt("First", "a")
file2 = Txt("Second", "b")

archive = Tar([file1, file2], "bundle")
states = list(archive.load())
```

Find states by pattern:

```python
from iokit import filtrate, first

results = filtrate(archive.load(), "*.txt")
first_match = first(archive.load(), "a*")
```

Download:

```python
from iokit import web

state = web("https://example.com/data.json", Json)
data = state.load()
```

A `data:` URL is decoded where it stands, without reaching the network. It carries no name
for its payload, so the state is left with the bare extension of its media type:

```python
state = web("data:application/json;base64,eyJhIjogMX0=")
assert state.path == ".json"
assert state.load() == {"a": 1}
```

Checksum:

```python
state.digest("sha256").base64
state.digest("xxh128").base64url
```

Payloads of your own, filed under a format that knows nothing of them:

```python
from dataclasses import dataclass
from typing import Any

from iokit import Json


@dataclass
class Person:
    name: str
    age: int


class PersonJson(Json[Person]):
    def dump(self, data: Person) -> dict[str, Any]:
        return {"name": data.name, "age": data.age}

    def parse(self, data: dict[str, Any]) -> Person:
        return Person(data["name"], data["age"])


state = PersonJson(Person("Joe", 32), "joe")
print(state.path)    # joe.json
print(state.load())  # Person(name='Joe', age=32)
```

The file stays an ordinary `joe.json`, and `state.load()` is a `Person` both at runtime and for
the type checker. Every format takes the same pair: `Txt`, `Csv`, `Npy`, and the rest.

## Storage

Store and retrieve records by uid:

```python
from iokit.storage import LocalStorage

storage = LocalStorage("/data")
storage.push("records/data.json", data)

loaded = storage.pull("records/data.json")
storage.remove("records/data.json")

for uid in storage.index(prefix="records/"):
    print(uid)
```

State-aware storage with automatic encoding:

```python
from iokit.storage import StateStorage

storage = StateStorage(
    LocalStorage("/data"),
    compression=6,
    password="secret"
)

storage.push("file.json", {"data": 123})
result = storage.pull("file.json")
```

## Contributing

Found a bug or have an idea? Open an issue or submit a pull request.
