Metadata-Version: 2.4
Name: readerkit
Version: 0.1.0
Summary: Cached HTTP sessions, cache-directory resolution, and a bulk artifact cache for data readers.
Keywords: http,cache,requests,retry,rate-limiting,data
Author: ONE Campaign
Author-email: ONE Campaign <data@one.org>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Typing :: Typed
Requires-Dist: platformdirs>=4.3
Requires-Dist: filelock>=3.30
Requires-Dist: requests>=2.32
Requires-Dist: requests-cache>=1.3
Requires-Dist: requests-ratelimiter>=0.10
Requires-Dist: urllib3>=2.0
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/ONEcampaign/readerkit
Project-URL: Repository, https://github.com/ONEcampaign/readerkit
Project-URL: Issues, https://github.com/ONEcampaign/readerkit/issues
Description-Content-Type: text/markdown

# readerkit

Cached HTTP sessions, cache-directory resolution, and a bulk artifact cache for data readers.

Every cache here has an off switch, and a refresh reaches every readerkit cache in the stack
whatever the intervening APIs offer.

## Installation

readerkit reaches PyPI with its first release. Until then, install it from the repository.

```bash
uv add git+https://github.com/ONEcampaign/readerkit
```

readerkit requires Python 3.12 or later.

## Usage

### Cache-directory resolution

`resolve_cache_dir` turns a kwarg, an environment variable, or `platformdirs` into an absolute,
per-app, per-version cache root. Every other surface below takes its `cache_dir` from here.

```python
from readerkit import resolve_cache_dir

cache_dir = resolve_cache_dir(app="imf-reader", app_version="1.5.2")
# .../v1/imf-reader/1.5.2, under $IMF_READER_CACHE_DIR, $BBLOCKS_CACHE_DIR, or
# platformdirs.user_cache_dir("readerkit"), in that order.
```

`resolve_cache_dir` only resolves a path. To switch caching off, pass `cache_dir=None` to the
surfaces that consume it.

### HTTP sessions

`build_session` returns a configured `requests` session with bounded retries and jittered
backoff, an enforced default timeout, one redirect policy, optional per-host rate limiting, and
HTTP response caching when you give it a cache directory.

```python
from readerkit import build_session

session = build_session(app="imf-reader", cache_dir=cache_dir)
response = session.get("https://sdmxcentral.imf.org/...")
```

Pass `cache_dir=None` to switch the response cache off. Retries, timeout, pooling and rate
limiting stay identical.

Build a session inside each `multiprocessing` worker process. A session built before a fork
shares its socket descriptors with the child, so a response can be delivered to the wrong
process. Every session carries a fork guard that raises `SessionForkError` the first time it is
used from a process other than the one that built it.

### Bulk artifact cache

`ArtifactCache` caches large binary payloads (zips, parquet files, whatever a fetcher writes)
under per-entry sidecar metadata, with per-key locking and TTL-based staleness.

```python
from datetime import timedelta

from readerkit import ArtifactCache, bulk_fetcher

cache = ArtifactCache(cache_dir=cache_dir, namespace="weo_sdmx")
path = cache.ensure(
    "weo_2026_04",
    fetcher=bulk_fetcher("https://.../weo_2026_04.zip", session=session),
    ttl=timedelta(days=60),
    version="4",
)
```

Pass `cache_dir=None` to switch the cache off. Every call then downloads fresh into a scratch
directory owned by the cache instance, and managed cache directories are untouched. Call
`close()`, or use the cache as a context manager, to clean the scratch directory up.

```python
with ArtifactCache(cache_dir=None, namespace="weo_sdmx") as cache:
    path = cache.ensure("weo_2026_04", fetcher=bulk_fetcher(url, session=session))
```

### Cache keys

`cache_key` and `cache_key_for_call` derive a deterministic cache key from a function's fully
bound arguments, so a defaulted parameter stays in the key. oda-reader once hand-listed its key
components and left `pre_process` out, and two different preprocessing options returned
identical, wrong data.

```python
from readerkit import cache_key_for_call


def read_gdp(*, country: str, start_year: int, pre_process: bool = True):
    key = cache_key_for_call(
        read_gdp, country=country, start_year=start_year, pre_process=pre_process
    )
    ...
```

Two calls that differ only in `pre_process` produce different keys, whether the caller passed it
or relied on the default. Use the lower-level `cache_key(parts=...)` where you assemble the key
parts yourself, for example from a URL and a schema version.

### Forcing a refresh through layered caches

`refresh_scope()` forces one refresh of every readerkit-cached artifact and response touched
inside the block. It reaches any cache built on readerkit, however deep.

```python
import readerkit

with readerkit.refresh_scope():
    df = (
        pydeflate.read_weo()
    )  # refreshes pydeflate's own artifact, and imf-reader's underneath
```

`refresh_scope()` takes effect only for the duration of the `with` block.

A cached session also accepts `refresh=` and `force_refresh=` directly, for a bare call outside
any `refresh_scope()`. `refresh=True` sends a conditional request, so a 304 returns the cached
body unchanged. `force_refresh=True` issues a new request every time and overwrites whatever was
cached. `refresh_scope()` upgrades to `force_refresh`, so a caller asking for fresh data gets a
new response even from a server whose validators are unreliable.

```python
session.get(url, refresh=True)  # revalidate, may reuse the cached body
session.get(url, force_refresh=True)  # always hits the server, overwrites the cache
```

### Error handling

Every exception readerkit raises subclasses `ReaderkitError`. Catch that one name to handle
anything from the library at once, or catch a specific subclass to handle one failure mode:

- `ConfigurationError` for invalid arguments, raised eagerly at call time before any I/O.
- `CacheDirectoryError` when a cache directory cannot be resolved, created, or written to.
- `ArtifactCacheError` and its subclasses (`ArtifactWriteError`, `ArtifactCorruptError`,
  `CacheLockTimeout`, `CacheLockUnavailable`) for failures from `ArtifactCache`.
- `TransportError` and its subclasses (`SessionForkError`, `RedirectPolicyError`,
  `TruncatedDownloadError`) for failures from a `build_session` session.

Each carries an `is_retryable` class attribute, so a caller can branch on `exc.is_retryable`:

```python
from readerkit import ReaderkitError

try:
    path = cache.ensure(key, fetcher=fetcher)
except ReaderkitError as exc:
    if exc.is_retryable:
        path = cache.ensure(key, fetcher=fetcher, refresh=True)
    else:
        raise
```

HTTP status errors pass through unwrapped. Call `raise_for_status()` yourself and handle
`requests.HTTPError`, which keeps the status code intact.

## Development

readerkit uses [uv](https://docs.astral.sh/uv/) for dependency management. From a checkout:

```bash
uv sync --group dev
```

### Running tests

```bash
uv run pytest
```

### Code quality

```bash
uv run ruff check .
uv run ruff format .
uv run ty check src/readerkit
```

### Building

```bash
uv build
```

### Pre-commit hooks

Pre-commit hooks run on every commit once `pre-commit install` has been run. To run them across
the whole tree:

```bash
pre-commit run --all-files
```

## License

readerkit is licensed under the MIT License. See the LICENSE file for details.
