Metadata-Version: 2.5
Name: timeseries-cache
Version: 0.1.2
Summary: Persistent interval cache with caller-defined typed dimensions
License-Expression: MIT
License-File: LICENSE
Keywords: api,cache,sqlite,time-series
Classifier: Development Status :: 3 - Alpha
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Database
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: test
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# timeseries-cache

A persistent SQLite cache for time-series API data. It fetches only missing UTC
date ranges and reuses saved rows across runs. The package itself uses only the
Python standard library.

## Install

Requires Python 3.10 or newer.

```bash
uv add timeseries-cache
```

## Use the package

Declare a `Schema` with a timezone-aware `datetime` time dimension, then create a
`Cache` and call `load(selection, start, end, fetcher)`. The fetcher receives
`(start, end, refresh)` and returns rows with the declared dimensions and a
`value`. `load` returns rows, a status (`live`, `cache`, `stale`, or
`unavailable`), and any missing ranges. Pass `refresh=True` to fetch a range
again. Cache files are stored in the directory passed to `Cache`.

Here is a runnable example using generated hourly observations. Replace the
fetcher's body with your API call and return rows in the same format.

```python
from datetime import datetime, timedelta, timezone

from timeseries_cache import Cache, Schema

schema = Schema({"timestamp": datetime, "sensor": str}, "timestamp")
cache = Cache("data/cache", schema)
selection = {"sensor": "A"}


def fetch(start, end, refresh):
    timestamp = start
    while timestamp < end:
        yield {"timestamp": timestamp, **selection, "value": 12.5}
        timestamp += timedelta(hours=1)


start = datetime(2024, 1, 1, tzinfo=timezone.utc)
end = start + timedelta(days=1)

first = cache.load(selection, start, end, fetch)
second = cache.load(selection, start, end, fetch)
print(first.status, second.status, len(second.rows))
# With an empty cache: live cache 24
```

The file `data/cache/timeseries_cache.sqlite` persists across Python sessions.
Requesting a larger interval fetches only the uncovered parts. `read()` returns
stored rows without fetching; `missing()` returns uncovered intervals.

| Result status | Meaning |
| --- | --- |
| `live` | The requested fetches succeeded. |
| `cache` | No fetch was needed. |
| `stale` | Refreshing failed or was unavailable, but stored coverage exists. |
| `unavailable` | Some interval remains uncovered. |

Always inspect `result.error` and `result.missing_ranges` when a load is not
successful. A failed fetch can still return previously cached rows.

## Public API

Import `Cache`, `Schema`, and `CacheResult` from `timeseries_cache`.

| Call | Purpose |
| --- | --- |
| `Schema(dimensions, time_dimension, value_type=float, partition_dimension=None)` | Define row fields and their types. |
| `Cache(directory, schema)` | Create or reopen `directory/timeseries_cache.sqlite`. |
| `cache.load(selection, start, end, fetcher=None, *, options=None, refresh=False, max_age=None)` | Fetch missing ranges and return a `CacheResult`. |
| `cache.read(selection, start, end, *, options=None)` | Read saved rows, sorted by time. |
| `cache.missing(selection, start, end, *, options=None, max_age=None)` | List uncovered or expired intervals as `(start, end)` datetime pairs. |
| `cache.replace(selection, start, end, rows, *, options=None)` | Validate and atomically replace one interval. |
| `cache.fetched_ranges(selection, start, end, *, options=None)` | Inspect overlapping coverage records: `start`, `end`, `fetched_at`, and `outcome`. |

`CacheResult` contains `rows`, `status`, `missing_ranges`, `profile_id`,
`fetched_ranges`, and `error`. Rows are ordinary dictionaries. Modifying returned
rows does not change stored data. Coverage record dates are ISO 8601 strings;
row dates and missing-range bounds are UTC `datetime` objects.

### Schema and row types

Dimensions must include the time field and at least one other field. Their names
must be identifiers made from ASCII letters, digits, and underscores, starting
with a letter or underscore. `value` is reserved for the observation.

| Declared type | Accepted input |
| --- | --- |
| `str`, `int`, `bool` | That exact Python type; booleans do not count as integers. |
| `float` | A finite integer or float, stored as a float; booleans are rejected. |
| `datetime` | A timezone-aware datetime, normalized to UTC; dimensions only. |
| `dict`, `list` | JSON-serializable containers with string dictionary keys. |

The `value` field also accepts `None`, regardless of `value_type`. Convert API
timestamp strings to aware datetimes before returning rows. Each timestamp must
be unique within a selection; represent multiple metrics as a `metric` dimension
and load each metric separately. Nested containers follow JSON semantics, so
tuples inside them become lists on reading.

## Cache contract

- One `selection` fixes every non-time dimension. For multiple series, call
  `load` for each selection.
- A fetcher must return the **complete response for `[start, end)`**, including
  all pages/chunks, or raise. An empty iterable means the interval was checked
  successfully. Missing observations alone cannot reveal a failed download.
- Every row contains exactly the schema dimensions and `value`. Timestamps must
  be timezone-aware. Missing values use `None`; NaN and infinity are rejected.
  The start boundary is included; the end boundary is excluded.
- Include inputs that affect results in `options`, such as provider URL,
  aggregation or model version. Use separate directories for separate accounts.
  The cache cannot identify which API a callback uses. JSON dictionaries must
  have string keys at every nesting level.
- `refresh=True` reloads the whole interval. `max_age=timedelta(hours=1)` reloads
  only missing or expired intervals. Either option passes `True` to the fetcher
  when fetching, allowing it to bypass upstream caches.
- If fetching cannot proceed, covered data returns `stale`; uncovered intervals
  return `unavailable`. `missing_ranges` describes absent coverage, independent
  of age. Successful zero-row responses still count as coverage.

### Errors and freshness

Invalid schemas, selections, periods, and load arguments raise `ValueError`.
`replace` raises for invalid rows or failed writes; a failed replacement leaves
the previous interval intact. `load` captures exceptions from the fetcher, row
validation, and replacement in `result.error`, stops fetching further gaps, and
returns the available data. Earlier successful gaps remain saved. Database
failures while opening or reading still raise. `KeyboardInterrupt` and
`SystemExit` propagate.

Use `result.error is not None` to check for failure: an exception can have an
empty message. Coverage means a range was fetched successfully, not that every
possible timestamp has an observation. `missing_ranges` therefore cannot detect
missing samples inside a successful response.

The age of coverage starts after the response has been fully consumed.
`max_age` must be a nonnegative `timedelta`; zero requests a refetch of existing
coverage. Expiry affects reuse without deleting rows. A failed refresh can still
return those rows with status `stale`.

## Storage and concurrency

By default, SQLite stores observations in `cache_rows` and fetched intervals in
`cache_ranges`. Setting `Schema.partition_dimension` creates separate pairs of
tables for each selected partition value. Each record is scoped by schema and
profile identifiers derived from the schema, selection, and options. Existing
0.1.x database files can be reopened without migration.

Connections are opened and closed per operation; there is no `close()` call to
manage. SQLite uses WAL mode and a 30-second lock timeout. Each replacement
commits rows and coverage together. Readers and writers can use the same local
database, but concurrent loads may fetch the same gap more than once. For
overlapping replacements, the last committed write wins. A complete `load` is
several operations, not a single database snapshot.

Use a local filesystem for the database. Responses are validated in memory
before writing, and results are returned as a list, so split very large requests
into smaller intervals. Files persist until you remove them; expiry does not
limit disk usage.

## Development

```bash
uv sync --locked --extra test
uv run --no-sync python -m pytest -q
uv run --no-sync ruff check .
uv run --no-sync ruff format --check .
uv run --no-sync mypy
```

Tests use temporary databases and mocked fetchers. No credentials or live API
requests are needed. The CI workflow checks installed packages on Python
3.10–3.15 and on Linux, macOS, and Windows, using
[uv's GitHub Actions integration](https://docs.astral.sh/uv/guides/integration/github/).

Build the wheel and source archive with:

```bash
uv build --no-sources
```

The version is defined in `src/timeseries_cache/__init__.py`; Hatch reads it when
building. Record release changes in [CHANGELOG.md](CHANGELOG.md). The wheel
includes type information and the MIT license; tests ship in the source archive.

## License

Released under the [MIT License](https://opensource.org/license/mit).
The full license text is included in the `LICENSE` file.
