Metadata-Version: 2.5
Name: ohlcvault
Version: 0.2.0
Summary: Reproducible, checksummed daily OHLCV data for A-share, HK and US markets — static files, content-addressed snapshots, zero API keys.
Project-URL: Homepage, https://helpquant.com
Project-URL: Documentation, https://github.com/helpquant/ohlcvault#readme
Project-URL: Issues, https://github.com/helpquant/ohlcvault/issues
License: MIT
License-File: LICENSE
Keywords: a-share,backtest,hk-stocks,market-data,ohlcv,quant,reproducible,us-stocks
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Office/Business :: Financial :: Investment
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: duckdb>=1.0; extra == 'dev'
Requires-Dist: pandas>=2.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Provides-Extra: duckdb
Requires-Dist: duckdb>=1.0; extra == 'duckdb'
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == 'pandas'
Description-Content-Type: text/markdown

# OHLCVault

> OHLCV = Open/High/Low/Close/Volume, the universal bar format.
> Vault = immutable, checksummed snapshots.

Reproducible daily OHLCV data for **A-share, Hong Kong and US markets** — served as
static files from a CDN. No API keys, no rate limits, no per-request billing.

```python
import ohlcvault as ov

ov.connect()
df = ov.daily("600519.SH").to_pandas()      # 跨月自动拼接
ov.cross_section("cn", 20260918, limit=50)  # 当日全市场截面，按成交额排序
```

## Why this exists

Most free market-data endpoints are **APIs**: stateful, rate-limited, silently
revised, and impossible to reproduce. A backtest that ran last month can't be rerun
today with the same inputs.

OHLCVault publishes **immutable monthly shards with checksums** instead. Each read is
anchored to a snapshot id you can write down and reproduce later, on any machine.

## Install

```bash
pip install ohlcvault            # core, zero runtime dependencies
pip install "ohlcvault[pandas]"  # + DataFrame helpers
```

The core has **no third-party dependencies** — only the standard library. That's
deliberate: a data client shouldn't drag a dependency tree into your project, and it
matters even more for agent/tooling contexts.

## Usage

### Everything (the five things you actually need)

```python
import ohlcvault as ov
ov.connect()

# 1. Trading calendar
ov.calendar("cn", start=20260101)

# 2. Stock daily bars (cross-month stitching is handled for you)
ov.daily("600519.SH", start=20260101, end=20260918)

# 3. Index daily bars — a separate namespace, never mixed with stocks
ov.index_daily("000300.SH")

# 4. Symbol list — includes delisted stocks
ov.symbols("cn", type="stock")
ov.symbols("cn", type="stock", status="delisted")

# 5. Daily cross-section — sorted by turnover, no extra data files
ov.cross_section("cn", 20260918, sort_by="amount", limit=50)
```

### Batch backtests

Month shards hold **every symbol in the market** for that month. Loading 200 symbols
one-by-one would decompress the same file 200 times:

```python
bars = ov.daily_many(["600519.SH", "000001.SZ", "300750.SZ"], start=20260101)
bars["600519.SH"].to_pandas()
```

### Adjustment is a view, not a stored field

The dataset stores **unadjusted prices only**, plus the official cumulative
back-adjustment factor. Forward/backward adjusted prices are computed client-side:

```python
b = ov.daily("600519.SH")
ov.adjust(b, to="hfq")   # 后复权
ov.adjust(b, to="qfq")   # 前复权
```

This is not a limitation — it's the reason historical files never change. If
forward-adjusted prices were stored, every dividend would rewrite all of history, and
`immutable` caching would be impossible.

### Reproducibility

```python
st = ov.connect()
sid = st.snapshot                       # e.g. "6b197df3723871c5"
ov.connect(snapshot=sid)                # later, anywhere: exact same inputs
```

### Offline / self-hosted mirrors

A mirror can be an HTTP(S) URL **or a local directory**:

```python
ov.connect(mirrors=["/path/to/data"])
```

Mirrors are tried in order; whichever one succeeds is promoted to first place. Every
file is checked against the `sha256` in the snapshot manifest, and anything that
fails is **discarded and the next mirror is tried** — bad bytes are never handed to
the caller.

## Data integrity

| Guarantee | How |
|---|---|
| No silently-corrupted data | Every file verified against the snapshot's `sha256` |
| No silently-changed history | Sealed months are never rewritten |
| No unverifiable numbers | Missing adjustment factors raise, instead of returning raw prices |
| No hidden survivorship bias | Delisted stocks are kept in the universe and in the data |
| Byte-for-byte reproduction | Fixed-point integers, `gzip` with `MTIME=0`, no wall-clock timestamps |

**Delisted stocks matter.** If your backtest universe only contains companies that
are still listed today, your historical returns are systematically overstated.
`ov.symbols("cn", status="delisted")` returns them, and their daily bars are complete
over `ipo … out`.

## Coverage and known gaps

Coverage is declared explicitly in `meta/symbols/{market}.json` under `coverage`, and
`ov.connect()` prints it on startup. Current state:

| Market | Status | Gaps |
|---|---|---|
| `cn` | Daily bars + indices, 2000→present | **No Beijing Stock Exchange** (upstream source doesn't provide it) |
| `hk` | Symbol list only | No daily bars yet; **list is current listings only** |
| `us` | Symbol list only | No daily bars yet; **list is current listings only** |

`hk` / `us` symbol lists carry **survivorship bias** (the upstream source only returns
currently-listed securities) and have no IPO dates. Their daily bars are pending.

An honest data project states its gaps. A dataset that quietly omits them is worse
than one that is merely incomplete.

## Performance

The store layout (one file per market per month, all symbols inside) is optimized for
**cross-section reads** (one file = one whole trading day for the market) and for
**one-download-many-symbols** workflows. Three layers make repeated reads fast:

1. **HTTP keep-alive** — shard fetches reuse TLS connections instead of one handshake
   per request (cold-cache full-history pulls drop from minutes of handshake overhead).
2. **Shard LRU** — `connect(shard_cache=N)` keeps N parsed monthly shards in memory
   (~20–25 MB each for recent full-market months; default 24 ≈ 500 MB).
3. **Per-symbol materialization** (default on) — the first full-history read of a symbol
   writes a compact local copy under `{cache}/symbols/{snapshot_id}/`; any later read
   of the same symbol under the same snapshot is served in milliseconds and never
   touches a shard. Files are keyed by snapshot id, so a new snapshot invalidates them
   naturally. Disable with `connect(materialize=False)`.

For SQL workflows, optionally materialize a snapshot into a local DuckDB file
(`pip install "ohlcvault[duckdb]"` — an optional extra; the core stays zero-dependency):

```python
p = ov.to_duckdb(store, markets=["cn"])   # writes {cache}/duckdb/{snapshot_id}.duckdb
con = ov.connect_duckdb(p)                # a local file, not a server
con.sql("SELECT d, close/1000.0 AS close FROM bars WHERE symbol='600519.SH' ORDER BY d")
```

`to_duckdb()` is idempotent per snapshot: an existing file for the same snapshot id is
reused unless `refresh=True`. The DuckDB file is a derived local artifact — it is not
part of the checksum chain and can be deleted/rebuilt at any time.

## API

| Function | Purpose |
|---|---|
| `connect(mirrors=, cache_dir=, snapshot=, shard_cache=, materialize=)` | Build the default client |
| `symbols(market, type=, status=, board=)` | Symbol list |
| `symbol(code)` | Single symbol entry |
| `calendar(market, start=, end=)` | Trading calendar |
| `daily(code, start=, end=)` | Stock daily bars, cross-month stitching |
| `index_daily(code, start=, end=)` | Index daily bars |
| `daily_many(codes, start=, end=)` | Batch read (preferred for backtests) |
| `cross_section(market, date, sort_by=, limit=)` | Daily cross-section |
| `adjust(bars_or_df, to="qfq"\|"hfq"\|"none")` | Adjustment view |
| `to_duckdb(store, markets=, periods=)` | Materialize snapshot into a local DuckDB file (optional extra) |
| `connect_duckdb(path)` | Open a materialized DuckDB file for SQL |
| `snapshot()` | Current snapshot id |

Date parameters (`start` / `end` / `date`) accept an `int` `YYYYMMDD` (preferred)
or common string forms — `"2026-09-18"`, `"20260918"`, `"2026/09/18"`. Anything
unparseable raises `DateError` instead of failing deep inside the library.

The frozen data contract lives in [`SPEC.md`](SPEC.md) — the client and the pipeline
share nothing but this document and the files it describes. `schema/example-*.json`
are machine-generated from real data, so the examples cannot drift from the contract.

## License

MIT. See [LICENSE](LICENSE).

Data is gathered from public sources. Verify before relying on it for anything
consequential.
