Metadata-Version: 2.4
Name: tt-data-store
Version: 1.1.0
Summary: Local working copy of a remote market-data archive
Author-email: Apurv Salunke <salunke.apurv7@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/Tiny-Trader/data-store
Project-URL: Repository, https://github.com/Tiny-Trader/data-store
Project-URL: Issues, https://github.com/Tiny-Trader/data-store/issues
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas
Requires-Dist: pyarrow
Provides-Extra: s3
Requires-Dist: boto3; extra == "s3"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: moto[s3]; extra == "dev"
Requires-Dist: boto3; extra == "dev"
Dynamic: license-file

# tt-data-store

Work with a shared market-data archive as if it were a local folder — backed by S3.

[`tt-data-store` on PyPI](https://pypi.org/project/tt-data-store/) · [GitHub](https://github.com/Tiny-Trader/data-store)

**New here?** Start with the [usage guide](docs/usage.md).

---

## The idea

You have candle data (OHLCV) stored as Parquet files on S3. Multiple people or jobs need to read and write that data without stepping on each other.

`MarketStore` gives you a **local working copy** of the archive:

- Open a folder → catalogue syncs from S3 (instruments + file list)
- Call `get()` → downloads only the Parquet you need
- Call `write()` + `push()` → publish your changes for everyone else

You never manage S3 keys or checksums by hand. The library handles catalogue sync, downloads, and publishing.

---

## Install

```bash
pip install "tt-data-store[s3]"
```

Python 3.11+. The `[s3]` extra installs `boto3`.

You also need AWS credentials with access to your archive bucket — environment variables, `~/.aws/credentials`, or an IAM role.

---

## 30-second example

```python
from tt_data_store import MarketStore

store = MarketStore(
    "./data",
    remote="s3://my-bucket/archive",
    region="us-east-1",
)

# Read
df = store.get("NSE:INDEX:NIFTY", start="2026-01-01", end="2026-01-31")

# Write (new data you collected)
store.instruments.create(exchange="NSE", instrument_type="INDEX", symbol="NIFTY")
store.write(instrument="NSE:INDEX:NIFTY", candles=df, source="my-collector")
store.push()
```

That's the whole workflow: **open → read or write → push**.

---

## Documentation

| Guide | What you'll learn |
| ----- | ----------------- |
| [**Usage guide**](docs/usage.md) | Concepts, setup, reading, writing, sync, troubleshooting |
| [**API reference**](docs/api.md) | Method signatures, return types, errors |

---

## When to use what

| You want to… | Call |
| ------------ | ---- |
| Check if anyone else published changes | `store.fetch()` |
| Catch up with the remote catalogue | `store.pull()` |
| Load candles into a DataFrame | `store.get(instrument, start=..., end=...)` |
| Save new candles locally | `store.write(instrument=..., candles=df, source=...)` |
| Share your changes with others | `store.push()` |
| See what's out of sync | `store.status()` |

See the [usage guide](docs/usage.md) for when to call each one and what happens under the hood.

---

## Development

```bash
git clone https://github.com/Tiny-Trader/data-store.git
cd data-store
uv sync --extra dev --extra s3
uv run pytest
```

## License

Apache-2.0 — see [LICENSE](LICENSE).
