Metadata-Version: 2.5
Name: pluggy-extract
Version: 0.1.0
Summary: Headless daily extraction of Brazilian Open Finance data from the Pluggy API into a raw landing zone.
Project-URL: Homepage, https://github.com/GustavoRizzo/pluggy-extract
Project-URL: Repository, https://github.com/GustavoRizzo/pluggy-extract
Project-URL: Issues, https://github.com/GustavoRizzo/pluggy-extract/issues
Author: Gustavo Rizzo S M de Albuquerque
License-Expression: MIT
License-File: LICENSE
Keywords: brazil,etl,fintech,open-finance,pluggy
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Office/Business :: Financial
Requires-Python: >=3.11
Requires-Dist: requests>=2.31
Description-Content-Type: text/markdown

# pluggy-extract

[![PyPI](https://img.shields.io/pypi/v/pluggy-extract.svg)](https://pypi.org/project/pluggy-extract/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Headless daily extraction of Brazilian Open Finance data from the [Pluggy](https://pluggy.ai)
API into a raw landing zone.

It reads accounts, transactions and investments for the items you already connected, and
writes them **verbatim** as partitioned JSONL. Normalizing, deduplicating and enriching are
deliberately left to whatever consumes the landing zone.

Pure Python, no framework: it runs the same from cron, Airflow, mage.ai or a notebook.

## Install

```bash
uv add pluggy-extract     # or: pip install pluggy-extract
```

## Use

```bash
cp profiles.example.toml profiles.toml     # fill in your item ids
export GRIZZO_PLUGGY_CLIENT_ID=...
export GRIZZO_PLUGGY_CLIENT_SECRET=...

pluggy-extract run --profile grizzo --date 2026-08-18 --out ./raw
```

```
run 20260818T060000Z-a1b2c3  profile=grizzo  out=./raw

   nubank               ok  ok=[item,accounts,transactions,investments]
!! itau                 needs_user  ok=[item]  -- update your password

parcial: 1/2 itens sem ressalvas (exit 2)
```

Exit codes: `0` all clean, `2` partial, `1` nothing extracted. They are distinct so an
orchestrator does not mark a day where 9 of 10 items succeeded as a total failure.

### As a library

Fetching and writing are separate layers, so either can be used alone:

```python
from pluggy_extract import PluggyClient, extract

client = PluggyClient(client_id, client_secret)
accounts = extract.extract_accounts(client, item_id)      # plain dicts, no disk I/O
```

## Output layout

```
raw/profile=grizzo/item=<uuid>/dataset=transactions/dt=2026-08-18/<run_id>.jsonl
```

Every line is one record wrapped in a provenance envelope:

```json
{
  "run_id": "20260818T060000Z-a1b2c3",
  "extracted_at": "2026-08-18T06:00:12.482Z",
  "extractor_version": "0.1.0",
  "profile": "grizzo",
  "item_id": "...",
  "item_status": "UPDATED",
  "endpoint": "/v2/transactions",
  "params": {"accountId": "...", "dateFrom": "2026-07-19"},
  "payload": { "...the Pluggy object, untouched..." }
}
```

`item_status` matters more than it looks: when a monthly Open Finance quota runs out, Pluggy
answers `200` with **stale** data and flags the item `PARTIAL_SUCCESS`. Without the status in
the envelope, nothing downstream can tell the difference.

## Things worth knowing

- **Pluggy already syncs once a day.** This tool only reads; it never triggers an update.
  Schedule it *after* the item's `nextAutoSyncAt`, or you re-read yesterday's data.
- **Quotas are monthly and per CPF+institution**, from Open Finance regulation, not Pluggy.
  `identity` allows only **4/month**, so it is excluded from the default datasets.
- **Transactions are re-read over a rolling 30-day window** on every run. They mutate after
  creation (pending settles, descriptions get rewritten); the redundancy is how corrections
  are captured. Deduplicate by transaction id downstream.
- **Credit cards are accounts** (`type=CREDIT`), not a separate endpoint.
- **`status=UPDATED` does not mean fresh** — it means the *last* sync worked, which may have
  been weeks ago. Watch `lastUpdatedAt` and `autoSyncDisabledAt`.

## Development

```bash
uv sync
uv run pytest
```

Tests are fully mocked with `responses`, and a network guard fails any test that tries to
open a real socket — this library talks to a financial API with real credentials, so a
forgotten mock must never leak into production.

## License

[MIT](LICENSE)
