Metadata-Version: 2.4
Name: caeli-data
Version: 0.4.0
Summary: Python client of the Caeli data repository API (data.caeli.space): load tables, publish rasters, read the catalogue
Author-email: "Caeli B.V." <tim@caeli.nl>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://data.caeli.space
Project-URL: Catalogue, https://catalog.caeli.space
Keywords: caeli,air quality,STAC,Iceberg,data repository
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.27
Requires-Dist: pyarrow>=14
Provides-Extra: aws
Requires-Dist: boto3>=1.26; extra == "aws"

# caeli-data

The Python client of the Caeli data repository API (https://data.caeli.space). One door for every producer and reader
of datasets: the inventory platform, pipelines and colleagues all use this library; nothing writes to the tables or the
buckets directly.

    pip install caeli-data            # caeli-data[aws] also brings boto3, for Repository.from_secret

    from caeli_data import Repository, TableRejected, IngestFailed

    repo = Repository()               # CAELI_API_URL (default https://data.caeli.space) and CAELI_API_KEY
    repo.whoami()                     # {"name": "tim", "role": "admin", ...}

    # a pandas DataFrame with utc_time, latitude, longitude and the value columns: the platform derives the rest
    # (pt_region, pt_tile, row_hash; the coordinates rounded to 5 decimals), splits per region and month, loads, waits
    jobs = repo.load_dataframe(df, "cdse-sentinel-5p-l2-no2", software={"my-pipeline": "1.0"}, source_ref="download 4711")
    print(jobs[0]["message"])         # "200000 rows appended in cdse_sentinel_5p_l2_no2, ... item N_EU__2026-09__v1 now ..."

    # a source with its own row identity (a station id, an orbit with scanline and pixel) names it
    jobs = repo.load_dataframe(df, "openaq-stations-no2", row_key="location_id")

    # a prepared Parquet file with the six standard columns already in it, any regions and months
    jobs = repo.load_file("prepared.parquet", "cdse-sentinel-5p-l2-no2", software={"my-pipeline": "1.0"})

A complete month delivered again is a new version of that month: `mode="replace", reason="re-import of 2026-09"`.
A service takes its key from Secrets Manager: `Repository.from_secret("caeli/data-platform/api-key-inventory-platform")`.

The rules a table must meet are in Caeli_data_repository_conventions (section 6); the library checks them before
uploading and says which rule is broken (`TableRejected`; for an extra column the message carries the schema change to
make). `IngestFailed` means the platform failed; the same data sent again is the same job, never a double load.
Python 3.9 and later.

Rasters, the read side (catalogue, versions, downloads) and the command line follow in the next versions.
