Metadata-Version: 2.4
Name: commonmeta-rs
Version: 0.9.50
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing :: Markup
License-File: LICENSE-MIT
Summary: PyO3 bindings for commonmeta-rs's list/batch functions (VRAIX dump fetch+cache, Parquet I/O, batched archives)
Keywords: metadata,crossref,datacite,orcid,ror,commonmeta
Author-email: Front Matter <info@front-matter.de>
License-Expression: MIT
Requires-Python: >=3.14
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://commonmeta.org/
Project-URL: Repository, https://codeberg.org/front-matter/commonmeta-rs

# commonmeta-rs (Python bindings)

PyO3 bindings exposing commonmeta-rs's `list`-command batch functions to
Python: VRAIX daily-dump fetch with local caching, lossless Parquet
read/write, and batched archive rendering. Records cross the FFI boundary as
plain JSON-shaped dicts — the same shape `commonmeta-py`'s own readers and
writers already produce and consume.

## Install

```sh
uv pip install commonmeta-rs
```

Requires CPython 3.14 or newer. The wheels are built against the stable ABI
(`abi3-py314`), so one wheel per platform also covers later 3.x releases.
`commonmeta-py` itself keeps a 3.9 floor; only this backend needs 3.14.

## Development

The interpreter must be 3.14+, matching `requires-python`; an older venv fails
to build rather than falling back.

```sh
uv venv --python 3.14 .venv
uv pip install --python .venv/bin/python maturin pytest
source .venv/bin/activate
maturin develop
pytest tests/
```

## Release

Releases are published by hand. PyPI trusted publishing only issues OIDC
identities to GitHub, GitLab, Google Cloud and ActiveState, and this repo lives
on Codeberg, so CI cannot authenticate to PyPI — the workflow builds and tests
wheels only.

The version comes from `[workspace.package]` in the root `Cargo.toml` and is
shared with the `commonmeta` crate, so a wheel names the release it wraps. Bump
it there, not here.

Build one wheel per platform plus the sdist, then publish. The macOS wheel
builds natively; the Linux `manylinux` wheel builds in the official maturin
container (which bundles the manylinux toolchain), so it needs no cross linker
on the host — just a running Docker. All commands are run from `python/`.

```sh
rm -rf dist

# macOS (native — arm64 on Apple silicon)
maturin build --release --out dist

# Linux (manylinux2014 x86_64) via Docker. Mount the repo ROOT, not python/:
# the binding depends on the parent crate through `commonmeta = { path = ".." }`.
# --platform linux/amd64 runs the x86_64 image under emulation on Apple silicon
# (slow but reliable). Pinning --target keeps the Linux artifacts out of the
# host's target/release, so they don't collide with the native build.
docker run --rm --platform linux/amd64 \
  -v "$(git rev-parse --show-toplevel)":/io -w /io/python \
  ghcr.io/pyo3/maturin:latest \
  build --release --out dist --target x86_64-unknown-linux-gnu

# Source distribution (lets other platforms build from source as a fallback)
maturin sdist --out dist

uv publish --token "$PYPI_TOKEN" dist/*
```

Publishing from one machine uploads only the wheels present in `dist/`; any
platform without a wheel falls back to building from the sdist. To ship
binaries for more platforms (Windows, Intel macOS, Linux aarch64), add the
matching `maturin build --target …` invocations or collect the `wheels-*`
artifacts from the CI workflow into `dist/` first.

## API

- `fetch_vraix(source, date, input_path=None, limit=None, offset=0, cache_ttl_days=30) -> list[dict]`
  Fetch records from a VRAIX daily dump (`source` is `"crossref"` or
  `"datacite"`). With `input_path`, reads a local SQLite file directly (no
  network); otherwise downloads `{source}-{date}.sqlite3.zst` from
  metadata.vraix.org, caching it locally for `cache_ttl_days`.
- `write_parquet(records: list[dict]) -> bytes`
  Write records as a single lossless Parquet file.
- `read_parquet(data: bytes) -> list[dict]`
  Read records back from Parquet bytes written by `write_parquet`.
- `write_archive(records: list[dict], to: str, base_name: str, batch_size=100_000) -> list[tuple[str, bytes]]`
  Render records to `to` format, split into batches of at most `batch_size`
  records, returned as `(entry_name, bytes)` pairs ready to pack into a
  zip/tar archive.
- `convert(from_: str, to: str, input: str) -> bytes`
  Convert a single record between formats.

