Metadata-Version: 2.2
Name: cf-datahive
Version: 0.1.11
Summary: Canonical result and measurement data storage APIs for Cogniflow
Requires-Python: >=3.11
Requires-Dist: pyarrow>=12
Requires-Dist: cf-service-contracts>=0.1.0
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == "pandas"
Provides-Extra: test
Requires-Dist: pytest<9.0,>=8.0; extra == "test"
Requires-Dist: pandas>=2.0; extra == "test"
Description-Content-Type: text/markdown

# cf_datahive

`cf_datahive` owns the Data Hive write provider for `CfDataHiveSinkContractV1`, publishes the read-side `datahive_runs` service contract for first-party consumers, and exposes a local Python read/inspect API for manual inspection, owner-side tooling, and package-local validation.

For first-party package integration the boundary is strict:

- cross-package writes go only through `cf_package_contracts` and injected contract slots such as `CfExecutionContext.datahive_sink`
- cross-package reads go only through `cf_service_contracts` and the `datahive_runs` provider key
- first-party packages must not import `cf_datahive`
- first-party packages must not reach into `cf_datahive/native/*` or consume `cf_datahive_cpp_*` helper APIs

## Boundary

- `sandcastle/cf_datahive/src/cf_datahive/cpp` is the owner-side native gatekeeper for writes under `workspace/data_hive`
- `sandcastle/cf_datahive/src/cf_datahive/service_contract_provider.py` publishes the canonical read-side `datahive_runs` contract for first-party consumers
- `sandcastle/cf_datahive/src/cf_datahive` provides owner-side read/inspect APIs such as `DataHiveClient`
- `CfDataHiveSinkContractV1` in `cf_package_contracts` is the only supported cross-package sink interface
- `DataHiveRunsContractV1` in `cf_service_contracts` is the only supported cross-package read interface

First-party packages that need read-side runtime integration should resolve `ProviderKey.DATAHIVE_RUNS` against the canonical workspace root instead of importing `cf_datahive` directly.

## Usage

First-party read-side integration:

```python
from cf_service_contracts import ProviderKey, require_provider

datahive_runs = require_provider(ProviderKey.DATAHIVE_RUNS)
evidence = datahive_runs.find_latest_run(workspace_root, pipeline_id)
if evidence is not None:
  print(evidence["run_id"], evidence["manifest_status"])
```

Owner-side inspection workflow:

```python
from pathlib import Path

from cf_datahive import DataHiveClient

workspace_root = Path("workspace")
client = DataHiveClient(str(workspace_root))

runs = client.list_runs("opcua_fifo_avg")
if runs:
    latest = runs[0].run_id
    manifest = client.load_manifest("opcua_fifo_avg", latest)
    table = client.read_table("opcua_fifo_avg", latest, "measurements")
    print(manifest.status, table.num_rows)
```

`DataHiveClient`, `RunHandle`, and `StorageModePolicy` remain public for owner-side workflows, but they are not a first-party integration surface.

## Canonical Layout

```text
workspace/
  data_hive/
    <pipeline_id>/
      <yyyy>/<mm>/<dd>/<session_id>/
        runs/<run_id>/manifest.json
        runs/<run_id>/artifacts/<artifact_name>
        tables/<table_name>/run_id=<run_id>/part-0000.parquet
        tables/<table_name>/_session_compacted/part-0000.parquet
        session_manifest.json
      latest.txt
      latest_session.txt
```

- `latest.txt` stores the latest committed `run_id`
- `latest_session.txt` stores the latest committed `session_id`
- `manifest.json` is the run SOT for tables, artifacts, hashes, and session linkage
- `session_manifest.json` is the per-session SOT for storage-mode locks
- flat run layouts and direct `data_hive/` roots outside the canonical workspace root are not supported

## Local Build Notes

`cf_datahive` now consumes the workspace-store runtime contract instead of linking the storage backend directly.
Editable builds still use the shared native dependency cache under `~/.cogniflow/cache/native/` and the standard
`cf_setup` MSVC/CMake environment preparation.

Build caches are expected under `~/.cogniflow/cache/native/scikit-build/cf-datahive`.

## Guardrails

Run:

```bash
python tools/check_datahive_guardrails.py
```

The guardrail script hard-fails when first-party code outside `sandcastle/cf_datahive`:

- imports `cf_datahive`
- references removed `cf_datahive_cpp_*` helper APIs
- reaches into the removed native consumer packaging surface

## Testing

```bash
pip install -e "sandcastle/cf_datahive[test]"
pytest -q sandcastle/cf_datahive/tests
```

Published distribution name:

```bash
pip install cf-datahive
```

## Publishing

`cf_datahive` still publishes the provider binary plus provider descriptor used by `cf_setup` and `cf_pipeline_engine`.

- Workflow: `.github/workflows/cf_datahive_windows_publish.yml`
- Package directory: `sandcastle/cf_datahive`
- PyPI tag: `cf-datahive-v<version>`
- TestPyPI tag: `cf-datahive-v<version>-test`
