Metadata-Version: 2.4
Name: cloudsync-mlbridge
Version: 0.1.0
Summary: Google Cloud Datastore to BigQuery sync toolkit with Pub/Sub events, ML scoring, and workflow-ready orchestration.
Author: Raghava Chellu
License: MIT
Keywords: google-cloud,datastore,bigquery,pubsub,ml,sync,cdc,workflows,agentic-ai
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: typer>=0.12.0
Requires-Dist: rich>=13.7.0
Requires-Dist: pydantic>=2.6.0
Requires-Dist: google-cloud-bigquery>=3.17.0
Requires-Dist: google-cloud-datastore>=2.19.0
Requires-Dist: google-cloud-pubsub>=2.21.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Requires-Dist: build>=1.2.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"
Dynamic: license-file

# CloudSync MLBridge

**Google Cloud Datastore to BigQuery sync toolkit with Pub/Sub events, ML scoring, and workflow-ready orchestration.**

`cloudsync-mlbridge` helps teams move operational data from Google Cloud Datastore into BigQuery while keeping analytics tables current through event-driven sync patterns.

It is designed for modern Google Cloud architectures using Datastore, BigQuery, Pub/Sub, Google Workflows, Cloud Run, and ML-based decision systems.

## Use Cases

- Datastore to BigQuery synchronization
- Pub/Sub event-driven change propagation
- BigQuery analytics table loading
- ML-assisted record scoring before analytics ingestion
- Workflow-ready sync payload generation
- Operational dashboards and reporting
- Agentic AI data movement orchestration

## Install

```bash
pip install cloudsync-mlbridge
```

## Python Usage

```python
from cloudsync_mlbridge import SyncRecord, score_record_freshness, build_bigquery_row

record = SyncRecord(
    entity_key="customer-1001",
    kind="CustomerProfile",
    operation="UPSERT",
    data={"status": "active", "region": "US"}
)

print(score_record_freshness(record))
print(build_bigquery_row(record))
```

## CLI Usage

```bash
cloudsync-mlbridge score --entity-key customer-1001 --kind CustomerProfile --operation UPSERT
```

## Architecture Pattern

Recommended enterprise pattern:

1. Application writes to Datastore.
2. Application emits a Pub/Sub event.
3. Cloud Run, Dataflow, or Workflows receives the event.
4. `cloudsync-mlbridge` normalizes the record.
5. Optional ML scoring is applied.
6. Record is written to BigQuery raw/current tables.
7. Scheduled reconciliation compares Datastore export with BigQuery.

## Build and Publish

```bash
python -m pip install --upgrade build twine
python -m build
twine check dist/*
twine upload dist/*
```

## License

MIT

Author: Raghava Chellu
