Metadata-Version: 2.4
Name: datalier
Version: 0.1.0
Summary: Datalier by Concave AI - data infrastructure for AI model training
Author-email: Aniket Nerali <aniket.nerali@theconcaveai.com>
License-Expression: MIT
Project-URL: Homepage, https://theconcaveai.com
Project-URL: Documentation, https://theconcaveai.com/docs
Project-URL: Repository, https://github.com/thesineo/concave-platform
Project-URL: Issues, https://github.com/thesineo/concave-platform/issues
Keywords: datalier,concave-ai,ai,machine-learning,training-data,annotation,rlhf,rlaif,data-quality,data-infrastructure
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27.0
Dynamic: license-file

# Datalier — by Concave AI

Data infrastructure for AI model training. One SDK — raw data in, quality-assured training data out.

## Installation

```bash
pip install datalier
```

## Quick Start

```python
from datalier import DatalierClient

client = DatalierClient(
    api_key="sk_concave_...",
    base_url="https://api.theconcaveai.com",  # or http://localhost:8000 for local
)

# Upload a dataset
dataset = client.upload("training_pairs.jsonl", dataset_type="rlhf")
print(f"Uploaded: {dataset.id} — {dataset.row_count} rows")

# Transform (validate, deduplicate, detect PII)
result = client.transform(dataset.id, steps=["validate", "dedup", "pii_scan"])
print(f"PII found: {result['pii_detection']['total_pii_found']}")

# Label with RLAIF (AI handles 80-90%, humans review edge cases)
result = client.label(dataset.id, dataset_type="rlhf", min_kappa=0.70)
print(f"Auto-labeled: {result['ai_labeled']}/{result['total_tasks']}")

# Check quality metrics
quality = client.get_quality(dataset.id)
print(f"Kappa: {quality.kappa} | Gold Accuracy: {quality.gold_accuracy}")

# Approve and version
client.approve(dataset.id)
version = client.snapshot(dataset.id)
print(f"Snapshot: {version['version']} — hash: {version['snapshot_hash'][:8]}...")

# Export as DPO format for training
download_url = client.export(dataset.id, fmt="dpo", version="v1.0")

# Monitor model performance (Layer 5)
model = client.register_model("my_model_v1", trained_on_dataset_id=dataset.id, trained_on_version="v1.0")
client.submit_metrics(model["model_id"], accuracy=0.89, f1=0.85)
drift = client.get_drift(model["model_id"])
```

## Platform Layers

| Layer | Function | SDK Methods |
|-------|----------|-------------|
| **1 - Ingest** | Upload from any source | `upload()`, `list_datasets()`, `get_dataset()` |
| **2 - Prepare** | Validate, dedup, PII scan | `transform()`, `get_profile()`, `get_pii_report()`, `redact()` |
| **3 - Label** | RLAIF + human review | `label()`, `get_quality()`, `approve()` |
| **4 - Version** | Snapshots + lineage | `snapshot()`, `list_versions()`, `get_lineage()`, `rollback()`, `export()` |
| **5 - Observe** | Monitor + re-label loop | `register_model()`, `submit_metrics()`, `get_drift()`, `trigger_relabel()` |

## Using with Local Dev Server

```python
client = DatalierClient(
    api_key="your-jwt-token",
    base_url="http://localhost:8000",
)
```

## Links

- **Website**: [theconcaveai.com](https://theconcaveai.com)
- **Documentation**: [theconcaveai.com/docs](https://theconcaveai.com/docs)

## License

MIT — Concave AI 2026
