Metadata-Version: 2.4
Name: decompressed-sdk
Version: 0.4.0
Summary: Decompressed public SDK (Python)
Author: Decompressed
License-Expression: MIT
Project-URL: Homepage, https://github.com/decompressed/decompressed
Project-URL: Repository, https://github.com/decompressed/decompressed
Project-URL: Issues, https://github.com/decompressed/decompressed/issues
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.31.0
Dynamic: license-file

# decompressed-sdk

Python SDK for [Decompressed](https://decompressed.io) — chunk, embed, and version your vector data using strategies validated in the RAG Lab.

## Install

```bash
pip install decompressed-sdk
```

## Quick start

```python
from decompressed_sdk import DecompressedClient

dc = DecompressedClient(api_key="dck_your_key_here")
```

---

## RAG Lab — `client.lab`

Embed text and build production pipelines using strategies you tested in the RAG Lab. Every call applies the same chunking method, chunk size, overlap, and embedding model you validated, so what you benchmarked is exactly what runs in production.

### Embed pre-chunked texts

Use when you handle chunking yourself and just need embeddings.

```python
result = dc.lab.embed(
    texts=["Document 1", "Document 2"],
    preset_id="balanced",       # or strategy="My Strategy" / strategy_id="uuid"
)

print(result.embeddings[0][:5])     # first 5 floats
print(result.model)                  # text-embedding-3-large
print(result.dimensions)             # 3072
print(result.usage["token_count"])
```

**Built-in presets**

| ID | Name | Model | Dims | Search |
|----|------|-------|------|--------|
| `ghost` | Economy | text-embedding-3-small | 256 | vector |
| `balanced` | Balanced | text-embedding-3-large | 3072 | vector |
| `scholar` | High Accuracy | text-embedding-3-large | 3072 | hybrid + rerank |
| `hybrid` | Hybrid Search | gte-large | 1024 | hybrid + rerank |

**Supported embedding models** (use any via `chunk_and_embed` or custom strategies in the Lab)

| Model | Provider | Dims | Cost/1M tokens | Best for |
|-------|----------|------|----------------|----------|
| `text-embedding-3-small` | OpenAI | 1536 | $0.02 | Cost-effective, general use |
| `text-embedding-3-large` | OpenAI | 3072 | $0.13 | Higher accuracy, complex queries |
| `pplx-embed-v1-4b` | Perplexity | 2560 | $0.03 | Strong multilingual and reasoning |
| `pplx-embed-v1-0.6b` | Perplexity | 1024 | $0.004 | Ultra-low cost, high throughput |
| `gte-large` | Thenlper | 1024 | $0.01 | Open-source, solid retrieval quality |
| `e5-large-v2` | Intfloat | 1024 | $0.01 | Instruction-tuned retrieval |

---

### Chunk and embed a single document

Decompressed handles chunking for you using the strategy's config. Returns every chunk with its text, embedding, and metadata.

```python
with open("handbook.pdf", "r") as f:
    text = f.read()

result = dc.lab.chunk_and_embed(
    text=text,
    source="handbook.pdf",   # optional — attached to every chunk's metadata
    preset_id="balanced",
)

print(f"{len(result.chunks)} chunks · {result.dimensions}d · {result.chunking_method}")
print(f"Cost: ${result.usage['billable_cost_usd']:.6f}")

# Upsert to your vector DB
index.upsert(vectors=result.to_pinecone(id_prefix="handbook"))           # Pinecone
qdrant.upsert("docs", [PointStruct(**p) for p in result.to_qdrant()])   # Qdrant
collection.insert(list(result.to_milvus().values()))                     # Milvus
records = result.to_records()                                            # plain dicts
```

**ChunkAndEmbedResponse**

| Field | Type | Description |
|-------|------|-------------|
| `chunks` | `list[Chunk]` | Every chunk with text, embedding, and metadata |
| `model` | `str` | Embedding model used |
| `dimensions` | `int` | Vector dimensionality |
| `strategy_name` | `str` | Resolved strategy or preset name |
| `chunking_method` | `str` | Chunking method applied |
| `usage['chunk_count']` | `int` | Number of chunks produced |
| `usage['token_count']` | `int` | Tokens consumed |
| `usage['billable_cost_usd']` | `float` | Model cost + 20% platform fee |
| `usage['remaining_tokens']` | `int` | Monthly quota remaining |

**Chunk fields**

| Field | Type | Description |
|-------|------|-------------|
| `chunk_index` | `int` | Position within the document |
| `text` | `str` | Chunk text |
| `embedding` | `list[float]` | Embedding vector |
| `metadata['source']` | `str` | Source label you passed in |
| `metadata['start_char']` | `int` | Character offset start |
| `metadata['end_char']` | `int` | Character offset end |
| `metadata['model']` | `str` | Model used |
| `metadata['recommended_retrieval']` | `dict` | Suggested top_k, search_type, rerank for your vector DB |

---

### Chunk and embed many documents

Process a full corpus. Same strategy config applied to every document. All chunks collected in one flat response — upsert your entire corpus in a single call.

```python
import os

docs = [
    {"text": open(f).read(), "source": f}
    for f in os.listdir("./corpus")
    if f.endswith(".txt")
]

result = dc.lab.chunk_and_embed_many(
    documents=docs,
    preset_id="balanced",    # or strategy="My Strategy" / strategy_id="uuid"
)

print(f"{result.document_count} docs → {len(result.chunks)} chunks")
print(f"Total cost: ${result.usage['billable_cost_usd']:.4f}")
print(f"Tokens remaining: {result.usage['remaining_tokens']}")

# Upsert entire corpus in one call
index.upsert(vectors=result.to_pinecone(id_prefix="corpus"))

# Each chunk carries source + position metadata
for chunk in result.chunks:
    print(chunk.metadata["source"], chunk.metadata["doc_chunk_index"], chunk.text[:60])
```

**documents input format**

Each dict in the list:

| Key | Required | Description |
|-----|----------|-------------|
| `text` | Yes | Raw document text |
| `source` | No | Label for this document (filename, URL, ID) |

**ChunkAndEmbedManyResponse — additional fields**

| Field | Type | Description |
|-------|------|-------------|
| `document_count` | `int` | Number of documents processed |
| `chunks` | `list[Chunk]` | All chunks, flat. `chunk_index` is globally unique across the corpus. |
| `usage['chunk_count']` | `int` | Total chunks across all documents |
| `usage['token_count']` | `int` | Total tokens across all documents |
| `usage['billable_cost_usd']` | `float` | Total cost across all documents |
| `usage['remaining_tokens']` | `int` | Quota after all documents processed |
| `metadata['doc_chunk_index']` | `int` | Per-document chunk position (on each Chunk) |

Same output helpers: `to_pinecone()` · `to_qdrant()` · `to_milvus()` · `to_records()`

---

### List strategies

```python
available = dc.lab.list_strategies()

for preset in available["presets"]:
    print(f"{preset['id']}: {preset['name']} ({preset['model']})")

for s in available["saved_strategies"]:
    print(f"{s['name']} — used {s['usage_count']} times")
```

---

## Strategy reference priority

For all `lab` methods: `preset_id` → `strategy_id` → `strategy` (name).

---

## Datasets — `client.datasets`

```python
datasets = client.datasets.list()
```
