Metadata-Version: 2.4
Name: vectoralchemy
Version: 0.0.1a2
Summary: SQLAlchemy-inspired ORM for vector databases.
Author: VectorAlchemy Contributors
License: MIT
Project-URL: Homepage, https://github.com/tasmimul-huda/VectorAlchemy
Project-URL: Repository, https://github.com/tasmimul-huda/VectorAlchemy
Project-URL: Documentation, https://github.com/tasmimul-huda/VectorAlchemy#readme
Project-URL: Issues, https://github.com/tasmimul-huda/VectorAlchemy/issues
Keywords: vector database,orm,embeddings,rag,semantic search,llm,machine learning,qdrant,pinecone,weaviate,milvus,chroma,pgvector
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.0
Requires-Dist: typing-extensions>=4.8
Requires-Dist: typer>=0.12
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.9; extra == "qdrant"
Provides-Extra: pinecone
Requires-Dist: pinecone>=6.0; extra == "pinecone"
Provides-Extra: weaviate
Requires-Dist: weaviate-client>=4.0; extra == "weaviate"
Provides-Extra: milvus
Requires-Dist: pymilvus>=2.6; extra == "milvus"
Provides-Extra: chroma
Requires-Dist: chromadb>=1.0; extra == "chroma"
Provides-Extra: pgvector
Requires-Dist: asyncpg>=0.29; extra == "pgvector"
Requires-Dist: pgvector>=0.3; extra == "pgvector"
Requires-Dist: sqlalchemy[asyncio]>=2.0; extra == "pgvector"
Provides-Extra: openai
Requires-Dist: openai>=1.30; extra == "openai"
Provides-Extra: cohere
Requires-Dist: cohere>=5.0; extra == "cohere"
Provides-Extra: huggingface
Requires-Dist: sentence-transformers>=3.0; extra == "huggingface"
Provides-Extra: ollama
Requires-Dist: ollama>=0.2; extra == "ollama"
Provides-Extra: rag-example
Requires-Dist: httpx>=0.27; extra == "rag-example"
Requires-Dist: beautifulsoup4>=4.12; extra == "rag-example"
Requires-Dist: ollama>=0.2; extra == "rag-example"
Requires-Dist: sentence-transformers>=3.0; extra == "rag-example"
Provides-Extra: image-example
Requires-Dist: sentence-transformers>=3.0; extra == "image-example"
Requires-Dist: pillow>=10.0; extra == "image-example"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6; extra == "docs"
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.25; extra == "docs"
Provides-Extra: dev
Requires-Dist: numpy>=2.5.1; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=6.0; extra == "dev"
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: mypy>=1.10; extra == "dev"
Requires-Dist: pre-commit>=3.7; extra == "dev"
Requires-Dist: redis; extra == "dev"
Provides-Extra: all
Requires-Dist: vectoralchemy[chroma,milvus,pgvector,pinecone,qdrant,weaviate]; extra == "all"
Requires-Dist: vectoralchemy[cohere,huggingface,ollama,openai]; extra == "all"
Dynamic: license-file

# VectorAlchemy 🧪

> **The ORM for Vector Databases.**
> Define models. Search semantically. Switch backends without changing a line of code.

[![PyPI version](https://img.shields.io/pypi/v/vectoralchemy.svg)](https://pypi.org/project/vectoralchemy/)
[![Python](https://img.shields.io/pypi/pyversions/vectoralchemy.svg)](https://pypi.org/project/vectoralchemy/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Tests](https://github.com/vectoralchemy/vectoralchemy/actions/workflows/tests.yml/badge.svg)](https://github.com/vectoralchemy/vectoralchemy/actions)
[![Coverage](https://img.shields.io/codecov/c/github/vectoralchemy/vectoralchemy)](https://codecov.io/gh/vectoralchemy/vectoralchemy)


## ⚠️ Alpha Notice
> [!WARNING]
> **VectorAlchemy is currently in `0.0.1a1` (Alpha).**
>
> This initial PyPI release primarily establishes the project name while core development is underway. The public API is **not yet stable** and may change significantly before the first stable (`1.0.0`) release.
>
> We welcome early feedback, discussions, and contributors!


## Project Status

VectorAlchemy is an open-source project that aims to provide a **SQLAlchemy-inspired ORM for vector databases**.

Current development focuses on:

- ✅ Core architecture
- 🚧 Query builder
- 🚧 Qdrant adapter
- 🚧 Embedding providers
- 🚧 Backend abstraction

The current PyPI release (`0.0.1a2`) is an **early alpha preview** intended to reserve the package name and allow the community to follow development from the beginning.

---

## What is VectorAlchemy?

VectorAlchemy brings **SQLAlchemy-style ORM patterns** to the world of vector databases.
Stop writing boilerplate embedding code, wrestling with different client APIs, and juggling
filter syntax for every backend. Define your model once — VectorAlchemy handles the rest.

```python
from vectoralchemy import VectorModel, TextField, VectorField

class Document(VectorModel):
    class Config:
        backend    = "qdrant"
        embedder   = "openai/text-embedding-3-small"
        embed_field = "content"

    title:   str
    content: str = TextField(embed=True)
    author:  str
    year:    int

# Insert — embedding happens automatically
await Document(title="Attention Is All You Need", content="...", author="Vaswani", year=2017).save()

# Search semantically
results = await Document.search("transformer architecture", top_k=5)

# Filtered search — Django-style lookups
results = await (
    Document.objects
    .filter(year__gte=2020, author__in=["Vaswani", "LeCun"])
    .search("self-attention mechanism")
    .top_k(10)
    .min_score(0.75)
    .all()
)

# RAG context — ready to inject into your LLM prompt
context = await Document.objects.search("transformers").top_k(5).as_context()
```

---

## Features

| Feature | Description |
|---|---|
| 🏗 **Model Definition** | Pydantic-inspired class-based models with typed fields |
| 🔍 **Semantic Search** | Auto-embed queries, search by text or raw vector |
| 🔗 **Chainable Queries** | Django-style QuerySet with filter, exclude, top_k, min_score |
| 🎛 **Filter AST** | Backend-agnostic filter compilation for all 6 backends |
| 🔀 **Hybrid Search** | Dense + sparse with Reciprocal Rank Fusion (RRF) |
| 🔌 **6 Backends** | Qdrant, Pinecone, Weaviate, Milvus, Chroma, pgvector |
| 🤖 **4 Embedders** | OpenAI, Cohere, HuggingFace, Ollama |
| 🔄 **Backend Swap** | Change one line in Config — zero other changes |
| 📦 **Bulk Ops** | Auto-batched insert_many with configurable chunk size |
| 🧪 **RAG-ready** | `.as_context()` formats results for LLM prompt injection |
| ⚡ **Async-native** | Built on asyncio throughout |
| 🛠 **CLI** | Migrations, inspect, bench tools (coming in Phase 4) |

---

## Supported Backends

| Backend | Status | Install |
|---|---|---|
| [Qdrant](https://qdrant.tech) | 🚧 Phase 2 | `pip install vectoralchemy[qdrant]` |
| [Chroma](https://trychroma.com) | 🚧 Phase 3 | `pip install vectoralchemy[chroma]` |
| [pgvector](https://github.com/pgvector/pgvector) | 🚧 Phase 3 | `pip install vectoralchemy[pgvector]` |
| [Pinecone](https://pinecone.io) | 🚧 Phase 3 | `pip install vectoralchemy[pinecone]` |
| [Weaviate](https://weaviate.io) | 🚧 Phase 3 | `pip install vectoralchemy[weaviate]` |
| [Milvus](https://milvus.io) | 🚧 Phase 3 | `pip install vectoralchemy[milvus]` |

---
> **Alpha Notice**
>
> The current release is intended for experimentation and early adopters.
> APIs may change without backward compatibility until the first stable release.
## Installation

```bash
# Core only
pip install vectoralchemy

# With a specific backend
pip install vectoralchemy[qdrant]
pip install vectoralchemy[pinecone]
pip install vectoralchemy[chroma]
pip install vectoralchemy[pgvector]
pip install vectoralchemy[weaviate]
pip install vectoralchemy[milvus]

# With embedding provider
pip install vectoralchemy[qdrant,openai]
pip install vectoralchemy[qdrant,huggingface]

# Everything
pip install vectoralchemy[all]
```

---

## Quick Start

### 1. Configure

```python
from vectoralchemy import configure

configure(
    default_backend   = "qdrant",
    backend_url       = "http://localhost:6333",
    default_embedder  = "openai/text-embedding-3-small",
    embedding_api_key = "sk-...",
)
```

Or via environment variables:

```bash
VECTORALCHEMY_BACKEND=qdrant
VECTORALCHEMY_BACKEND_URL=http://localhost:6333
VECTORALCHEMY_EMBEDDER=openai/text-embedding-3-small
OPENAI_API_KEY=sk-...
```

### 2. Define a Model

```python
from vectoralchemy import VectorModel, VectorField, TextField

class BlogPost(VectorModel):
    class Config:
        backend     = "qdrant"
        collection  = "blog_posts"
        embedder    = "openai/text-embedding-3-small"
        embed_field = "body"

    title:    str
    body:     str  = TextField(embed=True)
    author:   str
    category: str
    views:    int
    embedding: list[float] = VectorField(dims=1536)
```

### 3. Insert Data

```python
post = BlogPost(
    title="Understanding Transformers",
    body="The transformer architecture revolutionized NLP...",
    author="Alice",
    category="AI",
    views=1500,
)
await post.save()

# Bulk insert
await BlogPost.insert_many([post1, post2, post3, ...])
```

### 4. Search & Filter

```python
# Simple semantic search
results = await BlogPost.search("attention mechanism", top_k=5)

# With filters
results = await (
    BlogPost.objects
    .filter(category="AI")
    .filter(views__gte=1000)
    .search("neural networks")
    .top_k(10)
    .all()
)

# Exclude
results = await BlogPost.objects.exclude(author="Anonymous").search("LLMs").all()

# Hybrid search (dense + sparse)
results = await (
    BlogPost.objects
    .hybrid_search("BERT pretraining", alpha=0.7)
    .top_k(10)
    .all()
)
```

### 5. RAG Integration

```python
question = "How do transformers handle long sequences?"

# Retrieve relevant context
context = await (
    BlogPost.objects
    .filter(category="AI")
    .search(question)
    .top_k(5)
    .as_context()
)

# Inject into your LLM prompt
prompt = f"""Answer the question based on the context below.

Context:
{context}

Question: {question}
Answer:"""
```

---

## Filter Operators

VectorAlchemy uses Django-style double-underscore lookup syntax:

```python
.filter(year=2020)              # exact match (implicit __eq)
.filter(year__eq=2020)          # explicit equal
.filter(year__ne=2020)          # not equal
.filter(year__gt=2020)          # greater than
.filter(year__gte=2020)         # greater than or equal
.filter(year__lt=2023)          # less than
.filter(year__lte=2023)         # less than or equal
.filter(author__in=["A", "B"])  # value in list
.filter(author__nin=["X"])      # value not in list
.filter(tags__contains="nlp")   # list/string contains
.filter(title__startswith="Att")# string starts with
.filter(title__endswith=".pdf") # string ends with
.filter(abstract__exists=True)  # field exists / not null
.filter(year__range=[2018,2023]) # between (inclusive)
```

Multiple filters are AND-ed by default:

```python
.filter(year__gte=2020).filter(author="Vaswani")
# → year >= 2020 AND author == "Vaswani"
```

---

## Switching Backends

The only change needed is in your model's `Config`:

```python
class Document(VectorModel):
    class Config:
        backend = "chroma"    # ← change this line only
        # ...rest stays the same
```

| Environment | Backend |
|---|---|
| Local development | `chroma` (no auth, in-memory option) |
| Staging | `qdrant` (self-hosted Docker) |
| Production | `pinecone` or `weaviate` (managed cloud) |
| Existing PostgreSQL stack | `pgvector` |

---

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    VectorAlchemy ORM                     │
├──────────────────┬──────────────────┬───────────────────┤
│   Model Layer    │   Query Layer    │  Embedding Layer  │
│  VectorModel     │  QuerySet        │  OpenAI           │
│  VectorField     │  FilterParser    │  Cohere           │
│  TextField       │  Filter AST      │  HuggingFace      │
│  ModelRegistry   │  QueryExecutor   │  Ollama           │
├──────────────────┴──────────────────┴───────────────────┤
│                   Adapter Layer                          │
│  BaseAdapter → compile_filter() → backend native format  │
├────────┬──────────┬────────┬────────┬────────┬──────────┤
│ Qdrant │ Pinecone │Weaviate│ Milvus │ Chroma │ pgvector │
└────────┴──────────┴────────┴────────┴────────┴──────────┘
```

---

## Roadmap

- [x] **Phase 1** — Core architecture (model, fields, filter AST, adapter interface)
- [ ] **Phase 2** — Qdrant adapter + OpenAI embeddings (full end-to-end MVP)
- [ ] **Phase 3** — Chroma, pgvector, Pinecone, Weaviate, Milvus adapters
- [ ] **Phase 4** — Hybrid search, migrations CLI, batch ops, embedding cache
- [ ] **Phase 5** — Docs site, PyPI release, community launch

---

## Coming Soon

- SQLAlchemy-style declarative models
- Async QuerySet API
- Automatic embedding generation
- Hybrid dense + sparse retrieval
- Backend-agnostic filter compiler
- Multi-backend support
- CLI tools
- Migrations
- Comprehensive documentation

---

## Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

```bash
git clone https://github.com/tasmimul-huda/VectorAlchemy.git
cd vectoralchemy
pip install -e ".[dev]"
pytest
```

---

## License

MIT — see [LICENSE](LICENSE).

---

<p align="center">
  Built for the LLM era. Inspired by SQLAlchemy. Powered by community.
</p>
