Metadata-Version: 2.4
Name: tracixx
Version: 0.1.0
Summary: AI pipeline observability and debugging toolkit for Databricks
Project-URL: Homepage, https://github.com/codezard/tracixx
Project-URL: Repository, https://github.com/codezard/tracixx.git
Project-URL: Issues, https://github.com/codezard/tracixx/issues
Author: Rhythem Jain
License: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Requires-Dist: deltalake>=0.17.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: pyarrow>=14.0
Requires-Dist: pydantic>=2.0
Requires-Dist: tiktoken>=0.5.0
Provides-Extra: all
Requires-Dist: langchain-core>=0.2.0; extra == 'all'
Requires-Dist: langchain>=0.2.0; extra == 'all'
Requires-Dist: mlflow>=2.10.0; extra == 'all'
Requires-Dist: pyspark>=3.5.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: mypy>=1.0; extra == 'dev'
Requires-Dist: pre-commit>=3.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.2.0; extra == 'langchain'
Requires-Dist: langchain>=0.2.0; extra == 'langchain'
Provides-Extra: mlflow
Requires-Dist: mlflow>=2.10.0; extra == 'mlflow'
Provides-Extra: spark
Requires-Dist: pyspark>=3.5.0; extra == 'spark'
Description-Content-Type: text/markdown

# Tracixx — AI Pipeline Observability and Debugging Toolkit

**Tracixx** is an open-source Python package that gives developers complete observability and debugging capabilities for AI/RAG pipelines built on Databricks.

## The Problem

Building RAG (Retrieval-Augmented Generation) pipelines is complex. When they fail, it's hard to know whether the issue came from:
- Data ingestion or chunking
- Embedding quality
- Retrieval scoring
- LLM reasoning

There's no structured way to trace issues across the entire pipeline. Debugging becomes trial-and-error.

## The Solution

Tracixx captures every stage of your pipeline:
- **Chunks** — text, size, source metadata, chunking strategy
- **Retrievals** — which chunks matched, similarity scores, embedding model
- **LLM responses** — answer, prompt, context used, token counts

All stored in Delta tables for analysis and debugging. Plus:
- **Query replay** — re-run a stored query with different configs and compare
- **Retrieval eval metrics** — Hit Rate, MRR, NDCG@k for labeled test sets
- **Pipeline analysis** — redundancy detection, orphaned chunks, grounding checks

## Quick Start

```python
from tracixx import RAGDebugger

debugger = RAGDebugger(
    catalog="main",
    schema="tracixx_debug",
    experiment_name="my-rag-v1"  # optional MLflow integration
)

# Wrap your RAG pipeline
with debugger.trace(query="What is Delta Lake?") as trace:
    # Log chunks from your document processor
    trace.log_chunks(chunks, strategy="recursive", chunk_size=512)

    # Log retrieved chunks with scores
    retrieved = vectorstore.similarity_search_with_score(query, k=5)
    trace.log_retrieval(chunks=retrieved, scores=scores, model="bge-large")

    # Log LLM response
    answer = llm.invoke(prompt)
    trace.log_response(answer=answer, prompt=prompt, tokens=usage)
```

## LangChain Integration

Drop-in callback for LangChain chains:

```python
from tracixx.integrations.langchain import RAGDebuggerCallback

chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=vectorstore.as_retriever(),
    callbacks=[RAGDebuggerCallback(debugger)]
)
```

## Installation

```bash
pip install tracixx
```

Optional dependencies:
```bash
pip install tracixx[langchain]  # LangChain integration
pip install tracixx[mlflow]     # MLflow experiment tracking
pip install tracixx[spark]      # PySpark storage backend
pip install tracixx[all]        # All optional dependencies
```

## Architecture

**Storage**: Delta tables in Unity Catalog (Databricks native)

**Tables**:
- `traces` — one row per query
- `chunks` — chunk metadata and text
- `retrievals` — retrieved chunks with scores
- `llm_responses` — generated answers and token usage

**Analysis**: Python-native diagnostics with no extra dependencies

## Roadmap

- **v0.1.0** (current): Core instrumentation + MLflow integration
- **v0.2.0**: Query replay, evaluation metrics, analysis modules
- **Phase 2**: Production monitoring, Genie Space integration, LLM-as-judge scoring

## Development

For developers contributing to Tracixx:

```bash
git clone https://github.com/codezard/tracixx.git
cd tracixx
pip install -e ".[dev]"
pytest
```

## License

MIT

## Contributing

Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## Support

- **Issues**: [GitHub Issues](https://github.com/codezard/tracixx/issues)
- **Discussions**: [GitHub Discussions](https://github.com/codezard/tracixx/discussions)
