Metadata-Version: 2.4
Name: thriftlm
Version: 0.1.0
Summary: Semantic caching layer for LLM applications — stop paying for the same call twice.
Project-URL: Homepage, https://thriftlm.dev
Project-URL: Repository, https://github.com/samujure/ThriftLM
Project-URL: Issues, https://github.com/samujure/ThriftLM/issues
Author: Srivamsi Amujure
License: MIT
License-File: LICENSE
Keywords: cache,embeddings,langchain,langgraph,llm,rag,semantic
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27.0
Requires-Dist: sentence-transformers>=2.7.0
Provides-Extra: api
Requires-Dist: fastapi>=0.111.0; extra == 'api'
Requires-Dist: pydantic[email]>=2.7.0; extra == 'api'
Requires-Dist: redis>=5.0.0; extra == 'api'
Requires-Dist: supabase>=2.4.0; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'api'
Provides-Extra: dev
Requires-Dist: httpx>=0.27.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# ThriftLM

Stop paying for the same LLM call twice just because users phrased it differently.

ThriftLM is a drop-in semantic caching layer for LLM applications. It embeds each query with SBERT, checks a vector store for semantically similar past queries, and returns the cached response when similarity is above threshold — no LLM call required.

## Install

```bash
pip install thriftlm
```

## Quickstart

```python
from semantic_cache import SemanticCache

cache = SemanticCache(api_key="sc_xxx")
response = cache.get_or_call(query, llm_fn=my_langchain_chain.invoke)
```

That's it. No architecture changes — just wrap the existing LLM call.

## Self-hosting

```bash
cp .env.example .env  # fill in SUPABASE_URL, SUPABASE_KEY, REDIS_URL
docker compose up
```

Set `THRIFTLM_URL=http://localhost:8000` in your app to point at the local instance.

## Docs

[thriftlm.dev/docs](https://thriftlm.dev/docs)

## License

MIT
