Metadata-Version: 2.4
Name: thinctx
Version: 0.1.0
Summary: ThinCtx: inference runtime and OpenAI-compatible server for the ThinCtx models (small, constant GPU memory for the context).
Author: Jan Rosecký (thinletter)
License: PolyForm-Small-Business-1.0.0
Project-URL: Homepage, https://thinletter.io
Project-URL: Models, https://huggingface.co/thinletter
Keywords: llm,inference,long-context,qwen,openai-compatible
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: POSIX :: Linux
Classifier: Intended Audience :: Developers
Requires-Python: <3.13,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE-PolyForm-Small-Business-1.0.0.txt
Requires-Dist: torch<2.12,>=2.11
Requires-Dist: transformers<6,>=5.18
Requires-Dist: fla-core==0.5.2
Requires-Dist: safetensors>=0.4
Requires-Dist: huggingface_hub>=0.25
Requires-Dist: numpy>=1.26
Provides-Extra: conv1d
Requires-Dist: causal_conv1d>=1.4; extra == "conv1d"
Dynamic: license-file

# thinctx

Inference runtime and OpenAI-compatible server for the **ThinCtx models** — compact language models
whose context memory on the GPU stays small and constant while the full context is held in system memory.

```bash
pip install thinctx

thinctx serve thinletter/ThinCtx-Qwen3.5-2B --port 8000 --auth user:pass   # OpenAI-compatible endpoint
thinctx chat  thinletter/ThinCtx-Qwen3.5-2B                                # terminal REPL
```

```python
import thinctx
engine = thinctx.load("thinletter/ThinCtx-Qwen3.5-2B")   # a hub repo id or a local released folder
```

The server implements the OpenAI chat-completions API (`/v1/chat/completions`, streaming and
non-streaming, tools, `enable_thinking`); point any OpenAI client at `http://host:8000/v1`.

## Requirements

Linux x86_64, CPython 3.12, an NVIDIA GPU with CUDA 12.8, PyTorch 2.11. Installs `transformers`,
`fla-core` and `safetensors`; `causal_conv1d` is optional (a pure-torch fallback is built in).

## License

Released under the [PolyForm Small Business License 1.0.0](https://polyformproject.org/licenses/small-business/1.0.0)
(`LICENSE`): free for companies with fewer than 100 people and under USD 1 million revenue (2019 dollars, adjusted
for inflation). `LICENSE` adds free use, for anyone, for personal non-commercial purposes, for non-commercial research
and education, and for evaluation and testing (non-production). Production use by larger organisations needs a
commercial licence: info@thinletter.io. The models are derivatives of Qwen3.5 (Apache-2.0); each model repo ships
Qwen's license and a `NOTICE`.
