Metadata-Version: 2.4
Name: redup-topicmodel
Version: 0.1.0
Summary: gRPC inference service for multilingual BigARTM topic embeddings.
Author: REDUP
License-Expression: MIT
Project-URL: Homepage, https://github.com/redup-ai/redup.python.topicmodel/blob/master/README.md
Project-URL: Repository, https://github.com/redup-ai/redup.python.topicmodel
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
License-File: third_party/text_categorization/LICENSE
License-File: third_party/text_categorization/README.md
Requires-Dist: redup-servicekit<2.0,>=1.4.3
Requires-Dist: redup-proto-topicmodel<1.0,>=0.1.0
Requires-Dist: bigartm==0.9.2
Requires-Dist: zhon<3,>=2
Dynamic: license-file

# redup.python.topicmodel

![Docker test](https://github.com/redup-ai/redup.python.topicmodel/actions/workflows/docker-test.yml/badge.svg?branch=master)
![Python test](https://github.com/redup-ai/redup.python.topicmodel/actions/workflows/python-test.yml/badge.svg?branch=master)

gRPC inference service for a multilingual BigARTM topic model: document embeddings and topic explanations.

Contract: `redup.topicmodel.v1` (`GetDocumentsEmbedding`, `GetTopicExplanation`).
PyPI package: [`redup-topicmodel`](https://pypi.org/project/redup-topicmodel/).

## Model

Weights and tokenizers are published on Hugging Face:

**[redup-ai/topicmodel-multilingual](https://huggingface.co/redup-ai/topicmodel-multilingual)**

Expected artifact layout:

```text
<data>/
  artm/parameters.bin
  artm/p_wt.bin
  tokenizers.json.gz
  config.json
```

Download locally:

```bash
pip install huggingface_hub
python -c "from huggingface_hub import snapshot_download; print(snapshot_download('redup-ai/topicmodel-multilingual'))"
```

## Configuration

`config/config.yaml`:

```yaml
service:
  port: "[::]:9878"

TopicModel:
  artifact_root: /data          # HF artifact root (Docker default mount)
```

Override without editing the file via servicekit env substitution (`section___key`):

```bash
export TopicModel___artifact_root=/path/to/topicmodel-multilingual
```

Instead of `artifact_root`, you can set paths explicitly: `model_path` (`artm/` directory) and `bpe_path` (`tokenizers.json.gz` or a tokenizers directory) — also overridable as `TopicModel___model_path` / `TopicModel___bpe_path`.

## Run with Docker

Pull the published image and mount the downloaded artifacts at `/data` (as in the config):

```bash
docker run --rm -p 9878:9878 \
  -v /path/to/topicmodel-multilingual:/data:ro \
  redup4ai/redup.python.topicmodel:0.1.0-3.11-slim
```

The service listens for gRPC on port **9878**.

Call it from Python:

```python
import asyncio
from redup_proto_topicmodel.client import Client
from redup_proto_topicmodel.redup.topicmodel.v1.topicmodel_pb2 import (
    Document,
    DocumentPack,
)

async def main():
    client = Client("grpc://localhost:9878")
    pack = DocumentPack(documents=[
        Document(
            document_id="doc-en",
            tokens=["hello", "world"],
            modalities={"lang": "en"},
        ),
    ])
    response = await client.get_documents_embedding("example", pack)
    print(response["embeddings"])

asyncio.run(main())
```

## Run locally without Docker

```bash
python -m venv .venv && source .venv/bin/activate
pip install -r src/requirements.txt
pip install -e src

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
export TopicModel___artifact_root=/path/to/topicmodel-multilingual
PYTHONPATH=src python -m redup_topicmodel.service config/config.yaml
```

## Tests

```bash
PYTHONPATH=src python -m pytest tests -q -m "not smoke"
```

Smoke test with a real model (requires `TOPICMODEL_DATA` or `TopicModel___artifact_root`):

```bash
export TOPICMODEL_DATA=/path/to/topicmodel-multilingual
PYTHONPATH=src python -m pytest tests -q --smoke
```

## License

MIT — see `LICENSE`.

Parts of the inference logic are adapted from [text_categorization](https://github.com/machine-intelligence-laboratory/text_categorization) (Copyright (c) 2020, Machine Intelligence Team), BSD 3-Clause. See `NOTICE` and `third_party/text_categorization/`.
