Metadata-Version: 2.5
Name: indic-language-utils
Version: 0.6.0b1
Summary: Provider-neutral foundations for Indian language applications
Project-URL: Documentation, https://github.com/hari31416/indic-language-utils
Project-URL: Source, https://github.com/hari31416/indic-language-utils
Author: indic-language-utils contributors
License: MIT License
        
        Copyright (c) 2026 Harikesh Kushwaha contributors
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: indic,language,speech,translation
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Typing :: Typed
Requires-Python: <3.13,>=3.11
Requires-Dist: httpx<1,>=0.28
Provides-Extra: googletrans
Requires-Dist: googletrans>=4.0.2; extra == 'googletrans'
Provides-Extra: local-tld
Requires-Dist: fasttext-langdetect>=1.1.1; extra == 'local-tld'
Provides-Extra: local-transliteration
Requires-Dist: aksharamukha>=2.0.0; extra == 'local-transliteration'
Provides-Extra: server
Requires-Dist: fastapi>=0.115.0; extra == 'server'
Requires-Dist: uvicorn[standard]>=0.34.0; extra == 'server'
Provides-Extra: stt
Requires-Dist: av>=12; extra == 'stt'
Requires-Dist: faster-whisper>=1.0.0; extra == 'stt'
Requires-Dist: speechrecognition>=3.10.0; extra == 'stt'
Provides-Extra: stt-google-free
Requires-Dist: av>=12; extra == 'stt-google-free'
Requires-Dist: speechrecognition>=3.10.0; extra == 'stt-google-free'
Provides-Extra: stt-whisper
Requires-Dist: faster-whisper>=1.0.0; extra == 'stt-whisper'
Provides-Extra: tts-edge
Requires-Dist: edge-tts>=6.1.12; extra == 'tts-edge'
Description-Content-Type: text/markdown

# indic-language-utils

Provider-neutral foundations for Indian language operations in Python.

`indic-language-utils` standardizes language detection, translation, transliteration, speech, script identification, and text processing across Indian languages. It abstracts cloud and local engines behind shared interfaces.

## Key Features

- Provider Neutrality: Code against high-level capability interfaces. Swap, configure, or chain providers without changing text processing or domain code.
- Text Translation: Translate plain text or complex Markdown documents across 22 scheduled Indian languages and English.
- Text Language Detection: Identify languages using offline FastText classification (`lid.176.ftz`) or cloud inference pipelines via Bhashini.
- Transliteration: Convert between Roman script and native Indic scripts via Bhashini or Aksharamukha.
- Speech to text: Transcribe audio with Bhashini, Sarvam, local Faster-Whisper, or keyless Google Speech.
- Text to speech: Generate audio with Bhashini, Sarvam, or Edge TTS and pass model-specific voice settings.
- Script Identification: Fast, zero-dependency Unicode script identification across 12+ Indic scripts and Latin.
- Document and Code Protection: Structural pre-processors and post-processors protect headings, bullet markers, inline code spans, URLs, and code blocks from neural translation corruption.
- Resilient Execution: Automatic multi-provider fallback routing, bounded concurrency limits per provider, and exponential backoff retries with jitter.
- High-Performance Caching: In-memory LRU and multi-process SQLite caches with write-ahead logging (WAL mode) and stampede protection.
- Canonical Normalization: Shared language registry recognizing all 22 Eighth Schedule Indian languages plus English, mapping aliases and regional codes to BCP 47.
- Localization Catalogs: Match reviewed human translations for critical UI strings before dispatching to neural engines.

## Getting started

Install the core package with `pip install indic-language-utils`. Provider selection is explicit,
and local detection requires an optional extra. The
[installation and quick start guide](docs/getting-started.md) covers a credential-free setup,
Bhashini configuration, and the first detection and translation calls.

## Supported Providers

| Provider             | Capability                              | Mode               | Prerequisites                 |
| :------------------- | :-------------------------------------- | :----------------- | :---------------------------- |
| **Aksharamukha**     | Transliteration (120+ scripts)          | Offline / Local    | `[local-transliteration]` extra |
| **Bhashini**         | Translation, Detection, Transliteration, STT, TTS | Cloud API          | API key, Endpoint, Service ID |
| **Sarvam AI**        | Translation, Detection, STT, TTS        | Cloud API          | API key (`SARVAM_API_KEY`)    |
| **FastText**         | Text Language Detection                 | Offline / Local    | `[local-tld]` extra           |
| **Faster-Whisper**   | Speech to text                          | Offline / Local    | `[stt-whisper]` extra         |
| **Google Free STT**  | Speech to text                          | Cloud (unofficial) | `[stt-google-free]` extra     |
| **Google Translate** | Translation                             | Cloud (unofficial) | `[googletrans]` extra         |
| **Microsoft Edge TTS** | Text to speech                        | Cloud (unofficial) | `[tts-edge]` extra            |

## Configuration

The library uses a tiered configuration system combining project TOML files (`.indic-language-utils.toml`), environment variables, and programmatic overrides.

Example `.indic-language-utils.toml`:

```toml
[cache]
enabled = true
backend = "sqlite"
path = ".cache/translations.sqlite3"
max_entries = 50000
ttl_seconds = 86400

[retry]
max_attempts = 3
base_delay_seconds = 0.25
max_delay_seconds = 5.0

[providers.bhashini]
endpoint = "https://dhruva-api.bhashini.gov.in/services/inference/pipeline"
translation_service_id = "default-translation-model-id"
detection_service_id = "default-tld-model-id"
transliteration_service_id = "default-transliteration-model-id"
max_concurrency = 8

[providers.sarvam]
endpoint = "https://api.sarvam.ai"
model = "sarvam-translate:v1"
stt_model_id = "saaras:v4"
tts_model_id = "bulbul:v3"
max_concurrency = 8

[routes]
translation = ["sarvam", "bhashini", "googletrans"]
text_language_detection = ["sarvam", "bhashini", "fasttext"]
transliteration = ["bhashini", "aksharamukha"]
speech_to_text = ["bhashini", "sarvam", "google_free", "faster_whisper"]
text_to_speech = ["bhashini", "sarvam", "edge_tts"]
```

Supply credentials securely through environment variables:

```bash
export BHASHINI_API_KEY="your-bhashini-api-key"
export SARVAM_API_KEY="your-sarvam-api-key"
```

## Documentation

Comprehensive guides are available in the documentation site:

- [Installation and Quick Start](docs/getting-started.md): Provider setup and first calls.
- [Provider Reference](docs/provider-reference.md): Capability matrix, setup requirements, and fallback behavior.
- [User Guide](docs/user-guide.md): Architecture overview, core capabilities, and usage styles.
- [Translation Guide](docs/translation.md): Synchronous and asynchronous translation, Markdown preservation, and catalogs.
- [Detection Guide](docs/detection.md): Local FastText and cloud Bhashini detection, script analysis, and candidate scoring.
- [Speech to text guide](docs/stt.md): Bhashini and Sarvam transcription and model IDs.
- [STT example](examples/stt/README.md): Transcribe audio with Bhashini or Sarvam.
- [Text to speech guide](docs/tts.md): Bhashini and Sarvam synthesis and voice options.
- [TTS example](examples/tts/README.md): Generate audio with Bhashini or Sarvam.
- [Configuration Reference](docs/configuration.md): Project TOML file formats, precedence rules, and environment variables.
- [Processor Pipelines](docs/processors.md): Structural processors, segment processors, and custom pipeline authoring.
- [Architecture Proposal](plans/architecture-proposal.md): Design philosophy and ecosystem review.
- [Changelog](CHANGELOG.md): Release history adhering to Keep a Changelog.

## Development

Install the locked development environment with `uv`:

```bash
uv sync --dev
uv run pre-commit install
```

Run test suite, code quality checks, and local documentation server:

```bash
uv run ruff format --check .
uv run ruff check .
uv run mypy
uv run pytest
uv build
uv run mkdocs serve
uv run mkdocs build --strict
```

## Interactive Testing Workbench

A local evaluation workbench and REST API server is included for testing translation, detection, transliteration, STT, and TTS interactively. The web assets are built from a source checkout and are not included in the Python wheel:

```bash
# Build the web interface
cd web && pnpm install && pnpm build && cd ..

# Launch the FastAPI server with static UI mounted at http://127.0.0.1:8000
uv run indic-server
```

Explore interactive API documentation at `http://127.0.0.1:8000/docs`.

## License

This project is licensed under the [MIT License](LICENSE).
