Metadata-Version: 2.4
Name: lingualdub
Version: 0.1.0
Summary: A composable, registry-based speech-AI framework for low-resource languages.
Author: LingualDub Authors and Contributors
Maintainer: LingualDub Core Team
License: Apache-2.0
Project-URL: Homepage, https://lingualdub.pages.dev
Project-URL: Documentation, https://lingualdub.pages.dev/docs
Project-URL: Repository, https://github.com/allaninfo-tech/lingualdub
Project-URL: Issues, https://github.com/allaninfo-tech/lingualdub/issues
Project-URL: Changelog, https://github.com/allaninfo-tech/lingualdub/blob/main/CHANGELOG.md
Keywords: speech-ai,low-resource-languages,speech-to-speech,dubbing,voice-cloning,asr,tts,machine-translation,luganda,runyankole,nlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6.0
Provides-Extra: test
Requires-Dist: pytest>=7.4; extra == "test"
Requires-Dist: pytest-cov>=4.1; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff>=0.6.0; extra == "lint"
Requires-Dist: mypy>=1.10.0; extra == "lint"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.5.0; extra == "docs"
Requires-Dist: mkdocs-material>=9.5.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.25.0; extra == "docs"
Provides-Extra: models
Requires-Dist: torch>=2.0.0; extra == "models"
Requires-Dist: transformers>=4.38.0; extra == "models"
Requires-Dist: torchaudio>=2.0.0; extra == "models"
Requires-Dist: scipy>=1.10.0; extra == "models"
Requires-Dist: soundfile>=0.12.0; extra == "models"
Provides-Extra: eval
Requires-Dist: jiwer>=3.0.0; extra == "eval"
Requires-Dist: sacrebleu>=2.4.0; extra == "eval"
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Requires-Dist: mypy>=1.10.0; extra == "dev"
Requires-Dist: pre-commit>=3.7.0; extra == "dev"
Requires-Dist: build>=1.1.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"
Requires-Dist: pyyaml>=6.0; extra == "dev"
Provides-Extra: all
Requires-Dist: torch>=2.0.0; extra == "all"
Requires-Dist: transformers>=4.38.0; extra == "all"
Requires-Dist: torchaudio>=2.0.0; extra == "all"
Requires-Dist: scipy>=1.10.0; extra == "all"
Requires-Dist: soundfile>=0.12.0; extra == "all"
Requires-Dist: jiwer>=3.0.0; extra == "all"
Requires-Dist: sacrebleu>=2.4.0; extra == "all"
Requires-Dist: pyyaml>=6.0; extra == "all"
Dynamic: license-file

<div align="center">

<a href="https://lingualdub.pages.dev">
  <img src="docs/assets/logo.png" alt="LingualDub Logo" width="340" />
</a>

# LingualDub

**A composable, registry-based speech-AI framework for low-resource languages.**

[![CI](https://github.com/allaninfo-tech/lingualdub/actions/workflows/ci.yml/badge.svg)](https://github.com/allaninfo-tech/lingualdub/actions/workflows/ci.yml)
[![Lint & Type Check](https://github.com/allaninfo-tech/lingualdub/actions/workflows/lint.yml/badge.svg)](https://github.com/allaninfo-tech/lingualdub/actions/workflows/lint.yml)
[![Python 3.10+](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Code style: ruff](https://img.shields.io/badge/code%20style-ruff-000000.svg)](https://github.com/astral-sh/ruff)
[![Checked with mypy](https://img.shields.io/badge/type_checker-mypy-blue.svg)](http://mypy-lang.org/)
[![Website](https://img.shields.io/badge/website-lingualdub.pages.dev-indigo)](https://lingualdub.pages.dev)

<br />

[Documentation](https://lingualdub.pages.dev/docs) • [Milestones & Roadmap](docs/milestones.md) • [Contributing](CONTRIBUTING.md) • [Architecture](docs/architecture.md)

</div>

---

## Overview

**LingualDub** is an open, modular speech-AI framework designed as reusable research and production infrastructure for **all low-resource language contexts**. It standardises how language metadata, speech/text data resources, model components, execution pipelines, and evaluation metrics interoperate — so that any low-resource language, model adapter, and evaluator can be wired together, adapted, and replaced without rewriting framework internals.

Initial reference validation languages:
- **Luganda (`lug`)**: Bantu (Great Lakes) — initial speech-moderate / text-moderate reference baseline.
- **Runyankole (`nyn`)**: Bantu (Great Lakes) — speech-sparse / text-sparse language-family generalisation test.
- **Expandable to any language**: Simply define a `Language` profile, attach resources, and compose your pipeline.

---

## Key Features

- **Language-Agnostic & Composable**: Five foundational abstractions (`Language`, `Resource`, `Component`, `Pipeline`, `Result`) forming a closed, provenance-tracked loop.
- **Registry & Dynamic Discovery**: Decoupled component registration supporting multiple conflict policies (`NAMESPACED`, `HIGHEST_VERSION`, `EXPLICIT`) and automatic `lingualdub.manifest.json` scanning.
- **Assembly-Time Capability Checking**: Validates stage input/output contracts (`requires` vs `provides`) at assembly time before any heavy model loads or runs.
- **Graceful Failure Handling**: Multi-tier fault tolerance (`ABORT`, `SKIP`, `DEGRADE`) with built-in degraded fallback paths and warning propagation.
- **Code-Switching Native**: `Segment.language` is authoritative per span, enabling dynamic per-segment language routing across different models in a single run.
- **Built-in Resource Management**: Local file caching, automatic downloads, and cryptographic SHA256 integrity verification.
- **Strict Provenance & Consent Tracking**: Every run automatically records pipeline structure, model versions, dataset provenance, and consent basis for ethical voice AI.

---

## Architecture

```
                 ┌─────────────────────────────┐
                 │    Language + Resources     │
                 │   (Any ISO 639 Language)    │
                 └──────────────┬──────────────┘
                                │
                                ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                     Registry Discovery                      │
  │     (Manifest Scanner • Conflict Resolution • Versioning)   │
  └─────────────────────────────┬───────────────────────────────┘
                                │
                                ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                     Pipeline Assembly                       │
  │     (Stage Composition • Assembly-Time Capability Check)    │
  └─────────────────────────────┬───────────────────────────────┘
                                │
                                ▼
  ┌─────────────────────────────────────────────────────────────┐
  │                     Pipeline Executor                       │
  │     (Per-Segment Routing • Abort / Skip / Degrade Paths)    │
  └─────────────────────────────┬───────────────────────────────┘
                                │
                                ▼
                 ┌─────────────────────────────┐
                 │   Result + Provenance Loop  │
                 │ (Segments • Metrics • Audio)│
                 └─────────────────────────────┘
```

---

## Quickstart

### Installation

```bash
# Clone and install in editable mode
git clone https://github.com/allaninfo-tech/lingualdub.git
cd lingualdub
pip install -e ".[dev]"
```

For GPU models (Whisper, Sunbird, NLLB, MMS-TTS, metrics):
```bash
pip install -e ".[all]"
```

---

## The 3-Tier Workflow

LingualDub uses **GitHub as the bridge** between local development and cloud GPU execution (e.g. Google Colab):

```text
       YOUR MACHINE                     GITHUB                     COLAB / GPU SERVER
   ┌───────────────────┐        ┌───────────────────┐        ┌───────────────────┐
   │ Fast Unit Tests   │  push  │ Versioned Code,   │ clone  │ Real Heavy Models │
   │ Deterministic     │ ─────► │ Configs & Result  │ ─────► │ Multi-Lingual /   │
   │ Local Integration │        │ Envelopes (JSON)  │        │ Fine-Tuned Models │
   └───────────────────┘        └─────────┬─────────┘        └─────────┬─────────┘
             ▲                            │                            │
             │            pull            │         push results       │
             └────────────────────────────┴────────────────────────────┘
```

1. **Level 1 (Local Unit Tests)**: Test core objects (`Language`, `Resource`, `Pipeline`, `Registry`) without GPUs or heavy downloads (`pytest`).
2. **Level 2 (Local Integration)**: Run full end-to-end pipelines locally with zero-dependency dummy adapters (`dummy_asr`, `dummy_translator`, `dummy_tts`) in under 0.1s.
3. **Level 3 (Remote Research Experiments)**: Execute real neural models (Sunbird AI, Whisper Large v3, NLLB, Meta MMS-TTS) on Colab GPUs via declarative configs and evaluate with WER, CER, and chrF.

---

### Running via CLI

```bash
# 1. Inspect registered components and languages
lingualdub registry list

# 2. Run a local pipeline test (generic to any text/language)
lingualdub experiment run configs/local_mock_pipeline.yaml \
  --sample-text "Oli otya nnyabo" \
  --output-dir experiments/local_test

# 3. Run a real GPU experiment on Colab (e.g. reference baseline)
lingualdub experiment run configs/luganda_english_baseline.yaml \
  --input-audio data/samples/sample_lug.wav \
  --output-dir experiments/luganda_dubbing/baseline_v1

# 4. Compare metric deltas across two runs
lingualdub compare \
  --baseline experiments/luganda_dubbing/baseline_v1/results.json \
  --candidate experiments/luganda_dubbing/run_v2/results.json
```

---

### Python API Example

```python
import lingualdub as ld

# 1. Define any language profile (e.g. Yoruba, Swahili, Luganda, Runyankole)
yoruba = ld.Language(
    code="yor",
    name="Yoruba",
    family="Niger-Congo (Defoid)",
    resource_profile="speech-sparse / text-moderate",
    supported_tasks=["asr", "translation", "tts"],
)

# 2. Initialize registry and load declarative pipeline config
registry = ld.Registry(conflict_policy=ld.ConflictPolicy.HIGHEST_VERSION)
scanner = ld.ManifestScanner(registry)
scanner.scan()

loader = ld.ConfigLoader(registry)
pipeline = loader.load_file("configs/luganda_english_baseline.yaml")

# 3. Execute pipeline with automatic capability validation & fault tolerance
executor = ld.PipelineExecutor(pipeline)
audio_resource = ld.Resource(
    id="sample_audio_01",
    kind=ld.ResourceKind.SPEECH,
    language="lug",
    version="1.0.0",
    path="data/samples/sample_lug.wav",
    provenance={"consent_basis": "research_evaluation"},
)

result = executor.run(audio_resource)

# 4. Inspect structured results and provenance
print(f"Status: {result.status.value.upper()}")
for seg in result.segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] ({seg.language}): {seg.text}")
print(f"Synthesised Audio Artifacts: {result.artifacts}")
```

---

## Core Abstractions

| Abstraction | Description |
|---|---|
| **`Language`** | Structured language profile capturing linguistic metadata, resource availability profile, supported tasks, and related language families. |
| **`Resource`** | Data or model asset (speech, parallel text, audio, checkpoint, eval set) with versioning, quality flags, and recorded consent basis. |
| **`Component`** | Replaceable processing unit with typed contracts (`requires` and `provides` capability tokens) and optional `degrade()` fallbacks. |
| **`Pipeline`** | Composed sequence of components with assembly-time compatibility validation and per-segment language routing. |
| **`Result`** | Structured execution envelope containing `Segment` lists, speaker info, confidence, status (`COMPLETE`, `PARTIAL`, `DEGRADED`, `FAILED`), and merged provenance. |
| **`Registry`** | Central repository enabling third-party extensions to register models, datasets, and evaluators without touching framework internals. |

---

## Research Challenge Modules

LingualDub provides modular workspaces for key open research problems in low-resource speech:

| Module | Priority | Focus Area |
|---|---|---|
| **Temporal Alignment** | Near-term | Duration modelling, speech-rate control, segment fitting, and cross-lingual synchronisation |
| **Code-Switching** | Near-term | Detection, representation, and per-segment routing for mixed-language speech |
| **Voice-Retention Evaluation** | Near-term | Repeatable speaker-similarity measurement and MOS human evaluation protocols |
| **Cross-Lingual Voice Transfer** | Open-ended | Speaker identity preservation across language boundaries with consent enforcement |
| **Audio-Visual Synchronisation** | Mid-term | Dialogue timing, scene cues, and lip-sync alignment for dubbed video |

---

## Project Roadmap

We follow a milestone-driven development model. Progress is tracked in [docs/milestones.md](docs/milestones.md):

- [x] **M0 — Foundation & Core Stabilization** (Serialization, Registry, Manifests, Resource Manager, Test Suite, CI)
- [ ] **M1 — First Real Dubbing Pipeline** (Luganda $\to$ English baseline with real model adapters)
- [ ] **M2 — Evaluation Infrastructure** (WER, CER, BLEU, chrF, run comparison utilities)
- [ ] **M3 — Code-Switching** (Per-segment language identification and routing)
- [ ] **M4 — Temporal Alignment** (Forced alignment & speech-rate adaptation)
- [ ] **M5 — Voice-Retention Evaluation** (Speaker embeddings & MOS protocol)
- [ ] **M6 — Cross-Lingual Voice Transfer** (Voice-conditioned TTS with consent checks)
- [ ] **M7 — Audio-Visual Synchronisation** (SyncNet alignment & video artifact export)
- [ ] **M8 — Generalisation Proof** (Runyankole language transfer validation)
- [ ] **M9 — Stable v0.1.0 Release** (Public PyPI package & full documentation)

---

## Repository Structure

```
lingualdub/
├── lingualdub/          # Core framework package (PEP 561 typed)
│   ├── core/            # Language, Resource, Component, Pipeline, Result, Segment
│   ├── registry/        # Registry, ManifestScanner, conflict resolution
│   ├── components/      # Task interfaces (ASR, TTS, MT, Alignment, Speaker, Eval, etc.)
│   ├── pipeline/        # PipelineExecutor and failure handling
│   ├── languages/       # Language profiles (Luganda, Runyankole, etc.)
│   └── utils/           # ResourceManager, provenance, comparison helpers
├── docs/                # Architecture, guides, milestones, manifest format
├── research/            # Research challenge module workspaces & notes
├── configs/             # Declarative pipeline YAML configurations
├── notebooks/           # Interactive demonstration & evaluation notebooks
├── tests/               # Pytest suite with >90% coverage on core modules
├── website/             # Source code for lingualdub.pages.dev (React + Tailwind)
├── .github/             # Issue templates, PR template, CI/CD workflows
├── pyproject.toml       # Build configuration, dependency groups, tool settings
├── LICENSE              # Apache 2.0 License
├── CITATION.cff         # Academic citation metadata
└── CONTRIBUTING.md      # Development and contribution guide
```

---

## Contributing

We welcome contributions! Please read our [Contributing Guide](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md) before submitting a pull request.

---

## Citation

If you use LingualDub in your research, please cite:

```bibtex
@software{lingualdub2026,
  author = {LingualDub Authors and Contributors},
  title = {LingualDub: A Composable, Registry-Based Speech-AI Framework for Low-Resource Languages},
  url = {https://github.com/allaninfo-tech/lingualdub},
  version = {0.1.0},
  year = {2026}
}
```

---

## License

This project is licensed under the **Apache License 2.0** — see the [LICENSE](LICENSE) file for details.
