Metadata-Version: 2.4
Name: bangla-punctuation
Version: 0.1.0
Summary: Restore punctuation in Bengali text and ASR transcripts
Author: Manjara Hasin Al Pitom
License-Expression: MIT
Project-URL: Homepage, https://huggingface.co/HasinManjare
Project-URL: Stable model, https://huggingface.co/HasinManjare/bangla-punctuation-v2
Project-URL: Extended model, https://huggingface.co/HasinManjare/bangla-punctuation-extended-v3
Keywords: bangla,bengali,punctuation,asr,nlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: Bengali
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.2
Requires-Dist: transformers<5,>=4.46
Dynamic: license-file

# bangla-punctuation

Python inference library for restoring punctuation in unpunctuated Bengali text
and automatic speech recognition (ASR) transcripts.

The package provides two checkpoints:

| Selection | Hugging Face model | Labels | Status |
|:---|:---|---:|:---|
| `base` | `HasinManjare/bangla-punctuation-v2` | 8 | Stable/default |
| `extended` | `HasinManjare/bangla-punctuation-extended-v3` | 10 | Experimental |

The extended model adds hyphen (`-`) and Bengali visarga (`ঃ`). Its rare-class
scores remain uneven, so it is not necessarily better than v2 for every input.

## Installation

```bash
pip install bangla-punctuation
```

For an NVIDIA GPU, install the PyTorch build appropriate for your CUDA setup as
described at <https://pytorch.org/get-started/locally/>.

## Python usage

```python
from bangla_punctuation import BanglaPunctuator

punctuator = BanglaPunctuator(model="base", device="auto")
result = punctuator.restore("আজ আকাশ খুব সুন্দর তুমি কি বাইরে যাবে")
print(result)
```

Select the experimental model:

```python
punctuator = BanglaPunctuator(model="extended")
```

Process multiple ASR results without reloading the model:

```python
raw_transcripts = [
    "আজ আকাশ খুব সুন্দর",
    "তুমি কি বাইরে যাবে",
]
restored = punctuator.restore_batch(raw_transcripts)
```

Get metadata alongside the restored text:

```python
details = punctuator.restore_with_details("আমি এখন ঢাকায় যাব তুমি যাবে")
print(details.text)
print(details.counts)
print(details.device)
```

A local model directory or another compatible Hub repository can also be used:

```python
punctuator = BanglaPunctuator(model="./bangla-bert-punctuation-v2")
```

## Command line

```bash
bangla-punctuate --model base "আজ আকাশ খুব সুন্দর তুমি কি বাইরে যাবে"
```

Or pipe ASR text through standard input:

```bash
printf '%s' 'আজ আকাশ খুব সুন্দর' | bangla-punctuate --model extended
```

## Models and training data

Both models use the ELECTRA-based
[`csebuetnlp/banglabert`](https://huggingface.co/csebuetnlp/banglabert)
encoder. They are not derived from `sagorsarker/bangla-bert-base`.

The stable v2 model was fine-tuned for eight token-level classes using the
project's original Bengali punctuation corpus. The published held-out result
was approximately `0.570` punctuation macro F1 and `0.623` overall macro F1.

The extended v3 model starts from v2 and uses:

- [`abdullahalmunem/ha-pr-bn-munem-generated`](https://huggingface.co/datasets/abdullahalmunem/ha-pr-bn-munem-generated)
  (Apache-2.0);
- [BanglaPRCorpus](https://github.com/mehedihasanbijoy/Jatikarok-and-BanglaPRCorpus)
  from the BLP/EMNLP 2023 punctuation-restoration work;
- replay examples from the original training data to reduce catastrophic
  forgetting.

Independent macro-F1 results reported after extended training were `0.447` on
Munem, `0.396` on BanglaPRCorpus, and `0.473` on the original test set. Refer to
the individual Hugging Face model cards for per-class results and limitations.

## ASR integration

Use punctuation restoration as a post-processing stage:

```python
raw_text = asr.transcribe(audio)["text"]
final_text = punctuator.restore(raw_text)
```

For long-running services, create one `BanglaPunctuator` instance at startup
and reuse it. Constructing it downloads/loads a checkpoint and is expensive.

## Important limitations

- The input parser currently keeps Bengali-block tokens and normalizes spacing;
  mixed English/Bengali text is not preserved exactly.
- Punctuation is predicted after each input word. The model does not rewrite or
  spell-correct ASR text.
- Rare punctuation such as exclamation, semicolon, ellipsis, and the extended
  labels is substantially less accurate than common labels.
- Validate on transcripts from the intended ASR engine and acoustic domain.

## Licensing

The Python wrapper code is MIT licensed. Model checkpoints and datasets retain
their own terms. The upstream `csebuetnlp/banglabert` release does not currently
state an explicit license, so the model repositories are marked `license: other`.
Confirm upstream permission before commercial redistribution or use.

