Metadata-Version: 2.4
Name: vectoralign
Version: 0.2.2
Summary: Bilingual word alignment using multilingual embeddings - a spiritual implementation of SimAlign
Author: Mayank Joshi
License-Expression: MIT
Project-URL: Homepage, https://github.com/lothnic/vectoralign
Project-URL: Repository, https://github.com/lothnic/vectoralign
Project-URL: Issues, https://github.com/lothnic/vectoralign/issues
Keywords: nlp,alignment,bilingual,embeddings,translation,dictionary
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.30.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: twine>=6.2.0
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# VectorAlign

Bilingual word alignment using multilingual embeddings — no training required!

For non-nerds: a word matching engine for any language pair using parallel data.

## Installation

```bash
pip install vectoralign
```

## Quick Start

```python
from vectoralign import align

# Your parallel sentences
english = [
    "Hello world",
    "How are you today",
    "The weather is nice"
]
hindi = [
    "नमस्ते दुनिया",
    "आज आप कैसे हैं",
    "मौसम अच्छा है"
]

# Align and build dictionary
dictionary = align(english, hindi)
```

## Features

- **No training required** — Uses pre-trained multilingual embeddings
- **Batch processing** — Batch processing with automatic CUDA detection
- **Multiple models** — Supports LaBSE, mBERT, and other HuggingFace models
- **Bidirectional alignment** — Intersection of forward and backward alignments

### Version 0.2.0 added features:
- **Memory management** — Automatic garbage collection and CUDA cache management for efficient memory usage

## Advanced Usage

```python
from vectoralign import align

# Custom model and batch size
dictionary = align(
    src_sentences,
    tgt_sentences,
    model_name="setu4993/LaBSE",  # or "bert" for mBERT
    batch_size=64,
    threshold=0.6,  # Sentence similarity threshold
    output="my_dictionary.txt"
)
```

## Supported Models

| Model | Name |
|-------|------|
| LaBSE | `setu4993/LaBSE` (default) |
| mBERT | `bert` with `mode='multilingual'` |
| Any HuggingFace model with `pooler_output` | Full model path |

## Output Format

The dictionary is saved as a TSV file:
```
word1    translation1    count
word2    translation2    count
```

## Acknowledgments

This is a spiritual implementation of [SimAlign](https://github.com/cisnlp/simalign) by the Centre for Language and Information Processing, LMU Munich.

- [Paper](https://arxiv.org/pdf/2004.08728)

## License

MIT License
