Metadata-Version: 2.4
Name: llm2vec-gen
Version: 0.1.1
Summary: LLM2Vec-Gen: Generative Embeddings from Large Language Models
Project-URL: Homepage, https://mcgill-nlp.github.io/llm2vec-gen/
Project-URL: Source, https://github.com/McGill-NLP/llm2vec-gen
Author-email: Parishad Behnamghader <parishad.behnamghader@mila.quebec>, Vaibhav Adlakha <vaibhav.adlakha@mila.quebec>
License: MIT License
        
        Copyright (c) 2026 McGill NLP
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.8
Requires-Dist: datasets
Requires-Dist: evaluate
Requires-Dist: flash-attn==2.7.4.post1
Requires-Dist: huggingface-hub[cli]
Requires-Dist: hydra-colorlog>=1.2.0
Requires-Dist: hydra-core>=1.3.0
Requires-Dist: hydra-optuna-sweeper>=1.2.0
Requires-Dist: mteb==1.34.10
Requires-Dist: numpy
Requires-Dist: peft==0.18.0
Requires-Dist: ruff
Requires-Dist: torch==2.6.0
Requires-Dist: transformers==4.56.2
Requires-Dist: wandb
Provides-Extra: dev
Requires-Dist: black>=22.0; extra == 'dev'
Requires-Dist: flake8>=4.0; extra == 'dev'
Requires-Dist: ipywidgets; extra == 'dev'
Requires-Dist: jupyter; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# *LLM2VEC-GEN: Generative Embeddings from Large Language Models* 

[![arxiv](https://img.shields.io/badge/arXiv-2603.10913-b31b1b.svg)](https://arxiv.org/abs/2603.10913)
[![PyPI](https://img.shields.io/pypi/v/llm2vec-gen?label=PyPI)](https://pypi.org/project/llm2vec-gen/)
[![HF Link](https://img.shields.io/badge/HF%20Models-LLM2Vec--Gen-FFD21E.svg)](https://huggingface.co/collections/McGill-NLP/llm2vec-gen)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/McGill-NLP/llm2vec-gen/blob/main/LICENSE)
[![WandB](https://img.shields.io/badge/WandB-Logs-yellow.svg)](https://wandb.ai/siva-reddy-mila-org/llm2vec-gen)

LLM2Vec-Gen is a recipe to train interpretable, generative embeddings that encode the potential answer of an LLM to a query rather than the query itself.


## Installation
Either use Pypi or clone the repository and install in editable mode:

```bash
pip install llm2vec-gen  # or pip install -e .
```


## Usage

Load a pretrained model:
```python
import torch
from llm2vec_gen import LLM2VecGenModel

model = LLM2VecGenModel.from_pretrained("McGill-NLP/LLM2Vec-Gen-Qwen3-8B")
```

As an example, you can use the model for **retrieval** using the following code snippet:
```python
q_instruction = "Generate a passage that best answers this question: "
d_instruction = "Summarize the following passage: "

queries = [
  "where do polar bears live and what's their habitat",
  "what does disk cleanup mean on a computer"
]
q_reps = model.encode([q_instruction + q for q in queries])

documents = [
  "Polar bears live throughout the circumpolar North in the Arctic, spanning across Canada, Alaska (USA), Russia, Greenland, and Norway. Their primary habitat is sea ice over the continental shelf, which they use for hunting, mating, and traveling. They are marine mammals that rely on this environment to hunt seals.",
  "Disk Cleanup is a built-in Windows tool that frees up hard drive space by scanning for and deleting unnecessary files like temporary files, cached data, Windows updates, and items in the Recycle Bin. It improves computer performance by removing \"junk\" files, which can prevent the system from running slowly due to low storage.",
]
d_reps = model.encode([d_instruction + d for d in documents])

# Compute cosine similarity
q_reps_norm = torch.nn.functional.normalize(q_reps, p=2, dim=1)
d_reps_norm = torch.nn.functional.normalize(d_reps, p=2, dim=1)
cos_sim = torch.mm(q_reps_norm, d_reps_norm.transpose(0, 1))

print(cos_sim)
"""
tensor([[0.8750, 0.1182],
        [0.0811, 0.9336]])
"""
```

Note that in all examples, the instructions should be as if you are generating the answer to the input. 
<br>
You may find other examples to try LLM2Vec-Gen in other tasks (e.g., classification and clustering) in the paper's [GitHub repository](https://github.com/McGill-NLP/llm2vec-gen).

<br>

LLM2Vec-Gen provides interpretable embeddings. You can use the following code to **decode the content** embedded in the embeddings:

```python
_, recon_hidden_states = model.encode("what does disk cleanup mean on a computer", get_recon_hidden_states=True)
# recon_hidden_states: torch.Tensor with shape (1, compression token size, hidden_dim)

answer = model.generate(recon_hidden_states=recon_hidden_states, max_new_tokens=55)

print(answer)
"""
* **\n\n**Disk Cleanup** is a built-in utility in Windows that helps you **free up disk space** by **removing unnecessary files** from your computer. It is designed to clean up temporary files, system cache, and other files that are no longer needed.\n\n
"""
```
This code snippet will return the answer of the LLM2Vec-Gen model generated from the generative embeddings of the input (`recon_hidden_states`).


## Citation

If you use this code, models, or data, please cite the LLM2Vec-Gen paper.

```
@article{behnamghader2026llm2vecgen,
  title={LLM2Vec-Gen: Generative Embeddings from Large Language Models},
  author={BehnamGhader, Parishad and Adlakha, Vaibhav and Schmidt, Fabian David and Chapados, Nicolas and Mosbach, Marius and Reddy, Siva},
  journal={arXiv preprint: arXiv:2603.10913},
  year={2026}
  url={https://arxiv.org/abs/2603.10913}
}
```