Metadata-Version: 2.4
Name: optimum-intel
Version: 2.0.0
Summary: Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.
Home-page: https://www.intel.com
Author: HuggingFace Inc. Special Ops Team
Author-email: hardware@huggingface.co
License: Apache
Keywords: transformers,quantization,pruning,knowledge distillation,optimization,training
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Requires-Dist: safetensors<0.8.0
Requires-Dist: optimum~=2.2.0
Requires-Dist: transformers<5.1,>=4.45
Requires-Dist: setuptools
Requires-Dist: huggingface-hub<2.0,>=0.23.2
Requires-Dist: nncf>=2.19.0
Requires-Dist: openvino>=2026.0
Requires-Dist: openvino-tokenizers>=2026.0
Requires-Dist: requests<3.0,>=2.33
Provides-Extra: nncf
Requires-Dist: nncf>=2.19.0; extra == "nncf"
Provides-Extra: openvino
Requires-Dist: nncf>=2.19.0; extra == "openvino"
Requires-Dist: openvino>=2026.0; extra == "openvino"
Requires-Dist: openvino-tokenizers>=2026.0; extra == "openvino"
Provides-Extra: diffusers
Requires-Dist: diffusers; extra == "diffusers"
Provides-Extra: quality
Requires-Dist: black~=23.1; extra == "quality"
Requires-Dist: ruff==0.4.4; extra == "quality"
Provides-Extra: tests
Requires-Dist: accelerate; extra == "tests"
Requires-Dist: pytest<8.0.0,>=7.2.0; extra == "tests"
Requires-Dist: parameterized; extra == "tests"
Requires-Dist: Pillow; extra == "tests"
Requires-Dist: evaluate; extra == "tests"
Requires-Dist: py-cpuinfo; extra == "tests"
Requires-Dist: sacremoses; extra == "tests"
Requires-Dist: sentencepiece; extra == "tests"
Requires-Dist: torchaudio; extra == "tests"
Requires-Dist: rjieba; extra == "tests"
Requires-Dist: timm; extra == "tests"
Requires-Dist: invisible-watermark>=0.2.0; extra == "tests"
Requires-Dist: transformers_stream_generator; extra == "tests"
Requires-Dist: einops; extra == "tests"
Requires-Dist: tiktoken; extra == "tests"
Requires-Dist: sentence-transformers<5.4.0; extra == "tests"
Requires-Dist: open_clip_torch>=2.26.1; extra == "tests"
Requires-Dist: peft; extra == "tests"
Requires-Dist: datasets<4.0.0,>=1.4.0; extra == "tests"
Requires-Dist: tbb; extra == "tests"
Requires-Dist: langchain-huggingface; extra == "tests"
Requires-Dist: hf_xet; extra == "tests"
Requires-Dist: num2words; extra == "tests"
Requires-Dist: vocos; extra == "tests"
Requires-Dist: vector_quantize_pytorch; extra == "tests"
Requires-Dist: openvino-genai; extra == "tests"
Requires-Dist: av; extra == "tests"
Requires-Dist: decord; extra == "tests"
Requires-Dist: imageio; extra == "tests"
Requires-Dist: kokoro; extra == "tests"
Requires-Dist: protobuf; extra == "tests"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: summary

<p align="center">
    <img src="https://huggingface.co/datasets/optimum/documentation-images/resolve/main/intel/logo/hf_intel_logo.png" />
</p>

# Optimum Intel

🤗 [Optimum Intel](https://huggingface.co/docs/optimum-intel/en/index) is the interface between the 🤗 Transformers, Diffusers, Sentence Transformers and timm libraries and the different tools and libraries provided by [OpenVINO](https://docs.openvino.ai) to accelerate end-to-end pipelines on Intel architectures.

[OpenVINO](https://docs.openvino.ai) is an open-source toolkit that enables high performance inference capabilities for Intel CPUs, GPUs, and special DL inference accelerators ([see](https://docs.openvino.ai/2024/about-openvino/compatibility-and-support/supported-devices.html) the full list of supported devices). It is supplied with a set of tools to optimize your models with compression techniques such as quantization, pruning and knowledge distillation. Optimum Intel provides a simple interface to optimize your model, convert it to the OpenVINO Intermediate Representation (IR) format and run inference using OpenVINO Runtime.


## Installation

To install the latest release of 🤗 Optimum Intel with the corresponding required dependencies, you can use `pip` as follows:

```bash
python -m pip install -U "optimum-intel[openvino]"
```

Optimum Intel is a fast-moving project with regular additions of new model support, so you may want to install from source with the following command:

```bash
python -m pip install "optimum-intel"@git+https://github.com/huggingface/optimum-intel.git
```

**Deprecation Notice:** The `extras` for `openvino` (e.g., `pip install optimum-intel[openvino,nncf]`), `nncf`, `neural-compressor`, `ipex` are **deprecated** and will be **removed in a future release**.  


## Export:

To export your model to [OpenVINO IR](https://docs.openvino.ai/2025/documentation/openvino-ir-format.html) format, use the optimum-cli tool.
Below is an example of exporting [TinyLlama/TinyLlama_v1.1](https://huggingface.co/TinyLlama/TinyLlama_v1.1) model:

```sh
optimum-cli export openvino --model TinyLlama/TinyLlama_v1.1 ov_TinyLlama_v1_1
```

To export a [model](https://huggingface.co/docs/optimum-intel/en/openvino/models) hosted on the Hub you can use our [space](https://huggingface.co/spaces/openvino/export). After conversion, a repository will be pushed under your namespace, this repository can be either public or private.

Additional information on exporting models is available in the [documentation](https://huggingface.co/docs/optimum-intel/en/openvino/export).

## Inference:

To load an exported model and run inference using Optimum Intel, use the corresponding `OVModelForXxx` class instead of `AutoModelForXxx`:

```python
from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer, pipeline

model_id = "ov_TinyLlama_v1_1"
model = OVModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("Hey, how are you doing today?", max_new_tokens=100)
```

For more details on Optimum Intel inference, refer to the [documentation](https://huggingface.co/docs/optimum-intel/en/openvino/inference).

**Note:** Alternatively, an exported model can also be inferred using [OpenVINO GenAI](https://github.com/openvinotoolkit/openvino.genai) framework,
that provides optimized execution methods for highly performant Generative AI.

## Quantization:

Post-training static quantization can also be applied. Here is an example on how to apply static quantization on a Whisper model using the [LibriSpeech](https://huggingface.co/datasets/openslr/librispeech_asr) dataset for the calibration step.

```python
from optimum.intel import OVModelForSpeechSeq2Seq, OVQuantizationConfig

model_id = "openai/whisper-tiny"
q_config = OVQuantizationConfig(dtype="int8", dataset="librispeech", num_samples=50)
q_model = OVModelForSpeechSeq2Seq.from_pretrained(model_id, quantization_config=q_config)

# The directory where the quantized model will be saved
save_dir = "nncf_results"
q_model.save_pretrained(save_dir)
```

You can find more information in the [documentation](https://huggingface.co/docs/optimum-intel/en/openvino/optimization).

## Running the examples

Check out the [`notebooks`](https://github.com/huggingface/optimum-intel/tree/main/notebooks) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.

Do not forget to install requirements for every example:

```sh
cd <example-folder>
pip install -r requirements.txt
```
