Metadata-Version: 2.4
Name: mlenergy-benchmark
Version: 3.0.0
Summary: The ML.ENERGY Benchmark
Author: The ML.ENERGY Initiative
License-Expression: Apache-2.0
Requires-Python: <3.14,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp>=3.12.14
Requires-Dist: datasets>=4.0.0
Requires-Dist: mistral-common[sentencepiece]>=1.8.2
Requires-Dist: pillow>=11.3.0
Requires-Dist: pydantic>=2.11.7
Requires-Dist: requests>=2.32.4
Requires-Dist: transformers>=4.53.2
Requires-Dist: tyro>=0.9.26
Requires-Dist: zeus>=0.13.1
Requires-Dist: quart>=0.20.0
Requires-Dist: msgpack>=1.1.1
Requires-Dist: pyzmq>=27.0.0
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: protobuf>=6.33.0
Requires-Dist: opencv-python-headless>=4.11.0.86
Requires-Dist: numpy>=2.3.4
Requires-Dist: scipy>=1.16.3
Provides-Extra: diffusion
Requires-Dist: diffusers>=0.35.2; extra == "diffusion"
Requires-Dist: xfuser==0.4.5; extra == "diffusion"
Requires-Dist: insightface; extra == "diffusion"
Requires-Dist: peft==0.17.0; extra == "diffusion"
Requires-Dist: onnxruntime; extra == "diffusion"
Requires-Dist: onnxruntime-gpu; extra == "diffusion"
Requires-Dist: consisid_eva_clip; extra == "diffusion"
Requires-Dist: imageio-ffmpeg; extra == "diffusion"
Dynamic: license-file

# The ML.ENERGY Benchmark

[![Leaderboard](https://custom-icon-badges.demolab.com/badge/ml.energy-leaderboard-23d175.svg?logo=home&logoColor=white&logoSource=feather)](https://ml.energy/leaderboard)
[![Paper](https://custom-icon-badges.herokuapp.com/badge/NeurIPS'25%20D%26B-Spotlight-b31b1b.svg)](https://arxiv.org/abs/2505.06371)
[![Apache-2.0 License](https://custom-icon-badges.herokuapp.com/github/license/ml-energy/leaderboard?logo=law)](/LICENSE)

Benchmarking framework for measuring energy consumption and performance of generative AI models like Large Language Models (LLMs), Multimodal LLMs (MLLMs), and Diffusion models.

You can browse [The ML.ENERGY Leaderboard](https://ml.energy/leaderboard) for the latest benchmarking results.

- **[Overview](docs/overview.md)**: Tasks, datasets, runtime
- **[Data Preparation](docs/data-preparation.md)**: Downloading necessary datasets and processing them
- **[Running Benchmarks](docs/running-benchmarks.md)**: Job generation and manual execution
- **[Analyzing Results](docs/analyzing-results.md)**: Analyzing and understanding benchmarking results


## Citation

```bibtex
@inproceedings{mlenergy-neuripsdb25,
    title={The {ML.ENERGY Benchmark}: Toward Automated Inference Energy Measurement and Optimization}, 
    author={Jae-Won Chung and Jeff J. Ma and Ruofan Wu and Jiachen Liu and Oh Jun Kweon and Yuxuan Xia and Zhiyu Wu and Mosharaf Chowdhury},
    year={2025},
    booktitle={NeurIPS Datasets and Benchmarks},
}
```
