Metadata-Version: 2.3
Name: py-gen-ml
Version: 0.2.0
Summary: A Python library for generating machine learning tooling.
Project-URL: Home, https://github.com/jostosh/py-gen-ml
Project-URL: Documentation, https://jostosh.github.io/py-gen-ml
Author-email: Jos van de Wolfshaar <jos.vandewolfshaar@gmail.com>
License: MIT License
        
        Copyright (c) 2023 Jos van de Wolfshaar
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: code generation,hyperparameter optimization,machine learning,mlops,optuna,protobufs
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.9
Requires-Dist: grpcio-tools>=1.46.0
Requires-Dist: jsonmerge>=1.9.2
Requires-Dist: mkdocs-material[docs,imaging]>=9.5.39
Requires-Dist: more-itertools>=10.5.0
Requires-Dist: networkx>=3.0
Requires-Dist: numpy<2
Requires-Dist: optuna>=3.2.0
Requires-Dist: protobuf<5.28.0,>=5.0.0
Requires-Dist: protogen>=0.3.1
Requires-Dist: pydantic<3,>=2
Requires-Dist: rich>=13.8.1
Requires-Dist: ruamel-yaml>=0.17.32
Requires-Dist: typer>=0.12.5
Requires-Dist: yapf>=0.40.2
Provides-Extra: docs
Requires-Dist: mdx-truly-sane-lists>=1.3; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5.39; extra == 'docs'
Requires-Dist: mkdocs-section-index>=0.3.9; extra == 'docs'
Requires-Dist: mkdocs-video>=1.5.0; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.26.1; extra == 'docs'
Requires-Dist: pymdown-extensions>=10.11.2; extra == 'docs'
Description-Content-Type: text/markdown

<p align="center">
   <a href="https://jostosh.github.io/py-gen-ml"><img src="docs/assets/images/logo.svg" alt="py-gen-ml" width="200"></a>

</p>
<p align="center">
    <em>py-gen-ml, generate machine learning code from a protobuf schema.</em>
</p>
<p align="center">

---

**Documentation**: <a href="https://jostosh.github.io/py-gen-ml" target="_blank">https://jostosh.github.io/py-gen-ml</a>

---

# 🌟 Project Introduction

`py-gen-ml` simplifies the configuration and management of machine learning projects. It leverages Protocol Buffers (protobufs) to provide a robust, strongly typed, and extensible way to define and manipulate configuration schemas for machine learning projects.

## 🔑 Key Features

**📌 Single Source of Truth**:

- The Protobuf schema provides a centralized definition for your configurations.

**🔧 Flexible Configuration Management**:

- **Minimal Change Amplification**: Automatically generated code reduces cascading manual changes when modifying configurations.
- **Flexible Patching**: Easily modify base configurations with patches for quick experimentation.
- **Flexible YAML**: Use human-readable YAML with support for advanced references within and across files.
- **Hyperparameter Sweeps**: Effortlessly define and manage hyperparameter tuning.
- **CLI Argument Parsing**: Automatically generate command-line interfaces from your configuration schemas.

**✅ Validation and Type Safety**:

- **JSON Schema Generation**: Easily validate your YAML content as you type.
- **Strong Typing**: The generated code comes with strong typing that will help you, your IDE, the type checker and your team to better understand the codebase and to build more robust ML code.

# 🚦 Getting Started

To start using py-gen-ml, you can install it via pip:

```console
pip install py-gen-ml
```

For a quick example of how to use py-gen-ml in your project, check out our [Quick Start Guide](https://jostosh.github.io/py-gen-ml/quickstart/).

# 💡 Motivation

Machine learning projects often involve complex configurations with many interdependent parameters. Changing one config (e.g., the dataset) might require adjusting several other parameters for optimal performance. Traditional approaches to organizing configs can become unwieldy and tightly coupled with code, making changes difficult.

`py-gen-ml` addresses these challenges by:

1. 📊 Providing a single, strongly-typed schema definition for configurations.
2. 🔄 Generating code to manage configuration changes automatically.
3. 📝 Offering flexible YAML configurations with advanced referencing and variable support.
4. 🛠️ Generating JSON schemas for real-time YAML validation.
5. 🔌 Seamlessly integrating into your workflow with multiple experiment running options:
   - Single experiments with specific config values
   - Base config patching
   - Parameter sweeps via JSON schema validated YAML files
   - Quick value overrides via a generated CLI parser
   - Arbitrary combinations of the above options

This approach results in more robust ML code, leveraging strong typing and IDE support while avoiding the burden of change amplification in complex configuration structures.

# 🎯 When to use `py-gen-ml`

Consider using `py-gen-ml` when you need to:

- 📈 Manage complex ML projects more efficiently
- 🔬 Streamline experiment running and hyperparameter tuning
- 🛡️ Reduce the impact of configuration changes on your workflow
- 💻 Leverage type safety and IDE support in your ML workflows

## 📚 Where to go from here

- [Quickstart](https://jostosh.github.io/py-gen-ml/quickstart/): A quick intro to the most important concepts.
- [Command Line Interface](https://jostosh.github.io/py-gen-ml/guides/cli_argument_parsing/): How to use the generated CLI parser.
- [Parameter Sweeps](https://jostosh.github.io/py-gen-ml/guides/sweep/): How to run parameter sweeps.
- [Generated factories](https://jostosh.github.io/py-gen-ml/guides/builders/): How to generate factories to instantiate your classes.
- [Cifar 10 example project](https://jostosh.github.io/py-gen-ml/example_projects/cifar10/): A more elaborate example of a machine learning project using `py-gen-ml`.
