Metadata-Version: 2.4
Name: chameo
Version: 1.0.0
Summary: CLI for Chameo template generator for Creating Reproducible Machine Learning Projects on Chameleon Cloud
Author-email: Ahmed Alghali <a7medalghali777@gmail.com>, Mohamed Saeed <mohdsaeed18x3@gmail.com>, Fraid Fund <ffund@nyu.edu>
License: MIT License
        
        Copyright (c) 2025 Ahmed Alghali
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: homepage, https://github.com/A7med7x7/Chameo
Project-URL: repository, https://github.com/A7med7x7/Chameo
Keywords: Chameleon Cloud,MLOps,MLflow,Reproducibility
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: GPU :: NVIDIA CUDA :: 11
Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
Classifier: Environment :: OpenStack
Classifier: Framework :: Jupyter :: JupyterLab
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: MacOS
Classifier: Operating System :: Unix
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: copier>=9.0
Dynamic: license-file

# Chameo: A Template Generator for Reproducible Machine Learning Projects on Chameleon Cloud

[![Copier](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/copier-org/copier/master/img/badge/badge-grayscale-inverted-border-orange.json)](https://github.com/copier-org/copier)
![Python](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11-blue.svg)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![GitHub release](https://img.shields.io/github/v/release/a7med7x7/chamelab-cli)](https://github.com/a7med7x7/Chameo/releases)
[![Demo Tutorial](https://img.shields.io/badge/demo-LLM%20Experiment-purple.svg)](https://github.com/A7med7x7/Chameo/tree/training-demo)


>[!NOTE]
>
> Use this template generator when you are ready to start a machine learning project on [Chameleon Cloud](https://www.chameleoncloud.org), or if you want to integrate reproducible experiment tracking into your existing project.

**Reproducibility** in Machine Learning means being able to reproduce the results of an experiment using the same code, with the same data and environment, and obtain the same results.  
in research, this remains a **significant problem** — many papers lack proper experiment tracking, dependency capturing, versioning, and artifact sharing. As a result, even the original authors often struggle to reproduce their own results several months later.
this weakens trust and slows scientific progress, since others cannot easily validate or build upon previous work.  
in production, however, **MLOps practices** (automation, versioning, monitoring) have proven to be key enablers of reproducible and trustworthy ML systems. 
but for researchers, **adapting these tools and practices is full of friction** — stitching everything together takes considerable time
and effort.  

**Chameo removes this barrier** by providing ready-to-use templates with reproducibility baked in, so you can focus on spinning up your VMs instead of infrastructure.

**Supporting Reproducibility with Chameo**:  
- It makes it easier for researchers and scientists to adopt MLOps practices in their projects.  
- It provides ready-to-launch templates with reproducibility baked in (experiment tracking, consistent environments, artifact persistence).  
- It lowers the barrier to reproducible research on platforms like **Chameleon Cloud**.  
 > [!TIP]    
> You can use our **[mlfow-replay](https://github.com/A7med7x7/Chameo/tree/mlflow-replay)** setup to **revisit an experiment previously run on Chameleon Cloud**.  
> This branch includes an MLflow setup that **reads and load artifacts of a previous experiment generated by our Chameo template generator** from a public Chameleon Cloud object store containers.  
> This allows you to **reproduce, inspect, and extend** experiments easily.

> [!NOTE]  
> You can try a **hands-on demo tutorial** with Chameo 🦎.  
> Check out our **[LLM Experiment Demo](https://github.com/A7med7x7/chameo/tree/llm-demo)** where we
fine-tune and track a language model experiment on Chameleon Cloud using Chameo's reproducible template. The tutorial shows how **experiment tracking, MLflow logging, and artifact persistence** work in practice. It’s a great starting point if you want to adapt Chameo for your own deep learning workflows.  

---


## How to use this template generator to create a new project

This repository is not intended to be cloned or forked directly. Instead use Copier to generate a project from the template, which will prompt you for configuration.

you will learn more about this in **[Getting Started](#getting-started)** 

---
## Requirements

- [Chameleon Cloud account](https://www.chameleoncloud.org)
- [Python](https://www.python.org/downloads/)

> [!Caution]
> If you have python pre-installed on your machine, make sure its version ≥ 3.9 

---

##  Installation

Install the `chameo` CLI:

```sh
pip install chameo
```
or 

```
pip3 install chameo
```

---

## Getting Started

Create a New Project with `Chameo` CLI
```sh
chameo create myproject
```
> [!IMPORTANT]
> `myproject` = `path/to/destination`  make sure it points to an empty directory, and replace `path/to/destination` with the name of the path and directory you want

Answer a few questions, and the CLI will generate the project in that directory. See **[Setup Parameters and Their Roles](#setup-parameters-and-their-roles)** below if you want to know the role of the questions and what they will generate. 

> [!TIP]
> If you're new to Chameleon Cloud, we recommend using `Basic` in [setup mode](#setup_mode). It's beginner-friendly!
---
### Follow your README.md 
When the project is generated, a README.md file will be created at the root.it contains all the instructions to guide you through setting your environment.

---
## Setup Parameters and Their Roles
Your answers will generate a project based on your input values, below you can find the variables with their description and implications  

### `setup_mode`
Defines the overall setup approach for your environment. tweak the customization you want during server and Docker environment creation.
- **Basic**: minmal prompts. you will be asked to input your project name ([`project_name`](#project_name)), remote repository link ([`repo_url`](#repo_url)) and framework of your choice ([`ml_framework`](#ml_framework)) it recommend defaults for most options.
- **Advanced**: Lets you control Compute site ([chameleon_site](#chameleon_site)), GPU type ([`gpu_type`](#gpu_type)), CUDA version ([`cuda_version`](#cuda_version)), storage site ([`bucket_site`](#bucket_site))
The rest of the documentation shows what these options are and their implications 
- **Type**: Single-select
- **Default**: "Basic"

--- 

### `project_name`

- We recommend setting `project name` as the prefix for the lease name 
- It is used everywhere your project is referenced:   
    - object store names (e.g., `project-name-data`, `project-name-mlflow-artifacts`)
    - Compute instances /servers are going to include the `project_name` as their prefix 
    
        ```python
            # when creating a server
			s = server.Server(
			f"{{ project_name }}-node-{username}"
        ```
			
     
	- Your material on the compute instance will be under a directory named after your `project_name`
    - The containerized environment will look for a directory with the `project_name` directory 
	- most commands and scripts assume a unified `project_name`
- **Rules**: only letters, numbers, hyphen (-), and underscore (_). no spaces.
- **Tip**: Choose something short and memorable — remember this will show up in multiple commands and URLs
- **Type:** `str`

---

### `repo_url`

- The remote Git repository where the generated project will live, we recommend creating a remote repository (e.g [GitHub](github.com) or [GitLab](gitlab.com)). 
- Accepts HTTPS or SSH URLs (e.g., `https://github.com/user/repo.git` or `git@gitlab.com:user/repo.git`).
- After having your project generated, you need to push the code there. (see [Github](https://docs.github.com/en/migrations/importing-source-code/using-the-command-line-to-import-source-code/adding-locally-hosted-code-to-github) / [Gitlabl](https://forum.gitlab.com/t/how-do-i-push-a-project-to-a-newly-created-git-repo-on-gitlab/68426) Guide on pushing code to remote repo)
- **Type:** `str`
- **Note**: can accept blank if you need to set the repository later,
you will need to manully input the remote repo in create_server notebook. 

---

### `chameleon_site`

The site where your leases at, and compute resources will be provisioned. 
- This doesn’t control persistent storage storage location (that’s [`bucket_site`](#bucket_site)).
#### **options**
- CHI@TACC → Most GPU bare metal nodes.
- CHI@UC → University of Chicago resources.
- KVM@TACC → VM(Virtual Machines )-based compute at TACC.
- **Type:** `select`
- **Default**: `CHI@TACC`

--- 

### `bucket_site`
###### *work only under advanced*

- This is where your [object storage contrainers](https://chameleoncloud.readthedocs.io/en/latest/technical/swift/index.html#object-store) for you project will live.
#### **options**

- CHI@TACC: Texas Advanced Computing Center
- CHI@UC: University of Chicago
- **auto** is usually the best choice unless you have a reason to store data in a specific  location. it matches your selected `chameleon_site` if object storage containers are available, if not it defaults to CHI@TACC site. 
- **Type:** `select`
- **Default**: `CHI@TACC`

--- 

### `gpu_type`
###### *work only under advanced*

- The type of GPU (or CPU-only) node you want to create and configure. this assumes that you have reserved a node and you know which type it is AMD, NVIDIA or CPU.
- configuring a server from a lease require the `gpu_type`, as different `gpus` have different setup process. 
- `nvidia` and `amd` require different container images to. so your decision will result in selecting the appropriate [container images](https://github.com/A7med7x7/Chameo/tree/dev/template/docker)
- **Type:** `multi-choice` - you can select multiple types. 
- **Default**: `NVIDIA`
- **Note**: when selecting `chemeleon_site` = KVM@TACC the GPU flavors run on NVIDIA hardware as there are no AMD variant. 

---  

### `ml_framework`

- Selects the primary ML/deep learning framework for your environment.
- It will decide which container image to include and use for your Jupyter Lab. 
- Custom training code for the selected `ml_framework` will be generated
- **pytorch** – Flexible, widely used deep learning library. Supports CUDA (NVIDIA) and ROCm (AMD).
- **pytorch-lightning** – High-level wrapper for PyTorch that simplifies training loops. Supports CUDA (NVIDIA) and ROCm (AMD).
- **tensorflow** – Popular deep learning library with a strong ecosystem. 
- **scikit-learn** –  Machine Learning and data science stack (pandas, scikit-learn, matplotlib, etc.) without deep learning frameworks.
- **Note**: _PyTorch_ and _PyTorch Lightning_ will prompt for CUDA/ROCm version if you select GPU types.
- **Type** `multi-choice`: you can select multiple frameworks. 

---

### `cuda_version` 
###### *work only under advanced and GPU == NVIDIA*

- Choose the CUDA version that matches your code and driver requirements.
    - `cuda11-latest` : highly compatible with most GPUs in Chameleon Cloud 
    - `cuda12-latest` : The latest version designed to work with newer GPU architectures
- **Type** `select`
- **Default**: `cuda11-latest`
---
### `env_setup_mode`
###### *work only under advanced* 
How would you like to configure your server (.env file generation & docker compose setup)?
You have two options:
- **SSH** : 
SSH into your reserved compute instance and follow the README instructions to generate the .env file and launch your docker compose environment.
- **Notebook**: 
stay inside Chameleon JupyterHub and use a notebook (2_configure_server.ipynb) provided in the chi directory to configure your server.
This approach is more guided and beginner-friendly.
- both methods generate the same .env file and launch the same docker compose environment.
The only difference is where you perform the steps: via SSH (manual control) or Notebook (fully browser-based).
- **Type**: `select`
- **Default**: `"notebook"` if `setup_mode == 'Basic'`

---

### `include_huggingface` 
###### *work only under advanced* 

If enabled, it configures the environment to include a HuggingFace token for seamless Hugging Face Hub access and caching of models/datasets . 
- During server setup you will be prompted to enter a [Hugging Face Token](https://huggingface.co/settings/tokens) 
- All models/datasets downloaded from Hugging Face will be stored on the mounted point `/mnt/data/`
- **Type**: `bool`
- **Default**: `true`

---

#### Acknowledgements

This project was supported by the 2025 [Summer of Reproducibility](https://ucsc-ospo.github.io/sor/).

Contributors: [Ahmed Alghali](https://ucsc-ospo.github.io/author/ahmed-alghali/), [Mohamed Saeed](https://ucsc-ospo.github.io/author/mohamed-saeed/), [Fraida Fund](https://ucsc-ospo.github.io/author/fraida-fund/).
