Metadata-Version: 2.4
Name: mlflow-crusoe
Version: 0.1.3
Summary: MLflow deployment plugin for Crusoe AI's managed inference API.
Author-email: Crusoe AI <devcommunity@crusoe.ai>
License: MIT
Project-URL: Homepage, https://github.com/crusoecloud/crusoe-developer-hub/tree/main/integrations/mlflow
Project-URL: Repository, https://github.com/crusoecloud/crusoe-developer-hub/tree/main/integrations/mlflow
Project-URL: Issues, https://github.com/crusoecloud/crusoe-developer-hub/issues
Project-URL: Documentation, https://docs.crusoecloud.com/serverless-inference/available-models
Keywords: mlflow,crusoe,ai,deployment,inference
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mlflow>=2.0
Requires-Dist: openai>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Dynamic: license-file

# mlflow-crusoe

[![PyPI](https://img.shields.io/pypi/v/mlflow-crusoe.svg)](https://pypi.org/project/mlflow-crusoe/)

An [MLflow](https://mlflow.org/) deployment plugin for [Crusoe AI](https://www.crusoe.ai/)'s managed inference API.

This plugin lets you manage named model endpoint configurations and run inference on Crusoe's renewable-powered GPU infrastructure directly from MLflow's deployment interface.

## Installation

```bash
pip install mlflow-crusoe
```

## Setup

Set your Crusoe API key:

```bash
export CRUSOE_API_KEY="your-api-key"
```

You can generate one from the [Crusoe Console](https://console.crusoe.ai/) under **Security > Inference API Key**.

## Usage

```python
import mlflow.deployments

client = mlflow.deployments.get_deploy_client("crusoe")

# Create a deployment
client.create_deployment(
    name="my-llm",
    model_uri="meta-llama/Llama-3.3-70B-Instruct",
    config={"temperature": 0.7, "max_tokens": 2048},
)

# Run inference
result = client.predict("my-llm", inputs={"prompt": "Hello!"})
print(result["choices"][0]["message"]["content"])

# List deployments
client.list_deployments()

# Update
client.update_deployment("my-llm", model_uri="zai/GLM-5.3")

# Delete
client.delete_deployment("my-llm")
```

### Input formats

The `predict` method accepts three input formats:

```python
# Chat messages (recommended)
client.predict("my-llm", inputs={
    "messages": [
        {"role": "system", "content": "You are helpful."},
        {"role": "user", "content": "Hello!"},
    ]
})

# Simple prompt
client.predict("my-llm", inputs={"prompt": "Hello!"})

# Plain string
client.predict("my-llm", inputs="Hello!")
```

### CLI

```bash
# Help
mlflow deployments help -t crusoe

# Create
mlflow deployments create -t crusoe --name my-llm -m meta-llama/Llama-3.3-70B-Instruct

# List
mlflow deployments list -t crusoe

# Predict
mlflow deployments predict -t crusoe --name my-llm --input '{"prompt": "Hi"}'

# Delete
mlflow deployments delete -t crusoe --name my-llm
```

## Available Models

| Model | Identifier |
|-------|-----------|
| Meta Llama 3.3 70B Instruct | `meta-llama/Llama-3.3-70B-Instruct` |
| DeepSeek V3 (0324) | `deepseek-ai/DeepSeek-V3-0324` |
| DeepSeek V4 Pro | `deepseek-ai/DeepSeek-V4-Pro` |
| DeepSeek V4 Flash | `deepseek-ai/DeepSeek-V4-Flash` |
| Google Gemma 4 31B | `google/gemma-4-31b-it` |
| Moonshot AI Kimi K2.6 | `moonshotai/Kimi-K2.6` |
| NVIDIA Nemotron 3 Nano 30B A3B | `nvidia/Nemotron-3-Nano-30B-A3B` |
| NVIDIA Nemotron 3 Nano Omni Reasoning 30B A3B | `nvidia/Nemotron-3-Nano-Omni-Reasoning-30B-A3B` |
| NVIDIA Nemotron 3 Super 120B A12B | `nvidia/Nemotron-3-Super-120B-A12B` |
| NVIDIA Nemotron 3 Ultra 550B | `nvidia/Nemotron-3-Ultra-550B` |
| NVIDIA Nemotron 3.5 Lightning 30B A3B | `nvidia/nemotron-3.5-lightning-30b-a3b` |
| OpenAI GPT-OSS 120B | `openai/gpt-oss-120b` |
| Qwen3 235B A22B Instruct | `Qwen/Qwen3-235B-A22B-Instruct-2507` |
| Z.ai GLM-5.1 | `zai/GLM-5.1` |
| Z.ai GLM-5.3 | `zai/GLM-5.3` |
| Z.ai GLM-5.3 Flash | `zai/GLM-5.3-Flash` |

Any other model ID available on Crusoe Managed Inference can be passed as a plain string. See the [Crusoe model list](https://docs.crusoecloud.com/serverless-inference/available-models) for current availability.

## Configuration

| Key | Default | Description |
|-----|---------|-------------|
| `api_key` | `CRUSOE_API_KEY` env var | API key |
| `api_base` | `https://api.inference.crusoecloud.com/v1` | API base URL |
| `temperature` | `0.1` | Sampling temperature (0-2) |
| `max_tokens` | `1024` | Max tokens to generate |
| `top_p` | unset | Nucleus sampling |
| `frequency_penalty` | unset | Frequency repetition penalty |
| `presence_penalty` | unset | Presence repetition penalty |
| `stop` | unset | Comma-separated stop sequences |

## Compatibility

Requires MLflow 2.0 or newer, including MLflow 3.x. The plugin registers through the `mlflow.deployments` entry point and implements the full plugin interface (`CrusoeDeploymentClient`, `target_help`, `run_local`).

## References

- [Crusoe Serverless Inference Docs](https://docs.crusoecloud.com/serverless-inference/overview)
- [Crusoe Model List](https://docs.crusoecloud.com/serverless-inference/available-models)
- [MLflow Deployment Plugins](https://mlflow.org/docs/latest/ml/plugins/)
- [Crusoe Console](https://console.crusoe.ai/)
