Metadata-Version: 2.1
Name: galadriel-node
Version: 0.0.19
Summary: 
Author: Kaspar Peterson
Author-email: kaspar@galadriel.com
Requires-Python: >=3.10,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Provides-Extra: dev
Requires-Dist: accelerate (>=1.1.0,<2.0.0)
Requires-Dist: aiohttp (>=3.10.5,<4.0.0)
Requires-Dist: black (>=24.8.0,<25.0.0) ; extra == "dev"
Requires-Dist: dataclasses-json (>=0.6.7,<0.7.0)
Requires-Dist: diffusers (>=0.31.0,<0.32.0)
Requires-Dist: fastapi (>=0.115.0,<0.116.0)
Requires-Dist: gpustat (>=1.1.1,<2.0.0)
Requires-Dist: openai (>=1.55.3,<2.0.0)
Requires-Dist: ping3 (>=4.0.8,<5.0.0)
Requires-Dist: psutil (>=6.0.0,<7.0.0)
Requires-Dist: py-cpuinfo (>=9.0.0,<10.0.0)
Requires-Dist: pytest (>=8.3.2,<9.0.0) ; extra == "dev"
Requires-Dist: pytest-asyncio (==0.24.0) ; extra == "dev"
Requires-Dist: pytest-mock (>=3.14.0,<4.0.0) ; extra == "dev"
Requires-Dist: python-dotenv (>=1.0.1,<2.0.0)
Requires-Dist: requests (==2.31.0)
Requires-Dist: speedtest-cli (>=2.1.3,<3.0.0)
Requires-Dist: transformers (>=4.0.0,<5.0.0)
Requires-Dist: typer (>=0.12.3,<0.13.0)
Requires-Dist: vllm (==0.6.4.post1) ; sys_platform == "linux"
Requires-Dist: websockets (>=13.0,<14.0)
Description-Content-Type: text/markdown

# Galadriel inference node

Galadriel is building the world’s largest distributed LLM inference network - for developers to scale their LLM apps while cutting costs.

Run a Galadriel GPU node to provide LLM inference to the network.

Check out the quickstart in the [documentation](https://docs.galadriel.com/nodes/quickstart) to get started.


## Requirements

### Hardware requirements

- At least 4 CPU cores
- At least 8GB RAM
- A supported Nvidia GPU

### Software requirements
- linux (Ubuntu recommended)
- python (version 3.10+)
- nvidia drivers, version > 450. `nvidia-smi` must work

### API keys
- A valid galadriel API key, gotten from the Galadriel [dashboard](https://dashboard.galadriel.com/)

### Run a GPU node from the command line

**Create a (separate) python environment**
```shell
deactivate
mkdir galadriel
cd galadriel
python3 -m venv venv
source venv/bin/activate
```

**Install galadriel-node**
```shell
pip install galadriel-node
```

**Setup the environment**

Only update values that are not the default ones, and make sure to set the API key
```shell
galadriel init
```

**Run the node**
```shell
galadriel node run
```
If this is your first time running the GPU node, it will perform hardware validation and LLM benchmarking, 
to ensure your setup is working correctly and is fast enough.

**Or run with nohup to run in the background**
```shell
nohup galadriel node run > logs.log 2>&1 &
```

**Check node status**
```shell
galadriel node status
```
Should see status: online


### Additional commands

**Check LLM status**
```shell
galadriel node llm-status
```
The output should contain:
```
✓ LLM server at http://your_llm_address is accessible via HTTP.
✓ LLM server at http://your_llm_address successfully generated tokens.
```

**Check node stats**
```shell
galadriel node stats
```

