Metadata-Version: 2.2
Name: gpufl
Version: 1.0.1rc1
Summary: GPU Monitoring Client
Author-Email: Myoungho Shin <myounghoshin84@gmail.com>
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: C++
Classifier: Topic :: System :: Monitoring
Project-URL: Homepage, https://github.com/gpufl/gpufl-client
Project-URL: Issues, https://github.com/gpufl/gpufl-client/issues
Requires-Python: >=3.12
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == "torch"
Requires-Dist: requests>=2.28; extra == "torch"
Provides-Extra: cupy
Requires-Dist: cupy-cuda12x>=12; extra == "cupy"
Provides-Extra: jax
Requires-Dist: jax>=0.4; extra == "jax"
Provides-Extra: triton
Requires-Dist: triton>=2.1; extra == "triton"
Provides-Extra: numba
Requires-Dist: numba-cuda; extra == "numba"
Requires-Dist: numpy; extra == "numba"
Provides-Extra: viz
Requires-Dist: pandas>=1.5; extra == "viz"
Requires-Dist: matplotlib>=3.7; extra == "viz"
Provides-Extra: analyzer
Requires-Dist: pandas>=1.5; extra == "analyzer"
Requires-Dist: rich; extra == "analyzer"
Provides-Extra: all
Requires-Dist: torch>=2.1; extra == "all"
Requires-Dist: requests>=2.28; extra == "all"
Requires-Dist: cupy-cuda12x>=12; extra == "all"
Requires-Dist: jax>=0.4; extra == "all"
Requires-Dist: triton>=2.1; extra == "all"
Requires-Dist: numba-cuda; extra == "all"
Requires-Dist: numpy; extra == "all"
Requires-Dist: pandas>=1.5; extra == "all"
Requires-Dist: matplotlib>=3.7; extra == "all"
Requires-Dist: rich; extra == "all"
Description-Content-Type: text/markdown

# GPUFlight Client Library (gpufl)

**The Flight Recorder for GPU Production Workloads.**

`gpufl` is a **low-overhead, always-on** C++ observability library for GPU applications. Built on CUPTI (NVIDIA) and rocprofiler-sdk (AMD), it captures kernel telemetry, SASS-level profiling, and system metrics with under 2% overhead in monitoring mode.
Unlike traditional profilers (Nsight) that stop the world with 20-200x slowdown, GPUFlight is designed to run continuously in production — capturing kernel telemetry and logical scopes into structured logs.

## Project Status: Beta (Pre-1.0)

GPUFlight is usable today and published to PyPI, but it's pre-1.0 — current releases are `0.1.0.devN` dev builds. The public API and the on-the-wire log format may still change between releases; the first stable contract is `v1.0.0`. Pin an exact version if you depend on it.

To keep the initial design coherent, **we are not currently accepting major feature Pull Requests.** However, we welcome:
- Bug reports and local build issues.
- Documentation improvements and typo fixes.
- Feature requests and architectural suggestions via GitHub Issues.

## Live Demo

Try the portal with real session data — no sign-up required:

**[Demo Link](https://demo.gpuflight.com)**

## Key Features

- **Kernel Monitoring**: Automatically intercepts all CUDA kernel launches via CUPTI.
- **Production Grade**: Uses a **Lock-Free Ring Buffer** and a **Background Collector Thread** to decouple logging from your hot path.
- **Logical Scoping**: Group thousands of micro-kernels into meaningful phases (e.g., "Inference", "PhysicsStep") using `GFL_SCOPE` or `gpufl.Scope`.
- **Rich Metadata**: Captures kernel names, grid/block dimensions, register counts, shared memory usage, occupancy with per-resource breakdown, and CPU stack traces.
- **Profiling Engines**: Choose from PC Sampling (stall analysis), SASS Metrics (instruction-level divergence), or Range Profiler (hardware counters) — one engine per session.
- **System Monitoring**: Collects GPU utilization, VRAM, temperature, power, and clock speeds via NVML.
- **Sidecar Ready**: Outputs structured NDJSON logs with automatic rotation and gzip compression.
- **Direct Upload**: Opt-in `remote_upload=True` mode POSTs telemetry straight to the GPUFlight backend — ideal for local dev, SSH, and Jupyter workflows where running the sidecar agent is overkill.
- **Vendor Agnostic Design**: Architecture ready for AMD (ROCm) support.

---

## Installation

### Python (PyPI)
```bash
pip install "gpufl[analyzer,viz]"
```

For full NVML support (GPU utilization/VRAM monitoring), build from source inside a CUDA devel container:
```bash
git clone https://github.com/gpu-flight/gpufl-client.git
CMAKE_ARGS="-DBUILD_TESTING=OFF" pip install "./gpufl-client[analyzer,viz]"
```

### C++ (CMake FetchContent)
```cmake
cmake_minimum_required(VERSION 3.31)
project(my_app LANGUAGES CXX CUDA)

include(FetchContent)
FetchContent_Declare(
    gpufl
    GIT_REPOSITORY https://github.com/gpu-flight/gpufl-client.git
    GIT_TAG        v0.1.0.dev7   # pin a release tag — see the Releases page for the latest
)
FetchContent_MakeAvailable(gpufl)

add_executable(my_app main.cu)
target_link_libraries(my_app PRIVATE gpufl::gpufl CUDA::cudart CUDA::cupti)
```

---

## Quick Start (Python + Docker)

The recommended way to get started is with a Docker container. See [example/python/docker/Dockerfile](example/python/docker/Dockerfile) for a ready-to-use setup with PyTorch and Jupyter Lab.

```bash
cd example/python/docker
docker build -t gpufl-python .
docker run --gpus all -p 8888:8888 -v $(pwd)/notebooks:/workspace gpufl-python
```

```python
import torch
import gpufl

gpufl.init("my-app",
           log_path="./my_logs",
           sampling_auto_start=True,
           enable_stack_trace=True)

a = torch.randn(1024, 1024, device="cuda")
b = torch.randn(1024, 1024, device="cuda")
c = a @ b
torch.cuda.synchronize()

gpufl.shutdown()
```

---

## Profiling Engines

CUPTI only allows one profiling mode per CUDA context at a time. Choose an engine at init:

```python
from gpufl import ProfilingEngine

gpufl.init("my-app",
           log_path="./logs",
           profiling_engine=ProfilingEngine.PcSampling)
```

| Engine | What it collects | Analyzer method | Best for |
|---|---|---|---|
| `PcSampling` | Warp stall reasons (statistical sampling) | `session.inspect_stalls()` | Finding *why* warps are stalling |
| `SassMetrics` | Per-instruction execution counts (binary instrumentation) | `session.inspect_profile_samples()` | Thread divergence detection |
| `RangeProfiler` | SM throughput, L1/L2 hit rates, DRAM bandwidth, tensor core % | `session.inspect_perf_metrics()` | Hardware counter deep-dives |
| `None` | Kernel metadata only (names, timing, occupancy) | `session.inspect_hotspots()` | Production monitoring with minimal overhead |

### C++ Usage

```cpp
gpufl::InitOptions opts;
opts.app_name = "my_app";
opts.log_path = "my_logs";
opts.enable_stack_trace = true;
opts.sampling_auto_start = true;
opts.profiling_engine = gpufl::ProfilingEngine::SassMetrics;

gpufl::init(opts);

GFL_SCOPE("training_step") {
    // your CUDA code here
}

gpufl::shutdown();
```

---

## Talking to the Backend

`gpufl` can optionally interact with the GPUFlight backend in two
independent ways, both opt-in:

| Capability | Opt in via | What happens |
|---|---|---|
| **Fetch remote named config** | `config_name="..."` | `gpufl.init()` does a one-off `GET /api/v1/config?config=<name>` before monitoring starts and applies the returned fields to your `InitOptions`. |
| **Direct log upload** | `remote_upload=True` | A background thread POSTs every NDJSON line to `/api/v1/events/<type>` in parallel with the on-disk write. Intended for local / SSH / Jupyter workflows. |

Both require `backend_url` + `api_key` to be set. **Setting those two
fields alone does nothing** — you must opt into at least one of the
two capabilities above.

### Configuration precedence

When multiple sources set the same field, higher beats lower:

```
5. The kwargs you pass to gpufl.init()            ← highest
4. Env vars (GPUFL_BACKEND_URL, GPUFL_API_KEY, ...)
3. Local config file (config_file=...)
2. Remote named config (when config_name is set)
1. Built-in defaults                              ← lowest
```

### Quick start

```python
import gpufl

# Just live upload — no remote config fetch
gpufl.init("my_app",
           log_path="./logs",
           backend_url="https://api.gpuflight.com",
           api_key="gpfl_xxxxxxxxxxxx",
           remote_upload=True)

# Just remote config fetch — no upload (production uses the sidecar)
gpufl.init("my_app",
           backend_url="https://api.gpuflight.com",
           api_key="gpfl_xxxxxxxxxxxx",
           config_name="production")

# Both
gpufl.init("my_app",
           backend_url="https://api.gpuflight.com",
           api_key="gpfl_xxxxxxxxxxxx",
           config_name="production",
           remote_upload=True)
```

Or via environment:

```bash
export GPUFL_BACKEND_URL=https://api.gpuflight.com
export GPUFL_API_KEY=gpfl_xxxxxxxxxxxx
export GPUFL_CONFIG_NAME=production     # opt into remote config
export GPUFL_REMOTE_UPLOAD=1            # opt into live upload
python my_training_script.py
```

### Upload mechanics

- The client writes NDJSON to disk as usual AND POSTs each line in
  parallel via a dedicated background thread — no added latency on the
  measurement hot path.
- Best-effort delivery: 3 retries with exponential backoff per line,
  then drop. The on-disk NDJSON is still there, so a monitor daemon
  (or manual re-upload) can always back-fill.
- An invalid or unauthorized API key disables live upload for the
  session; the file sink keeps working.

For production workloads with guaranteed delivery, pipe delivery, or
cross-process fan-in, keep using the `gpufl-monitor` sidecar agent —
it's the same NDJSON on the wire, just decoupled from the measurement
process.

---

## Python Analysis

The `gpufl.analyzer` module loads NDJSON logs and provides Rich-formatted terminal dashboards.

```python
from gpufl.analyzer import GpuFlightSession

session = GpuFlightSession("./logs", log_prefix="my_logs")

# Executive Summary: session duration, kernel count, GPU utilization, VRAM
session.print_summary()

# Top kernels by GPU time with occupancy breakdown and stack traces
session.inspect_hotspots(top_n=5)

# Time breakdown by user-defined Scope regions
session.inspect_scopes()

# PC Sampling: per-kernel stall reason distribution
session.inspect_stalls(top_n=10)

# SASS Metrics: instruction-level execution counts and divergence
session.inspect_profile_samples(top_n=10)

# Range Profiler: SM throughput, cache hit rates, DRAM bandwidth
session.inspect_perf_metrics(top_n=10)
```

![Analyzer example output](images/Screenshot1.png)

### Visualization (Timeline)
The `viz` module provides interactive `matplotlib` plots to correlate kernel execution with system metrics.

```python
import gpufl.viz as viz

viz.init("./logs/*.log")
viz.show()
```

---

## Report Generation

For a quick, shareable text summary of a session — session metadata, kernel
hotspots, duration percentiles, and system metrics — generate a **text report**.
It's the fastest way to see "what happened" without opening the dashboard, and
it drops cleanly into CI logs, PR comments, or a plain terminal.

![Text report example](images/Screenshot2.png)

The report includes:
- **Session Summary** — app name, session ID, duration, GPU device + SM count.
- **Kernel Execution Summary** — total / unique kernels, GPU-busy %, and
  duration statistics (avg / median / P90 / P99 / min / max). When a SASS
  profiling engine was active, kernel durations include instrumentation
  overhead and the report labels them accordingly.
- **Top kernels by total GPU time** — with per-kernel call counts.
- **Per-kernel details** — grid/block dimensions, occupancy, registers,
  shared memory (static + dynamic), register spills, and Waves/SM.

### From C++

Call `generateReport()` after `shutdown()` — it reads the NDJSON logs written
during the session:

```cpp
gpufl::init(opts);
// ... your CUDA / HIP work ...
gpufl::shutdown();

gpufl::generateReport();               // print to stdout
gpufl::generateReport("report.txt");   // or save to a file
```

### From Python

```python
from gpufl.report import generate_report

# Print the report — wrap in print() so newlines render. In a Jupyter
# notebook this also keeps the table columns aligned (stdout renders in
# a monospace font). A bare `generate_report(...)` as a cell's last
# expression shows an escaped one-line string, so always print() it.
text = generate_report("./logs", log_prefix="my_app", top_n=10)
print(text)

# Or save it straight to a file
generate_report("./logs", log_prefix="my_app", top_n=10, output_path="report.txt")
```

The Python version reads the same NDJSON logs the analyzer uses — no GPU
required, so you can generate reports from logs copied off another machine.

---

## Testing

### C++ Tests
The C++ tests use GoogleTest and are hardware-aware — NVIDIA-specific tests skip automatically if no compatible GPU is detected.

```bash
cmake --build cmake-build-debug --target gpufl_tests
ctest --test-dir cmake-build-debug --output-on-failure
```

### Python Tests
```bash
pip install pytest
pytest tests/python/
```

---

## Linux Configuration (Required for CUPTI)

To allow non-root users to profile GPU kernels (using CUPTI/PC Sampling) on Linux, you must relax the NVIDIA driver security restrictions. Without this, `gpufl` may fail to capture kernel activity.

1. **Create a configuration file:**
   ```bash
   sudo nano /etc/modprobe.d/nvidia-profiler.conf
   ```

2. **Add the following line:**
   ```
   options nvidia NVreg_RestrictProfilingToAdminUsers=0
   ```

3. **Apply changes and reboot:**
   ```bash
   sudo update-initramfs -u
   sudo reboot
   ```

---

## Where your logs go

By default the client writes NDJSON to disk. To stream them to a hosted
dashboard, set `backend_url` + `api_key` (or the `GPUFL_BACKEND_URL` /
`GPUFL_API_KEY` env vars) and they're delivered live to
[app.gpuflight.com](https://app.gpuflight.com). Create a workspace at
[gpuflight.com](https://gpuflight.com)

This client (gpufl-client) is open source. The ingestion service and
the dashboard UI are proprietary and managed-only today.
