Metadata-Version: 2.4
Name: graphsignal
Version: 0.20.2
Summary: Graphsignal Profiler
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: Inference profiler,CUDA profiler
Author: Graphsignal, Inc.
Author-email: devops@graphsignal.com
Requires-Python: >=3.9,<4
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: System :: Monitoring
Provides-Extra: cu12
Provides-Extra: cu13
Requires-Dist: grpcio
Requires-Dist: nvidia-cuda-cupti (>=13,<14) ; (sys_platform == "linux") and (extra == "cu13")
Requires-Dist: nvidia-cuda-cupti-cu12 (>=12,<13) ; (sys_platform == "linux") and (extra == "cu12")
Requires-Dist: nvidia-ml-py (>=12)
Requires-Dist: opentelemetry-proto
Requires-Dist: prometheus-client
Requires-Dist: protobuf
Requires-Dist: psutil (>=5.9)
Requires-Dist: requests
Requires-Dist: xxhash
Project-URL: Homepage, https://graphsignal.com
Project-URL: Repository, https://graphsignal.com
Description-Content-Type: text/markdown

# Graphsignal: Inference Profiler

[![License](http://img.shields.io/github/license/graphsignal/graphsignal-profiler)](https://github.com/graphsignal/graphsignal-profiler/blob/main/LICENSE)
[![Version](https://img.shields.io/github/v/tag/graphsignal/graphsignal-profiler?label=version)](https://github.com/graphsignal/graphsignal-profiler)


Graphsignal is a production-scale inference profiling platform that helps engineers optimize AI performance across models, engines, GPUs, and other accelerators. It provides essential visibility across the inference stack, including:

* Continuous, high-resolution profiling timelines exposing operation durations and resource utilization across inference workloads.
* LLM generation tracing with per-step timing, token throughput, and latency breakdowns for major inference frameworks.
* System-level metrics for inference engines and hardware (CPU, GPU, accelerators).
* Error monitoring for device-level failures and inference errors.
* Inference telemetry for AI agents to identify bottlenecks and drive targeted improvements across the inference stack.

[![Dashboards](https://graphsignal.com/external/screenshot-dashboard.png?v=2)](https://graphsignal.com/)

Learn more at [graphsignal.com](https://graphsignal.com).


## Install

```bash
UV_TOOL_BIN_DIR=/usr/local/bin uv tool install 'graphsignal[cu12]'   # CUDA 12.x
# or
UV_TOOL_BIN_DIR=/usr/local/bin uv tool install 'graphsignal[cu13]'   # CUDA 13.x
```

### Alternative: install into your workload environment

If you prefer a single environment, or you use the `graphsignal.watch()` Python API (which requires `graphsignal` importable by your application), install it directly into your workload's environment instead:

```bash
pip install 'graphsignal[cu12]'   # CUDA 12.x
# or
pip install 'graphsignal[cu13]'   # CUDA 13.x
```


## Profile

Wrap your launch command with `graphsignal-run`:

```bash
export GRAPHSIGNAL_API_KEY=<my-api-key>
graphsignal-run vllm serve <model> --port 8001
```

Environment variables read by the profiler:

| Variable                              | Purpose                                                                |
| ------------------------------------- | ---------------------------------------------------------------------- |
| `GRAPHSIGNAL_API_KEY` *(required)*    | Your account API key.                                                  |
| `GRAPHSIGNAL_TAG_<KEY>=<value>`       | Arbitrary tag attached to all signals (e.g. `GRAPHSIGNAL_TAG_DEPLOYMENT=us-prod`). |

Sign up for a free account at [graphsignal.com](https://graphsignal.com); you'll find the API key in [Settings / API Keys](https://app.graphsignal.com/settings/api-keys).

See the [Profiler CLI](https://graphsignal.com/docs/reference/profile-cli/) reference for the full set of options.

Applications that bootstrap themselves can call `graphsignal.watch()` from Python instead — see the [Profiler API](https://graphsignal.com/docs/reference/profiler-api/) reference.

See integration documentation for libraries and inference engines:

* [PyTorch](https://graphsignal.com/docs/integrations/pytorch/)
* [vLLM](https://graphsignal.com/docs/integrations/vllm/)
* [SGLang](https://graphsignal.com/docs/integrations/sglang/)


## Optimize

[Log in](https://app.graphsignal.com/) to Graphsignal to monitor and analyze your application.

### Optimize with AI

Install the Graphsignal skill to let your AI coding agent (Claude Code, Codex, or Gemini) fetch and analyze signal context directly from your agent. See [AI Optimization](https://graphsignal.com/docs/guides/ai-optimization/) for setup instructions.


## Overhead

The profiler has minimal impact on production performance. CUDA kernel activity is collected via CUPTI with low-overhead APIs, and analysis and upload happen in the sidecar process.


## Security and Privacy

The profiler only establishes outbound connections to `api.graphsignal.com` to send data; inbound connections or commands are not possible.

Content and sensitive information, such as prompts and completions, are not recorded.


## Troubleshooting

If something doesn't look right, report it to our support team via your account.

In case of connection issues, please make sure outgoing connections to `https://api.graphsignal.com` are allowed.

