Metadata-Version: 2.4
Name: perfall-cli
Version: 0.1.3
Summary: Standalone CLI for uploading benchmark artifacts to perfall.it
Author: Perfall
License: LicenseRef-Proprietary
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Environment :: Console
Requires-Python: >=3.9
Description-Content-Type: text/x-rst
Provides-Extra: testing
Requires-Dist: pytest; extra == "testing"
Requires-Dist: ruff<0.16,>=0.15.8; extra == "testing"

perfall-cli
===========

``perfall-cli`` is the standalone client package for uploading benchmark
artifacts to `Perfall <https://perfall.it>`_. It is a lightweight client
that can be installed independently of the server.

Install from PyPI::

    pip install perfall-cli

Supported commands currently live under ``perfall ingest`` and cover:

- ``pytest-benchmark``
- ``go-test``
- ``google-benchmark``
- ``jmh``
- ``criterion``
- ``benchmarkdotnet``
- ``vitest``

Regression gating is available on every ingest adapter via the shared flags:

::

    perfall ingest pytest-benchmark \
      --file benchmark.json \
      --api-key "$PERFALL_API_KEY" \
      --testbed "github-actions-ubuntu-24.04-python-3.12" \
      --baseline-branch master \
      --baseline-window 5 \
      --fail-on-regression-percent 5

That upload still publishes the run, then compares each metric against the
recent compatible baseline-branch samples (five by default), aggregates them with a
median, and exits non-zero when the threshold is exceeded. Latency and cost metrics are lower-is-better; normalized throughput
metrics such as ``ops/s``, ``B/s``, and ``items/s`` are higher-is-better, so a
throughput drop past the threshold fails the gate. A metric warms up without failing
until the selected baseline window is available; pass ``--baseline-window 1`` to
explicitly compare only the latest sample.

Use ``--testbed`` for CI gates to keep runner or container changes from comparing
unlike results. Pick a stable explicit value such as
``github-actions-ubuntu-24.04-python-3.12``; a changed testbed warms up without
failing until it has enough history. Omitting ``--testbed`` preserves the legacy
unscoped baseline behavior for existing workflows.

For example, gate a JMH throughput benchmark exactly as you would a latency
benchmark::

    perfall ingest jmh --file jmh-results.json --api-key "$PERFALL_API_KEY" \
      --baseline-branch master --fail-on-regression-percent 5

Local development::

    python -m venv .venv
    source .venv/bin/activate
    pip install --upgrade pip
    pip install -e .[testing]
    pytest
