CI/CD Integration

Automate your load tests and gate your pipelines with performance assertions.

Overload is built to be a first-class citizen in CI/CD pipelines. It uses the universal signal for failure (exit code 1) when assertions fail, making it natively compatible with GitHub Actions, GitLab CI, Jenkins, and others. It can also output JUnit XML for native test reporting.

Assertions

Assertions allow you to define performance thresholds. If any threshold is breached, the test is marked as failed.

overload run --collection api.json --pattern load \
  --assert "p95_latency_ms<500" \
  --assert "error_rate_pct==0"

Available Metrics:

Available Operators: <, <=, >, >=, ==

JUnit XML Output

Most CI systems can parse JUnit XML to display test results natively in their UI. Use the --junit flag to generate this file.

overload run --collection api.json --assert "p95_latency_ms<500" --junit results.xml

Using a Config File

Instead of passing long commands, you can store your configuration and assertions in an overload.config.yaml file. You can generate this file from the Browser UI using the Save Config button.

test_type: load
config:
  target_rps: 100
  hold_duration_seconds: 60
  concurrency: 20
thresholds:
  - metric: p95_latency_ms
    operator: "<"
    value: 500
  - metric: error_rate_pct
    operator: "<"
    value: 1

Run the test using the config file:

overload run --collection api.json --config overload.config.yaml --junit results.xml

GitHub Actions Example

This workflow runs the load test on every push, uploads the HTML report as an artifact, and publishes the JUnit results to the GitHub UI.

name: Performance Tests

on: [push, pull_request]

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install Overload
        run: pip install overload-cli

      - name: Run Load Test
        run: |
          overload run \
            --collection tests/postman_collection.json \
            --config tests/overload.config.yaml \
            --output reports \
            --junit results.xml

      - name: Upload HTML Report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: overload-report
          path: reports/
          retention-days: 7

      - name: Publish Test Results
        uses: dorny/test-reporter@v1
        if: always()
        with:
          name: Overload Assertions
          path: results.xml
          reporter: java-junit

GitLab CI Example

GitLab CI has native support for JUnit reports. The results will appear in the Merge Request widget.

load_test:
  stage: test
  image: python:3.12
  script:
    - pip install overload-cli
    - overload run --collection api.json --config overload.config.yaml --junit results.xml
  artifacts:
    when: always
    reports:
      junit: results.xml
    paths:
      - reports/

Jenkins Example

Using a Jenkinsfile to run Overload and publish the JUnit XML.

pipeline {
    agent any
    stages {
        stage('Load Test') {
            steps {
                sh 'pip install overload-cli'
                sh 'overload run --collection api.json --config overload.config.yaml --junit results.xml'
            }
        }
    }
    post {
        always {
            junit 'results.xml'
            archiveArtifacts artifacts: 'reports/*.html', allowEmptyArchive: true
        }
    }
}

Best Practices

• Store config in git: Keep your overload.config.yaml in version control so thresholds evolve alongside your code.
• Use conservative thresholds: CI environments can be noisy. A P95 threshold of 100ms might be too flaky; 500ms might be a safer CI gate.
• Secret injection: Don't commit API tokens. Use --var api_key=$SECRETS_KEY in your CI runner to inject auth securely at runtime.