Metadata-Version: 2.4
Name: benchmarkingcli
Version: 0.4.1
Summary: Android SoC benchmarking CLI for latency, loaded latency, and memory subsystem characterization
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: typer>=0.9.0
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Requires-Dist: pytest-cov>=4.0; extra == "test"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"

# BenchmarkingCLI

A command-line tool for running latency benchmarks on Android devices. This tool automates the process of pushing benchmark binaries to connected Android devices, executing latency tests, and collecting performance metrics in CSV format.

## Features

- **Latency Testing**: Measure memory latency across different buffer sizes on a specific core
- **Loaded Latency Testing** (v0.3+): Measure latency under concurrent bandwidth loads with configurable kernel modes and thread settings
- **GPU Loaded Latency Testing** (v0.4+): Measure memory latency under concurrent GPU load
- **CSV Export**: Results are automatically parsed and exported to timestamped CSV files
- **CLI Interface**: Easy-to-use command-line interface

## Requirements

### Host Machine

- **Python**: 3.8 or higher
- **Android Debug Bridge (ADB)**: Version 1.0.40+
    - **Linux**: `apt-get install android-tools-adb`
    - **macOS**: `brew install android-platform-tools`
    - **Windows**: Download the [SDK Platform Tools](https://developer.android.com/tools/releases/platform-tools) and add to PATH
    - Verify installation: `adb version`

### Android Device

- USB debugging must be enabled on the device. To enable it:
    1. Go to **Settings → About Phone**
    2. Tap **Build Number** 7 times until you see "You are now a developer"
    3. Go to **Settings → Developer Options**
    4. Enable **USB Debugging**
- Connect the device via USB and accept the "Allow USB Debugging" prompt on the device
- Confirm the device is detected: `adb devices`

## Installation

Install directly from PyPI:

```bash
pip install benchmarkingcli
```



## Usage

### Latency Test

Run a basic latency benchmark on a specific core:

```bash
# Run on default core (7)
benchmarkingcli latency

# Run on a specific core
benchmarkingcli latency --core 4
```

**Parameters**:
- `--core`: CPU core to run the latency test on (default: 7)
- `--output-dir`: Directory to save results (default: current directory)

## Output

Results are exported as CSV files with the following columns:

- `buffer_size_kb`: Size of the buffer tested (in KB)
- `min_ns`: Minimum latency (nanoseconds)
- `mean_ns`: Mean latency (nanoseconds)
- `max_ns`: Maximum latency (nanoseconds)

Example output file: `latency_results_20260427_115540.csv`

### Loaded Latency Test (v0.3+)

Run a latency benchmark under concurrent bandwidth load:

```bash
# Run with default settings
benchmarkingcli loadedlatency

# Run with custom configuration
benchmarkingcli loadedlatency --lat-buffer-mb 128 --bw-buffer-mb 128 --latency-core 7 --bw-cores 0,1,2,3 --kernel-mode TRIAD --ilp 4 --injection-rate desc --output-dir ./results
```

**Parameters**:
- `--lat-buffer-mb`: Latency buffer size in MB (default: 128)
- `--bw-buffer-mb`: Bandwidth buffer size in MB (default: 128)
- `--latency-core`: Core to run latency thread on (default: 7)
- `--bw-cores`: Comma-separated cores for bandwidth threads (default: 0,1,2,3,4,5)
- `--kernel-mode`: Kernel mode: SUM/COPY/UPDATE/TRIAD or 1/2/3/4 (default: TRIAD)
- `--ilp`: Unroll factor: 1/2/4/8/16/32 (default: 4)
- `--injection-rate`: Injection rate: asc or desc (default: asc)
- `--output-dir`: Directory to save results (default: current directory)

> **Important Update (v0.4+)**: The `--tile-delay-sweep-order` parameter has been renamed to `--injection-rate`. Supported values are `asc` and `desc`.

### GPU Loaded Latency Test (v0.4+)

Run a latency benchmark under concurrent GPU load:

```bash
# Run with default settings
benchmarkingcli gpuloadedlatency

# Run with custom configuration
benchmarkingcli gpuloadedlatency --latency-core 7 --lat-buffer-mb 128 --gpu-buffer-mb 128 --kernel-mode TRIAD --output-dir ./results
```

**Parameters**:

* `--latency-core`: Core to run latency thread on (default: 7)
* `--lat-buffer-mb`: Latency buffer size in MB (default: 128)
* `--gpu-buffer-mb`: GPU workload buffer size in MB (default: 128)
* `--kernel-mode`: Kernel mode: SUM/COPY/UPDATE/TRIAD or 1/2/3/4 (default: TRIAD)
* `--output-dir`: Directory to save results (default: current directory)


## Troubleshooting

### "No ADB devices found"
- Check that your Android device is connected: `adb devices`
- Ensure USB debugging is enabled on the device
- Try: `adb kill-server` then `adb start-server`

### "Latency test binary not found"
- Reinstall the package: `pip install --force-reinstall benchmarkingcli`

### Binary push fails
- Check device storage: `adb shell df /data/local/tmp`
- Verify ADB permissions: `adb shell ls -la /data/local/tmp`
 
