Metadata-Version: 2.4
Name: benchmarkingcli
Version: 0.0.2
Summary: CLI tool for running memory latency benchmarks on Android devices
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: typer>=0.9.0

# BenchmarkingCLI

A command-line tool for running latency benchmarks on Android devices. This tool automates the process of pushing benchmark binaries to connected Android devices, executing latency tests, and collecting performance metrics in CSV format.

## Features

- **Latency Testing**: Measure memory latency across different buffer sizes on a specific core
- **Loaded Latency Testing**: Measure latency under load while bandwidth threads are concurrently running
- **Kernel Mode Selection**: Choose between SUM, COPY, UPDATE, and TRIAD bandwidth kernels
- **Automated Device Management**: Automatic ADB device detection and binary deployment
- **CSV Export**: Results are automatically parsed and exported to timestamped CSV files
- **CLI Interface**: Easy-to-use command-line interface with Typer

## Requirements

### System Requirements

- **Android Debug Bridge (ADB)**: Version 1.0.40+
  - Install via: `apt-get install android-tools-adb` (Linux) or `brew install android-platform-tools` (macOS)
  - Verify installation: `adb version`
  - Ensure your Android device has USB debugging enabled
  - Device must be connected and appear in `adb devices`

### Python Requirements

- **Python**: 3.8 or higher
- **Dependencies**:
  - `typer>=0.9.0` - CLI framework

## Installation

1. **Clone the repository**:
   ```bash
   git clone https://git.truminds.co.in/truminds/performance-benchmarking/benchmarkingcli.git
   cd benchmarkingcli
   ```

2. **Install Python dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

3. **Verify ADB is installed and connected**:
   ```bash
   adb devices
   ```
   You should see your connected device listed.

4. **Place the benchmark binary**:
   - Copy the `latency_test` binary to the `helper/` directory
   - The script will automatically push it to `/data/local/tmp/` on the device

## Usage

### Latency Test

Run a basic latency benchmark on a specific core:

```bash
# Run on default core (7)
python main.py latency

# Run on a specific core
python main.py latency --core 4
```

This command will:
1. Check for connected ADB devices
2. Push the benchmark binary to the device
3. Execute the latency test on the specified core
4. Parse results and save to `latency_results_<TIMESTAMP>.csv`

### Loaded Latency Test

Run a latency test with concurrent bandwidth load:

```bash
# Default — TRIAD kernel, core 7 for latency, cores 0-5 for bandwidth
python main.py loadedlatency \
  --lat-buffer-mb 128 \
  --bw-buffer-mb 128 \
  --latency-core 7 \
  --bw-cores 0,1,2,3,4,5 \
  --kernel-mode TRIAD

# SUM kernel (lightest — 1 array, read only)
python main.py loadedlatency \
  --lat-buffer-mb 128 \
  --bw-buffer-mb 128 \
  --latency-core 7 \
  --bw-cores 0,1,2,3,4,5 \
  --kernel-mode SUM

# COPY kernel (2 arrays — read + write)
python main.py loadedlatency \
  --lat-buffer-mb 128 \
  --bw-buffer-mb 128 \
  --latency-core 7 \
  --bw-cores 0,1,2,3 \
  --kernel-mode COPY

# UPDATE kernel (2 arrays — read + write with scalar multiply)
python main.py loadedlatency \
  --lat-buffer-mb 256 \
  --bw-buffer-mb 256 \
  --latency-core 7 \
  --bw-cores 0,1,2,3,4,5 \
  --kernel-mode UPDATE

# Numeric mode values also accepted (1=SUM, 2=COPY, 3=UPDATE, 4=TRIAD)
python main.py loadedlatency \
  --lat-buffer-mb 128 \
  --bw-buffer-mb 128 \
  --latency-core 7 \
  --bw-cores 0,1,2,3,4,5 \
  --kernel-mode 4
```

**Parameters**:
- `--lat-buffer-mb`: Buffer size (MB) for the latency thread (default: 128)
- `--bw-buffer-mb`: Buffer size (MB) for each bandwidth thread (default: 128)
- `--latency-core`: CPU core to run the latency thread on (default: 7)
- `--bw-cores`: Comma-separated list of cores for bandwidth threads (default: 0,1,2,3,4,5)
- `--kernel-mode`: Bandwidth kernel to use — see table below (default: TRIAD)

### Kernel Modes

| Mode   | Value | Arrays | Description                              |
|--------|-------|--------|------------------------------------------|
| SUM    | 1     | 1      | Read-only sum — lightest BW pressure     |
| COPY   | 2     | 2      | Read + write — moderate BW pressure      |
| UPDATE | 3     | 2      | Read + scalar write — moderate BW pressure |
| TRIAD  | 4     | 3      | Read 2 + write 1 — heaviest BW pressure  |

Both the name (`TRIAD`) and numeric value (`4`) are accepted interchangeably.

## Output

Results are exported as CSV files with the following columns:
- `buffer_size_kb`: Size of the buffer tested (in KB)
- `min_ns`: Minimum latency (nanoseconds)
- `mean_ns`: Mean latency (nanoseconds)
- `max_ns`: Maximum latency (nanoseconds)

Example output files:
- `latency_results_20260427_115540.csv`
- `loaded_latency_results_20260427_115540.csv`

## Troubleshooting

### "No ADB devices found"
- Check that your Android device is connected: `adb devices`
- Ensure USB debugging is enabled on the device
- Try: `adb kill-server` then `adb start-server`

### "Latency test binary not found"
- Ensure the `latency_test` binary is placed in the `helper/` directory
- Verify the file exists: `ls -la helper/latency_test`

### Binary push fails
- Check device storage: `adb shell df /data/local/tmp`
- Verify ADB permissions: `adb shell ls -la /data/local/tmp`

### "Invalid kernel mode"
- Use one of: `SUM`, `COPY`, `UPDATE`, `TRIAD` or their numeric equivalents `1`, `2`, `3`, `4`

## Project Structure

```
benchmarkingcli/
├── main.py                              # CLI entry point
├── latency.py                           # Latency test runner
├── loaded_latency.py                    # Loaded latency test runner
├── parse_latency_result.py              # Result parsing and CSV export
├── requirements.txt                     # Python dependencies
├── helper/
│   ├── binary_helper.py                 # ADB and binary management
│   └── latency_test                     # Compiled benchmark binary
└── README.md                            # This file
```

## Support

For issues, questions, or contributions, please contact the development team or submit an issue through the project repository.
