Metadata-Version: 2.4
Name: pytest-cream
Version: 0.1.3
Summary: The cream of test execution - smooth pytest workflows with intelligent orchestration
Author: Sermet Pekin
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Framework :: Pytest
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: pytest
Requires-Dist: pytest-xdist
Requires-Dist: requests
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: black>=23.3.0; extra == "dev"
Requires-Dist: flake8>=3.9.2; extra == "dev"
Requires-Dist: mypy>=1.4.1; extra == "dev"
Requires-Dist: bandit>=1.7.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"

# pytest-cream

**The cream of test execution** - smooth pytest workflows with intelligent orchestration.

pytest-cream is a Python package designed to streamline the process of fetching tests from repositories, running them with advanced dependency management, and gathering statistics for parallel execution.

## Features
- Fetch tests from remote repositories or local directories
- Advanced dependency management with custom installation commands
- **Uses uv by default for faster and more reliable installations**
- Support for uv, pip, poetry, and other package managers
- Python version control for project compatibility
- Parallel test execution with intelligent orchestration
- Duration statistics and performance optimization
- Environment isolation for complex projects (Rust extensions, etc.)

## Installation

```bash
pip install pytest-cream
```

## Usage

### Quick Start (Automated Setup)
The easiest way to get started is with the `init` command, which fetches tests and installs dependencies in one step:

```bash
# Initialize: fetch tests and install dependencies from the same repository
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project --install-mode editable \
  --ws ~/workspace

# Then profile tests to collect duration statistics
pytest-cream profile --ws ~/workspace/owner_project_main --python python3.11

# Finally run tests with intelligent orchestration
pytest-cream run --ws ~/workspace/owner_project_main --python python3.11 --threshold 0.5
```

### Step-by-Step Workflow (Manual Install)
For projects requiring custom installation steps, handle dependencies manually:

```bash
# Step 1: Fetch tests from repository
pytest-cream fetch --repo owner/project --branch main --ws ~/my-workspace

# Step 2: Install dependencies manually
cd ~/my-workspace/owner_project_main
pip install -e .  # or your preferred installation method

# Step 3: Profile tests - run all tests once to collect duration statistics
pytest-cream profile --ws ~/my-workspace/owner_project_main --python python3.11

# Step 4: Run tests with intelligent orchestration
pytest-cream run --ws ~/my-workspace/owner_project_main --python python3.11 --threshold 0.5

# (Optional) Step 5: Filter tests by duration threshold for analysis
pytest-cream filter --ws ~/my-workspace/owner_project_main --threshold 0.5

# All files (tests, logs, results) are stored in the same workspace for easy access
```

### Basic Usage Examples
```bash
# Initialize with automatic dependency installation (uses uv by default)
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project --install-mode editable \
  --ws ~/workspace

# Then profile and run tests
pytest-cream profile --ws ~/workspace/owner_project_main --python python3.11
pytest-cream run --ws ~/workspace/owner_project_main --python python3.11

# Use pip instead of uv (if needed for compatibility)
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project \
  --install-mode editable \
  --use-pip \
  --ws ~/workspace

# Use custom installation commands for complex projects
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project \
  --install-cmd "uv sync --extra cpu" \
  --ws ~/workspace
```

### Advanced Usage Examples
```bash
# Multiple installation commands with fallback
pytest-cream init --repo owner/complex-project --branch main \
  --install-repo owner/complex-project \
  --install-cmd "uv sync --extra gpu; pip install torch" \
  --install-fallback \
  --ws ~/workspace

# Then profile and run
pytest-cream profile --ws ~/workspace/owner_complex-project_main --python python3.11
pytest-cream run --ws ~/workspace/owner_complex-project_main --python python3.11 \
  --exclude-tests "test_slow.py,test_integration/"
```

## Command Line Options

### Core Commands

#### `init` - Initialize Workspace
Fetch tests from a repository and optionally install dependencies. This is the first step in the workflow.

```bash
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project --install-mode editable \
  --ws ~/workspace
```

**Note:** After running `init`, you need to run `profile` and then `run` to actually execute tests.

#### `fetch` - Fetch Tests
Clone a repository to a workspace for testing.

```bash
pytest-cream fetch --repo owner/project --branch main --ws ~/workspace
```

**Required Parameters:**
- `--repo`: Repository to fetch (format: 'owner/repo' or GitHub URL)
- `--ws` / `--workspace`: Directory where the repository will be cloned

**Optional Parameters:**
- `--branch`: Git branch to checkout (default: main)

**Output:** Creates a directory in workspace named `owner_project_branch/` containing the cloned repository.

#### `profile` - Profile Tests
Run all tests once to collect duration statistics. This is essential before using the `run` command.

```bash
pytest-cream profile --ws ~/workspace/owner_project_main --python python3.11
```

**Required Parameters:**
- `--ws` / `--workspace`: Workspace directory containing the tests (from fetch command)

**Optional Parameters:**
- `--python`: Python executable to use (e.g., `python3.11`)
- `--tests-dir`: Subdirectory containing tests (default: tests)
- `--output`: Output filename for duration log (default: test_durations.log)

**Output:** Creates `test_durations.log` in the workspace directory.

#### `filter` - Filter Tests by Duration
Parse duration logs and categorize tests as long/short.

```bash
pytest-cream filter --ws ~/workspace/owner_project_main --threshold 0.5
```

**Required Parameters:**
- `--ws` / `--workspace`: Workspace directory containing duration logs
- `--threshold`: Duration threshold in seconds

**Optional Parameters:**
- `--input`: Input duration log filename (default: test_durations.log)
- `--output`: Output filename for filtered results (default: filtered_tests.txt)

**Output:** Creates `filtered_tests.txt` in the workspace directory.

#### `run` - Run Tests with Orchestration
Execute tests with intelligent orchestration based on duration analysis. Long tests run sequentially, short tests run in parallel.

```bash
pytest-cream run --ws ~/workspace/owner_project_main --python python3.11 --threshold 0.5
```

**Required Parameters:**
- `--ws` / `--workspace`: Workspace directory containing the tests and duration logs

**Optional Parameters:**
- `--python`: Python executable to use (e.g., `python3.11`)
- `--durations`: Duration log file (default: test_durations.log)
- `--threshold`: Duration threshold in seconds for splitting long/short tests (default: 0.5)
- `--workers`: Number of parallel workers for short tests (default: 4)
- `--limit`: Maximum number of tests to run
- `--exclude-tests`: Comma-separated patterns of test files/directories to exclude

**Output:** Runs tests with intelligent orchestration and displays results.

### `init` Options

For the workspace initialization command, these options are available:

**Basic Options:**
- `--repo`: Repository to fetch (owner/repo or URL) - **Required**
- `--branch`: Branch to checkout (default: main)
- `--ws` / `--workspace`: Directory to use as workspace - **Required**

**Installation Options:**
- `--install-repo`: Repository to install dependencies from (same as `--repo` typically)
- `--install-branch`: Branch to install from (default: same as `--branch`)
- `--install-mode`: How to install (`pip`, `editable`, `wheel`) - default: pip
- `--use-pip`: Use pip instead of uv (by default, uv is used for faster installations)
- `--install-cmd`: Custom install command (overrides `--install-mode`)
- `--install-fallback`: Fallback to standard installation if custom command fails
- `--python`: Python executable for installation (e.g., `python3.11`)

## Workspace Organization

All commands use a shared workspace for easy file management:

```
~/workspace/
└── owner_project_main/           # Created by fetch command
    ├── src/                       # Your project files
    ├── tests/                     # Test files
    ├── test_durations.log         # Created by run command
    └── filtered_tests.txt         # Created by filter command
```

This organization makes it easy to:
- Find all test-related files in one place
- Chain commands together using the same workspace
- Review logs and results after test execution

## Installation Modes

When using `--install-repo` in init, you have several options. **By default, uv is used for faster and more reliable installations.**

### Standard Modes:
- `pip`: Direct install from git+https URL (fastest)
- `editable`: Clone repository and install in editable mode (for development)
- `wheel`: Build wheel and install (most reproducible)

All standard modes use uv by default. To use pip instead, add `--use-pip`.

### Using pip Instead of uv:
If you encounter compatibility issues with uv, use `--use-pip` to fallback to pip:

```bash
# Standard install with pip (slower but maximum compatibility)
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project \
  --install-mode pip \
  --use-pip \
  --ws ~/workspace

# Editable install with pip
pytest-cream init --repo owner/project --branch main \
  --install-repo owner/project \
  --install-mode editable \
  --use-pip \
  --ws ~/workspace
```

### Custom Commands:
Use `--install-cmd` for complex dependency management that standard modes can't handle:

```bash
# uv with extras
--install-cmd "uv sync --extra cpu"

# Poetry with multiple extras
--install-cmd "poetry install --extras dev,test"

# pip with specific requirements file
--install-cmd "pip install -r requirements-dev.txt"

# Conda environment setup
--install-cmd "conda env update -f environment.yml"
```

**Note**: `--install-cmd` overrides `--install-mode` when provided. The command runs in the cloned repository directory.

### Usage Priority:
1. If `--install-cmd` is provided → Use custom command
2. If `--use-pip` is provided → Use pip instead of uv
3. Otherwise → Use uv with standard `--install-mode` (pip/editable/wheel)

**Note**: uv is used by default for faster installations. Install uv with: `pip install uv` or `curl -LsSf https://astral.sh/uv/install.sh | sh`

## Troubleshooting

### Python Version Issues
Some projects require specific Python versions, especially those with Rust extensions or compiled components:

```bash
# Use Python 3.10 for projects with Rust extensions
pytest-cream init \
  --repo owner/rust-project \
  --python python3.10 \
  --ws ~/workspace

# Use Python 3.11 for newer projects
pytest-cream init \
  --repo owner/modern-project \
  --python python3.11 \
  --ws ~/workspace
```

**Common Python version issues:**
- Rust extensions often require Python 3.10 or specific versions
- Some packages may not be compatible with Python 3.13+
- Use `--python` to specify the exact Python executable to use

### Excluding Problematic Tests
When some tests fail due to missing dependencies or environment issues, exclude them using the `run` command:

```bash
# Exclude specific test files during test execution
pytest-cream run --ws ~/workspace/owner_project_main \
  --exclude-tests "test_integration.py,test_slow.py" \
  --python python3.11

# Exclude test directories
pytest-cream run --ws ~/workspace/owner_project_main \
  --exclude-tests "tests/integration,tests/gpu" \
  --python python3.11
```

### Pip Issues
If you encounter "No module named pip" errors or issues with uv, try using the `--use-pip` flag to use pip instead of uv:

```bash
pytest-cream init --repo owner/repo --install-repo owner/repo \
  --use-pip --install-mode editable --python python3.11 --ws ~/workspace
```

By default, pytest-cream uses [uv](https://github.com/astral-sh/uv) for faster and more reliable installations.

### Custom Command Issues
If your `--install-cmd` fails:

1. **Check the command works manually**:
   ```bash
   git clone https://github.com/owner/repo
   cd repo
   uv sync --extra cpu  # or your command
   ```

2. **Common fixes**:
   - For uv: Ensure `pyproject.toml` has the required extras defined
   - For Poetry: Make sure `pyproject.toml` exists and has proper dependencies
   - For build errors: Check if system dependencies (like Rust, C compilers) are needed

3. **Debugging**: Check the workspace directory for logs and cloned repository to investigate build issues

## Complete Workflow Examples

### Example 1: Step-by-Step Testing
Execute each phase separately for maximum control:

```bash
# Set up variables for easier reuse
WORKSPACE=~/my-test-workspace
REPO=owner/my-project
BRANCH=main

# Step 1: Initialize - fetch the repository and install dependencies
pytest-cream init --repo $REPO --branch $BRANCH \
  --install-repo $REPO --install-mode editable \
  --ws $WORKSPACE

# Step 2: Profile - collect test duration statistics
pytest-cream profile --ws $WORKSPACE/owner_my-project_$BRANCH --python python3.11

# Step 3: Run - execute tests with intelligent orchestration
pytest-cream run --ws $WORKSPACE/owner_my-project_$BRANCH \
  --python python3.11 --threshold 0.5

# Step 4: (Optional) Filter - analyze test durations
pytest-cream filter --ws $WORKSPACE/owner_my-project_$BRANCH --threshold 0.5

# Step 5: Review results
echo "Tests are in: $WORKSPACE/owner_my-project_$BRANCH"
cat $WORKSPACE/owner_my-project_$BRANCH/filtered_tests.txt
```

### Example 2: Quick Testing
For rapid testing with automated setup:

```bash
# Initialize workspace
pytest-cream init \
  --repo owner/project \
  --branch develop \
  --install-repo owner/project \
  --install-mode editable \
  --ws ~/quick-workspace

# Profile tests
pytest-cream profile \
  --ws ~/quick-workspace/owner_project_develop \
  --python python3.11

# Run with custom settings
pytest-cream run \
  --ws ~/quick-workspace/owner_project_develop \
  --python python3.11 \
  --threshold 0.3 \
  --workers 8
```

### Example 3: Testing with Custom Dependencies
When your project needs special installation steps:

```bash
# Initialize with custom install command
pytest-cream init \
  --repo owner/ml-project \
  --branch main \
  --install-repo owner/ml-project \
  --install-cmd "uv sync --extra gpu,training" \
  --install-fallback \
  --ws ~/ml-workspace

# Profile tests
pytest-cream profile \
  --ws ~/ml-workspace/owner_ml-project_main \
  --python python3.10

# Run tests with exclusions
pytest-cream run \
  --ws ~/ml-workspace/owner_ml-project_main \
  --python python3.10 \
  --exclude-tests "test_slow_training.py,tests/integration/"
```

### Example 4: Development Testing
Test changes in a development branch with editable install:

```bash
# Initialize with editable install
pytest-cream init \
  --repo owner/my-library \
  --branch feature-branch \
  --install-repo owner/my-library \
  --install-branch feature-branch \
  --install-mode editable \
  --ws ~/dev-workspace

# Profile and run
pytest-cream profile --ws ~/dev-workspace/owner_my-library_feature-branch --python python3.11
pytest-cream run --ws ~/dev-workspace/owner_my-library_feature-branch --python python3.11
```

### Example 5: CI/CD Pipeline
Suitable for automated testing environments:

```bash
#!/bin/bash
set -e  # Exit on error

# Initialize workspace
pytest-cream init \
  --repo $CI_REPO \
  --branch $CI_BRANCH \
  --install-repo $CI_REPO \
  --install-mode pip \
  --ws /tmp/ci-workspace-$CI_JOB_ID

WORKSPACE="/tmp/ci-workspace-$CI_JOB_ID/$(echo $CI_REPO | tr '/' '_')_$CI_BRANCH"

# Profile tests
pytest-cream profile --ws $WORKSPACE --python python3.11

# Run tests and capture results
pytest-cream run \
  --ws $WORKSPACE \
  --python python3.11 \
  --exclude-tests "tests/manual/,test_interactive.py"

# Check if tests passed
if [ $? -eq 0 ]; then
  echo "✅ All tests passed!"
  exit 0
else
  echo "❌ Tests failed!"
  exit 1
fi
```

## Tips and Best Practices

1. **Always specify `--python`**: Avoid "python command not found" errors by explicitly setting the Python executable (e.g., `--python python3.11`)

2. **Use meaningful workspace names**: Organize workspaces by project or purpose (e.g., `~/workspaces/myproject-dev`, `~/workspaces/myproject-prod`)

3. **Keep workspaces outside project directories**: Store workspaces in a separate location (e.g., `~/pytest-cream-workspaces/`) to avoid pytest collection issues. **Never create workspaces inside your pytest-cream installation directory.**

4. **Reuse workspaces for iteration**: When testing the same project multiple times, you can reuse the workspace or let pytest-cream create timestamped directories

5. **Check workspace contents**: After running commands, inspect the workspace directory to review logs, test results, and any generated files

6. **Exclude flaky tests**: Use `--exclude-tests` with the `run` command to skip tests that are environment-dependent or have external dependencies

7. **Use `init` for setup**: The `init` command handles fetching and installing, then use `profile` and `run` for test execution

8. **Clean up old workspaces**: Periodically remove old workspace directories to free up disk space

9. **Profile before running**: Always run `profile` before `run` to collect test duration statistics for intelligent orchestration

## License
MIT License
