---
name: modern-python
description: Apply modern Python engineering hygiene to research code — ruff for linting+formatting, pytest for tests, type hints, clean project/import structure, and reproducible environments. Use when writing new Python modules, refactoring research scripts into reusable code, or cleaning up a "pile of scripts" so the factor lab stays maintainable.
---

# Modern Python hygiene

A lightweight engineering standard for keeping research code (which tends to sprawl into one-off scripts) clean, tested, and reproducible. Tools installed in this environment: **ruff** 0.15 (lint + format), **pytest** 9.0 (tests). Run them via `python -m ruff` / `python -m pytest` (the bare CLIs may not be on PATH).

> Note: the original Trail of Bits skill installs a `uv` toolchain and a SessionStart hook that redirects `python`/`pip` to `uv run`. That hook is intentionally **omitted here** to avoid changing this machine's PATH/conda behavior. This skill uses the existing conda Python + pip (Tsinghua mirror). If you ever want `uv`, install it deliberately, don't auto-redirect.

## The loop: format → lint → type → test

```bash
PY=/intern9/huhongkai/miniconda3/bin/python
$PY -m ruff format .          # auto-format (Black-compatible)
$PY -m ruff check . --fix     # lint + autofix imports, unused vars, etc.
$PY -m pytest -q              # run tests
```

## ruff config (drop in pyproject.toml)

```toml
[tool.ruff]
line-length = 100
target-version = "py313"

[tool.ruff.lint]
# E,F=pyflakes/pycodestyle  I=isort  UP=pyupgrade  B=bugbear  SIM=simplify  PD=pandas-vet
select = ["E", "F", "I", "UP", "B", "SIM", "PD", "NPY"]
ignore = ["E501"]   # line length handled by formatter

[tool.ruff.lint.isort]
known-first-party = ["hs300_factor_lab"]
```
`PD` (pandas-vet) and `NPY` (NumPy) rule sets are especially valuable here — they flag
chained-assignment, `inplace=True`, deprecated numpy aliases, etc.

## Principles for research code

1. **Pure functions for factor logic.** A factor is `f(panel_df) -> Series/df`. No global state, no I/O inside the computation — makes it testable (pair with [[property-based-testing]]) and cacheable.
2. **Separate I/O from compute.** Load → compute → persist as distinct layers; don't read parquet inside a factor function.
3. **Type hints on public functions.** `def zscore(x: pd.Series) -> pd.Series:` — documents intent and catches shape mistakes early.
4. **No magic numbers.** Lookback windows, thresholds → named constants or function args, not literals buried in code.
5. **Fail loud on bad data.** Assert shapes/index alignment instead of letting pandas silently broadcast or NaN-propagate.
6. **One module = one concern.** factors/, backtest/, data/, reports/ — not a 2000-line `utils.py`.

## Reproducible environment

This project uses conda + Tsinghua mirror (GitHub is unreachable from this host — see project memory). To freeze deps:
```bash
$PY -m pip freeze > requirements.txt
# or conda env export --no-builds > environment.yml
```
Pin versions for anything that affects numerical results (pandas, numpy).

## pytest layout

```
tests/
  test_factors.py      # example + property tests for each factor
  conftest.py          # shared fixtures: small synthetic panels
```
```bash
$PY -m pytest -q                    # all
$PY -m pytest tests/test_factors.py::test_zscore -x   # one, stop on first fail
$PY -m pytest -k "rank and not slow"
```

## Gotchas

- Run ruff/pytest through `python -m` since the standalone CLIs aren't on PATH in this conda env.
- `ruff check --fix` is safe for imports/unused, but review `--unsafe-fixes` diffs.
- pandas 3.0 is installed — Copy-on-Write is the default; chained assignment that used to "work" may now no-op. ruff's `PD` rules help catch this.
