# Coding Conventions & Patterns

**Project**: opendeviationbar-py | **Date**: 2026-03-19 | **Scope**: Rust crates + Python layer

---

## Rust Conventions

### Formatting & Style

**Tool**: `rustfmt` (v1.8.0, pinned via `rust-toolchain.toml`)
**Config**: `rustfmt.toml`

```toml
# Key settings
edition = "2024"
max_width = 100
hard_tabs = false
tab_spaces = 4

# Function layout
fn_params_layout = "Tall"

# Line wrapping
array_width = 60
chain_width = 60
single_line_if_else_max_width = 50
```

**File**: `rustfmt.toml`

**Enforcement**: `mise run fmt` checks formatting; CI blocks violations.

---

### Naming Conventions

| Category         | Pattern                     | Example                                                            |
| ---------------- | --------------------------- | ------------------------------------------------------------------ |
| **Structs**      | PascalCase                  | `OpenDeviationBar`, `StreamManager`                                |
| **Enums**        | PascalCase                  | `DataSource`, `ProcessingError`                                    |
| **Functions**    | snake_case                  | `compute_microstructure_features()`, `process_agg_trade_records()` |
| **Constants**    | SCREAMING_SNAKE_CASE        | `GLOBAL_ENTROPY_CACHE_CAPACITY`, `DAY_MS`                          |
| **Type aliases** | PascalCase                  | (rare in codebase)                                                 |
| **Modules**      | snake_case                  | `processor`, `interbar_cache`, `fixed_point`                       |
| **Private fns**  | snake*case with leading `*` | `_validate_timestamp()`, `_compute_kyle_lambda()`                  |

**Location**: Throughout `crates/opendeviationbar-core/src/`

---

### Error Handling

**Library**: `thiserror` (custom error types) + `anyhow` (context wrapping)

```rust
// Define custom error types
use thiserror::Error;

#[derive(Error, Debug)]
pub enum ProcessingError {
    #[error("Invalid threshold: {0}")]
    InvalidThreshold(i32),

    #[error("Timestamp out of bounds: {0}")]
    InvalidTimestamp(i64),

    #[error("Trade processing failed: {0}")]
    ProcessingFailed(String),
}

// Usage in Result
pub fn process_agg_trade_records(
    &mut self,
    trades: &[AggTrade],
) -> Result<Vec<OpenDeviationBar>, ProcessingError> {
    // ...
}

// Return early with context
if threshold_decimal_bps < MIN_THRESHOLD {
    return Err(ProcessingError::InvalidThreshold(threshold_decimal_bps));
}
```

**Files**: `crates/opendeviationbar-core/src/errors.rs` | `crates/opendeviationbar-core/src/processor.rs`

---

### Common Patterns

#### 1. Type-Safe Numeric Arithmetic (Fixed-Point)

All prices use 8-decimal fixed-point arithmetic to avoid floating-point errors:

```rust
use opendeviationbar_core::fixed_point::FixedPoint;

// Create from string (lossless)
let price = FixedPoint::from_str("42000.12345678")?;

// Convert to f64 for export
let price_f64: f64 = price.to_f64();

// Do NOT round or truncate — preserve full precision
```

**Location**: `crates/opendeviationbar-core/src/fixed_point.rs`

#### 2. Timestamp Normalization

Timestamps internally stored as `i64` microseconds (UTC):

```rust
// Timestamp normalization: accepts multiple formats (ms, s, µs)
pub fn normalize_timestamp(ts: i64) -> Result<i64, ProcessingError> {
    // Detects format and converts to microseconds
    if ts < 1_000_000_000_000 {
        // Seconds → microseconds
        Ok(ts * 1_000_000)
    } else if ts < 1_000_000_000_000_000 {
        // Milliseconds → microseconds
        Ok(ts * 1_000)
    } else {
        // Already microseconds
        Ok(ts)
    }
}

// Export to Python uses milliseconds
"open_time_ms": (bar.open_time / 1_000) as i64,
"close_time_ms": (bar.close_time / 1_000) as i64,
```

**Location**: `crates/opendeviationbar-core/src/timestamp.rs`

#### 3. State Machine in Processor

Streaming processor maintains state between calls:

```rust
pub struct OpenDeviationBarProcessor {
    threshold: FixedPoint,
    current_bar: Option<BarBuilder>,
    defer_open: bool,  // Tracks whether next trade should open new bar
    processor_checkpoint: Option<Checkpoint>,
}

impl OpenDeviationBarProcessor {
    // Stateful: processes trades incrementally
    pub fn process_single_trade(
        &mut self,
        trade: AggTrade,
    ) -> Option<OpenDeviationBar> {
        // Updates internal state, returns completed bar if breach occurs
    }

    // For checkpointing
    pub fn create_checkpoint(&self) -> Result<Checkpoint, CheckpointError> {
        // Serialize defer_open + incomplete bar state
    }
}
```

**Location**: `crates/opendeviationbar-core/src/processor.rs` (2,883 lines, split into tests + export processor)

---

### Memory Optimization

**Strategy**: Mimalloc allocator, Rayon parallelization, SmallVec for small collections

```rust
// Global allocator (Issue #147)
#[cfg(not(target_env = "msvc"))]
#[global_allocator]
static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;

// SmallVec for small collections (avoids heap allocation)
use smallvec::SmallVec;
let mut trades: SmallVec<[Trade; 32]> = SmallVec::new();

// Parallel processing
use rayon::prelude::*;
bars.par_iter()
    .map(|bar| compute_feature(bar))
    .collect::<Vec<_>>()
```

**Files**: `crates/opendeviationbar-core/src/lib.rs`, `Cargo.toml`

---

### Compiler Flags

**Location**: `.cargo/config.toml`

```toml
[build]
# Cross-compilation friendly defaults
rustflags = ["-C", "opt-level=3", "-C", "strip=symbols"]

# Native macOS ARM64 (optimized)
[target.aarch64-apple-darwin]
rustflags = ["-C", "opt-level=3", "-C", "target-cpu=native", "-C", "strip=symbols"]

# Linux x86_64 (generic, no native features)
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "opt-level=3", "-C", "strip=symbols"]
```

---

## Python Conventions

### Formatting & Linting

**Formatter**: Black (88 char line length)
**Linter**: Ruff (extensive ruleset, >150 rules enabled)
**Type Checker**: mypy (strict mode)

**Config**: `pyproject.toml`

```toml
[tool.black]
line-length = 88
target-version = ["py313"]

[tool.ruff]
line-length = 88
target-version = "py313"

[tool.mypy]
strict = true
python_version = "3.13"
disallow_untyped_defs = true
```

**File**: `pyproject.toml` lines 116-122, 120-159

---

### Naming Conventions

| Category            | Pattern                     | Example                                                    |
| ------------------- | --------------------------- | ---------------------------------------------------------- |
| **Classes**         | PascalCase                  | `OpenDeviationBar`, `TickStorage`, `StreamManager`         |
| **Functions**       | snake_case                  | `get_open_deviation_bars()`, `populate_cache_resumable()`  |
| **Constants**       | SCREAMING_SNAKE_CASE        | `TIER1_SYMBOLS`, `DAY_MS`, `TRADE_ID_RANGE_COLUMNS`        |
| **Type aliases**    | PascalCase                  | (use `TypeAlias` annotation, rare)                         |
| **Modules**         | snake_case                  | `sidecar.py`, `clickhouse/cache.py`, `validation/tier1.py` |
| **Private fns**     | snake*case with leading `*` | `_checkpoint_dir()`, `_guard_volume_non_negative()`        |
| **Private classes** | PascalCase with leading `_` | `_CheckpointMixin` (rare)                                  |

**Enforcement**: Ruff rule `N` (pep8-naming)

---

### Type Annotations (PEP 561)

**Requirement**: Strict typing for all public functions

```python
from typing import TYPE_CHECKING
from datetime import date

def get_open_deviation_bars(
    symbol: str,
    start_date: str | date,
    end_date: str | date,
    *,
    threshold_decimal_bps: int = 250,
    include_microstructure: bool = False,
) -> pd.DataFrame:
    """Fetch open deviation bars from cache.

    Args:
        symbol: Trading pair (e.g., "BTCUSDT").
        start_date: Start date (inclusive).
        end_date: End date (inclusive).
        threshold_decimal_bps: Threshold in decimal basis points.
        include_microstructure: Include 10 intra-bar features.

    Returns:
        DataFrame with OHLCV + optional microstructure.

    Raises:
        SymbolNotRegisteredError: If symbol not in registry.
        ValueError: If threshold invalid.
    """
    pass
```

**Stub Files** (PEP 561 per-module layout):

- `__init__.pyi` — Re-export index only, no type definitions
- Per-module `.pyi` files — Contain actual type definitions

**Example**:

```python
# __init__.pyi
from .clickhouse.cache import OpenDeviationBarCache as OpenDeviationBarCache
from .sidecar import SidecarConfig as SidecarConfig, run_sidecar as run_sidecar
```

**Location**: Throughout `python/opendeviationbar/`, enforced by `pyproject.toml` [tool.ruff.lint.per-file-ignores]

---

### Error Handling

#### 1. Custom Exceptions

Defined in `python/opendeviationbar/exceptions.py`:

```python
class SymbolNotRegisteredError(ValueError):
    """Raised when symbol not in registry."""
    pass

class CacheWriteError(RuntimeError):
    """Raised when ClickHouse write fails."""
    pass
```

#### 2. Retry Pattern (Stamina)

Production-grade retry using `stamina` library (replaces hand-rolled loops):

```python
import stamina

@stamina.retry(
    on=(ConnectionError, TimeoutError),
    attempts=5,
    timeout=30,
)
def _stamina_write() -> None:
    """Write batch to ClickHouse with stamina retry."""
    cache.store_bars_batch(df, overlap_strategy="off")

_stamina_write()
```

**Location**: `python/opendeviationbar/sidecar.py:2390+` (Charon dead-letter replay)

#### 3. Circuit Breaker

Module-level circuit breaker in `orchestration/open_deviation_bars_cache.py`:

```python
from stamina import BoundedCounter

fatal_write_breaker = BoundedCounter(
    failure_threshold=3,
    recovery_timeout=60,
    expected_exception=CacheWriteError,
)

# State: CLOSED → (3 failures) → OPEN → (60s) → HALF_OPEN → (success) → CLOSED
```

**Location**: `python/opendeviationbar/orchestration/open_deviation_bars_cache.py`

---

### Logging Patterns

**Library**: `loguru` for structured JSON logging (optional; fallback to stdlib logging)

```python
import logging

logger = logging.getLogger(__name__)

# INFO level for operations
logger.info("Processing %d bars for %s @ %d dbps", len(df), symbol, threshold)

# WARNING for recoverable issues
logger.warning("Symbol %s not registered, using fallback", symbol)

# ERROR for failures
logger.error("Cache write failed after %d retries", attempts, exc_info=True)

# DEBUG for diagnostics
logger.debug("Checkpoint saved: last_tid=%d, close_time_ms=%d", last_tid, close_time)
```

**NDJSON Events** (for Telegram/monitoring):

```python
from opendeviationbar.hooks import emit_hook

emit_hook(HookEvent.CACHE_WRITE_COMPLETE, {
    "symbol": "BTCUSDT",
    "threshold": 250,
    "bars_written": 1234,
    "duration_s": 2.3,
})
```

**Files**: Throughout `python/opendeviationbar/`, standard `logging.getLogger(__name__)` pattern

---

### Configuration (pydantic-settings)

**Pattern**: Centralized configuration with automatic priority: CLI > env vars > TOML > defaults

```python
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class ClickHouseConfig(BaseSettings):
    """Configuration for ClickHouse connection."""

    model_config = SettingsConfigDict(
        env_prefix="OPENDEVIATIONBAR_CH_",
        case_sensitive=False,
    )

    host: str = Field(
        default="localhost",
        description="ClickHouse host or SSH tunnel URL",
    )
    port: int = Field(
        default=8123,
        description="ClickHouse HTTP port",
    )
    user: str = Field(
        default="default",
        description="ClickHouse user",
    )
    # Environment variables: OPENDEVIATIONBAR_CH_HOST, OPENDEVIATIONBAR_CH_PORT, etc.
```

**Files**: `python/opendeviationbar/config/clickhouse.py`, `python/opendeviationbar/config/sidecar.py`, `python/opendeviationbar/config/algorithm.py`

**Usage**:

```python
from opendeviationbar.config import Settings

settings = Settings.get()
threshold = settings.algorithm.default_threshold_decimal_bps
ch_host = settings.clickhouse.host
```

---

### Import Organization

**Pattern**: Grouped imports per PEP 8 / isort

```python
# Standard library
from __future__ import annotations

import contextlib
import json
import logging
from collections import defaultdict
from pathlib import Path
from typing import TYPE_CHECKING, Any

# Third-party
import pandas as pd
import polars as pl
import stamina

# Local
from opendeviationbar._core import PyOpenDeviationBarProcessor
from opendeviationbar.config import Settings
from opendeviationbar.exceptions import SymbolNotRegisteredError
from opendeviationbar.validation.tier1 import validate_tier1

if TYPE_CHECKING:
    from opendeviationbar.clickhouse.cache import OpenDeviationBarCache
```

**Enforcement**: Ruff rule `I` (isort), linting via `mise run check-full`

**Location**: All Python files

---

### Common Patterns

#### 1. Mixin Classes

Organize ClickHouse operations via mixins:

```python
# File: clickhouse/bulk_operations.py
class BulkStoreMixin:
    """Store operations for bulk bars (pandas/Polars)."""

    def store_bars_bulk(
        self,
        df: pd.DataFrame,
        symbol: str,
        threshold_decimal_bps: int,
        overlap_strategy: str = "off",
    ) -> int:
        """Store bars from pandas DataFrame."""
        pass

    def store_bars_batch(
        self,
        df: pl.DataFrame,
        symbol: str,
        threshold_decimal_bps: int,
        overlap_strategy: str = "off",
    ) -> int:
        """Store bars from Polars DataFrame."""
        pass

# File: clickhouse/cache.py
class OpenDeviationBarCache(BulkStoreMixin, QueryOperationsMixin, MaintenanceMixin):
    """Main cache interface combining all mixins."""
    pass
```

**Location**: `python/opendeviationbar/clickhouse/`

#### 2. Thread-Local Cache Reuse

Prevents fd exhaustion in multi-threaded code:

```python
import threading

_thread_local = threading.local()

def _get_worker_cache() -> OpenDeviationBarCache:
    """Get thread-local cache instance (reused across iterations)."""
    if not hasattr(_thread_local, "cache"):
        _thread_local.cache = OpenDeviationBarCache()
    return _thread_local.cache

# Usage in parallel loop
with ThreadPoolExecutor(max_workers=4) as executor:
    futures = [
        executor.submit(process_symbol, symbol)
        for symbol in symbols
    ]
```

**Location**: Best practice pattern (documented in memory file)

#### 3. Validation Guards

Guard functions raise or warn on invalid state:

```python
# File: clickhouse/bulk_operations.py

def _guard_close_time_ms_scale(df: object, column: str = "close_time_ms") -> None:
    """Raise if timestamps not in millisecond scale (Issue #85, #166)."""
    cols = df.columns if hasattr(df, "columns") else []
    if column not in cols:
        return
    min_ts = df[column].min()
    max_ts = df[column].max()
    if min_ts < 1_000_000_000_000:
        raise ValueError(f"{column} in seconds, not ms")
    if max_ts > 2_500_000_000_000:
        raise ValueError(f"{column} in μs, not ms")

def _guard_volume_non_negative(
    df: object,
    columns: tuple[str, ...] = ("volume", "buy_volume", "sell_volume"),
) -> None:
    """Reject writes with negative volumes (overflow indicator, Issue #88)."""
    cols = df.columns if hasattr(df, "columns") else []
    for col in columns:
        if col in cols and df[col].min() < 0:
            raise ValueError(f"Issue #88: {col} has negative values (overflow)")

# Usage
_guard_close_time_ms_scale(df)
_guard_volume_non_negative(df)
```

**Location**: `python/opendeviationbar/clickhouse/bulk_operations.py`

#### 4. Lazy Imports (Circular Dependencies)

Use lazy imports after `__all__` to break cycles:

```python
__all__ = ["get_open_deviation_bars", "get_n_open_deviation_bars"]

# ... main API functions ...

# Lazy imports to avoid circular dependencies
from opendeviationbar._backends.polars_processor import ProcessorPolars

def process_trades_polars(trades_df: pl.DataFrame, **kwargs) -> pl.DataFrame:
    processor = ProcessorPolars(**kwargs)
    return processor.process(trades_df)
```

**Enforcement**: Ruff rule `PLC0415` (allowed via per-file-ignores)

**Location**: `python/opendeviationbar/__init__.py`

---

## Cross-Language Patterns

### PyO3 Bindings

**File**: `src/lib.rs` (~200 lines)

Map Rust types to Python exceptions:

```rust
#[pyo3(use_name = "PyOpenDeviationBarProcessor")]
pub struct PyOpenDeviationBarProcessor {
    inner: OpenDeviationBarProcessor,
}

impl PyOpenDeviationBarProcessor {
    #[new]
    pub fn new(threshold_decimal_bps: i32) -> PyResult<Self> {
        let processor = OpenDeviationBarProcessor::new(threshold_decimal_bps)
            .map_err(|e| {
                pyo3::exceptions::PyValueError::new_err(
                    format!("Invalid threshold: {}", e)
                )
            })?;
        Ok(PyOpenDeviationBarProcessor { inner: processor })
    }
}
```

**Exception Mapping**:

| Rust                                | Python                       |
| ----------------------------------- | ---------------------------- |
| `ProcessingError::InvalidThreshold` | `ValueError`                 |
| `CheckpointError::*`                | `RuntimeError`               |
| Division by zero (handled)          | Returns 0.0 (no information) |

**Location**: `src/lib.rs`, `src/core_bindings.rs`, `src/helpers.rs`

---

## Documentation Conventions

### Docstring Format (PEP 257)

```python
def populate_cache_resumable(
    symbol: str,
    start_date: str | date,
    end_date: str | date,
    *,
    threshold_decimal_bps: int = 250,
    force_refresh: bool = False,
    max_days: int | None = 366,
) -> int:
    """Populate ClickHouse cache with open deviation bars (resumable).

    Incremental, crash-safe cache population for date ranges up to 366 days.
    For longer ranges, use `mise run kintsugi:catchup` (historical backfill).

    Args:
        symbol: Trading pair (e.g., "BTCUSDT").
        start_date: Start date (inclusive).
        end_date: End date (inclusive).
        threshold_decimal_bps: Threshold in decimal basis points (dbps).
        force_refresh: Wipe existing cache and checkpoint before starting.
        max_days: Maximum span in days; None to bypass (for kintsugi only).

    Returns:
        Number of bars written to ClickHouse.

    Raises:
        SymbolNotRegisteredError: If symbol not in registry.
        ValueError: If threshold invalid or range exceeds max_days.
        RuntimeError: If ClickHouse write fails.

    Examples:
        >>> from opendeviationbar import populate_cache_resumable
        >>> written = populate_cache_resumable("BTCUSDT", "2024-01-01", "2024-06-30")
        >>> print(f"Wrote {written:,} bars")
    """
```

**Rust Equivalent** (for public APIs):

````rust
/// Non-lookahead open deviation bar processor.
///
/// Computes open deviation bars from aggregated trades with:
/// - Thresholds anchored to bar open price
/// - Temporal integrity guarantees
/// - Cross-file checkpoint support
///
/// # Examples
///
/// ```
/// use opendeviationbar_core::processor::OpenDeviationBarProcessor;
/// let mut processor = OpenDeviationBarProcessor::new(250)?;
/// let bars = processor.process_agg_trade_records(&trades)?;
/// ```
````

---

## Related

- `rustfmt.toml` — Rust formatting config
- `pyproject.toml` [tool.ruff] — Python linting rules (150+ enabled)
- `.cargo/config.toml` — Rust build flags
- `python/opendeviationbar/CLAUDE.md` — Python API details
- `crates/CLAUDE.md` — Rust crate architecture
- `docs/development/RELEASE.md` — Release workflow (semantic-release)
