# Symbol Registry Integration: Implementation Plan

**Task:** task-31 - Draft Implementation Plan  
**Generated:** 2026-03-12  
**Author:** SwiftGrove (crew-worker)

---

## Overview

This document provides a detailed 4-phase implementation plan for integrating the Symbol Registry (`symbols.toml`) as the single source of truth for all hardcoded values in the opendeviationbar codebase. The plan includes timeline estimates, specific milestones, file changes, and verification commands.

**Context:** The audit phase identified **45+ locations** with hardcoded threshold values (primarily `250`), symbol lists, date ranges, and other configuration values that should be sourced from the Symbol Registry.

---

## Phase 1: Quick Wins (Foundation)

**Timeline:** Week 1 (1-2 days)  
**Effort:** Low  
**Impact:** 15+ hardcoded values eliminated

### Objectives

1. Add helper functions to `symbol_registry.py`
2. Update `constants.py` to use registry
3. Update config files with registry fallback

### Milestones

| # | Milestone | Deliverable |
|---|-----------|-------------|
| 1.1 | Add `get_default_threshold()` | Function returns 250, reads from registry metadata if available |
| 1.2 | Add `get_tier1_symbols()` | Function returns list of tier 1 symbols from registry |
| 1.3 | Add `get_default_thresholds()` | Function returns (250, 500, 750, 1000) tuple |
| 1.4 | Update constants.py | Import from registry instead of hardcoded values |
| 1.5 | Update algorithm.py config | Use `get_default_threshold()` for fallback |
| 1.6 | Update population.py config | Use registry for min thresholds |

### File Changes

```diff
# python/opendeviationbar/symbol_registry.py
+ def get_default_threshold() -> int:
+     """Get the default threshold from registry metadata or return 250."""
+     # Implementation...

+ def get_tier1_symbols() -> tuple[str, ...]:
+     """Get tier 1 symbols from registry."""
+     # Implementation...

+ def get_default_thresholds() -> tuple[int, ...]:
+     """Get default threshold presets."""
+     return (250, 500, 750, 1000)

# python/opendeviationbar/constants.py
- DEFAULT_THRESHOLD: int = 250
+ from opendeviationbar.symbol_registry import get_default_threshold
+ DEFAULT_THRESHOLD: int = get_default_threshold()

- TIER1_SYMBOLS: tuple[str, ...] = ("BTCUSDT", "ETHUSDT", ...)
+ from opendeviationbar.symbol_registry import get_tier1_symbols
+ TIER1_SYMBOLS: tuple[str, ...] = get_tier1_symbols()

- THRESHOLD_PRESETS: tuple[int, ...] = (250, 500, 750, 1000)
+ from opendeviationbar.symbol_registry import get_default_thresholds
+ THRESHOLD_PRESETS: tuple[int, ...] = get_default_thresholds()

# python/opendeviationbar/config/algorithm.py
- default_threshold_decimal_bps: int = 250
+ default_threshold_decimal_bps: int = get_default_threshold()
```

### Verification Commands

```bash
# Test registry functions directly
python -c "
from opendeviationbar.symbol_registry import (
    get_default_threshold,
    get_tier1_symbols,
    get_default_thresholds
)

# Verify default threshold
dt = get_default_threshold()
assert dt == 250, f'Expected 250, got {dt}'
print(f'✓ get_default_threshold() = {dt}')

# Verify tier1 symbols returns non-empty
symbols = get_tier1_symbols()
assert len(symbols) > 0, 'TIER1_SYMBOLS should not be empty'
print(f'✓ get_tier1_symbols() returned {len(symbols)} symbols')

# Verify threshold presets
presets = get_default_thresholds()
assert presets == (250, 500, 750, 1000), f'Unexpected presets: {presets}'
print(f'✓ get_default_thresholds() = {presets}')
"

# Verify constants.py integration
python -c "
from opendeviationbar.constants import DEFAULT_THRESHOLD, TIER1_SYMBOLS, THRESHOLD_PRESETS

print(f'DEFAULT_THRESHOLD = {DEFAULT_THRESHOLD}')
print(f'TIER1_SYMBOLS = {TIER1_SYMBOLS}')
print(f'THRESHOLD_PRESETS = {THRESHOLD_PRESETS}')

# Assert they match registry
from opendeviationbar.symbol_registry import get_default_threshold, get_tier1_symbols, get_default_thresholds

assert DEFAULT_THRESHOLD == get_default_threshold(), 'DEFAULT_THRESHOLD mismatch'
assert set(TIER1_SYMBOLS) == set(get_tier1_symbols()), 'TIER1_SYMBOLS mismatch'
assert THRESHOLD_PRESETS == get_default_thresholds(), 'THRESHOLD_PRESETS mismatch'
print('✓ All constants match registry values')
"

# Verify config integration
python -c "
from opendeviationbar.config.algorithm import AlgorithmConfig
config = AlgorithmConfig()
print(f'AlgorithmConfig().default_threshold_decimal_bps = {config.default_threshold_decimal_bps}')
assert config.default_threshold_decimal_bps == 250, 'Config should use registry default'
print('✓ Config files use registry fallback')
"

# Run existing tests to ensure no regressions
cd python && python -m pytest tests/ -v -x --tb=short -k "not slow" 2>&1 | head -50
```

---

## Phase 2: Rust Tool Integration

**Timeline:** Week 2 (1-2 days)  
**Effort:** Low  
**Impact:** 5+ hardcoded values eliminated

### Objectives

1. Add environment variable support to Rust CLI tools
2. Update `.mise.toml` with default values
3. Document configuration options

### Milestones

| # | Milestone | Deliverable |
|---|-----------|-------------|
| 2.1 | Update tools/chart/src/main.rs | Add env var support for threshold and symbol |
| 2.2 | Update spot_tier1_processor | Add env var support for dates, threshold, workers |
| 2.3 | Update .mise.toml | Add OPENDEVIATIONBAR_DEFAULT_THRESHOLD |
| 2.4 | Document env vars | Update README or add CONFIG.md |

### File Changes

```diff
# tools/chart/src/main.rs
+ const DEFAULT_THRESHOLD: u32 = 250;
+ const DEFAULT_SYMBOL: &str = "BTCUSDT";
+
+ fn get_threshold_from_env() -> u32 {
+     std::env::var("OPENDEVIATIONBAR_DEFAULT_THRESHOLD")
+         .ok()
+         .and_then(|v| v.parse().ok())
+         .unwrap_or(DEFAULT_THRESHOLD)
+ }
+
+ fn get_symbol_from_env() -> String {
+     std::env::var("OPENDEVIATIONBAR_DEFAULT_SYMBOL")
+         .unwrap_or_else(|| DEFAULT_SYMBOL.to_string())
+ }

# .mise.toml
+ [env]
+ OPENDEVIATIONBAR_DEFAULT_THRESHOLD = "250"
+ OPENDEVIATIONBAR_DEFAULT_SYMBOL = "BTCUSDT"
+ OPENDEVIATIONBAR_CRYPTO_MIN_THRESHOLD = "1000"
```

### Verification Commands

```bash
# Test chart tool with default values
cargo run --manifest-path tools/chart/Cargo.toml -- --help

# Test with environment variables
OPENDEVIATIONBAR_DEFAULT_THRESHOLD=500 cargo run --manifest-path tools/chart/Cargo.toml -- --symbol ETHUSDT
OPENDEVIATIONBAR_DEFAULT_SYMBOL=ETHUSDT cargo run --manifest-path tools/chart/Cargo.toml

# Run chart tool tests
cargo test --manifest-path tools/chart/Cargo.toml

# Verify mise config
source .mise.toml && echo "OPENDEVIATIONBAR_DEFAULT_THRESHOLD=${OPENDEVIATIONBAR_DEFAULT_THRESHOLD}"
```

---

## Phase 3: Build-Time Constants Generation

**Timeline:** Week 3 (2-3 days)  
**Effort:** Medium  
**Impact:** 20+ hardcoded values eliminated automatically

### Objectives

1. Create `build.rs` for opendeviationbar-core
2. Generate Rust constants from registry at build time
3. Update test files to use generated constants

### Milestones

| # | Milestone | Deliverable |
|---|-----------|-------------|
| 3.1 | Create build.rs | Parse symbols.toml and generate constants |
| 3.2 | Generate DEFAULT_THRESHOLD | Build-time constant from registry |
| 3.3 | Generate TIER1_SYMBOLS | Build-time constant list |
| 3.4 | Update test files | Use generated constants |
| 3.5 | Add build verification | CI checks constants are in sync |

### File Changes

```diff
# crates/opendeviationbar-core/build.rs (new file)
+ use std::env;
+ use std::fs;
+ use std::path::Path;
+ 
+ fn main() {
+     // Find symbols.toml relative to crate root
+     let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
+     let symbols_path = Path::new(&manifest_dir)
+         .join("..")
+         .join("python")
+         .join("opendeviationbar")
+         .join("data")
+         .join("symbols.toml");
+ 
+     let symbols_content = fs::read_to_string(&symbols_path)
+         .expect("Failed to read symbols.toml");
+ 
+     // Parse and extract values (simplified)
+     let default_threshold = extract_default_threshold(&symbols_content);
+     let tier1_symbols = extract_tier1_symbols(&symbols_content);
+ 
+     // Generate constants.rs
+     let output = format!(
+         r#"// Auto-generated by build.rs - DO NOT EDIT
+ pub const DEFAULT_THRESHOLD: u32 = {};
+ pub const TIER1_SYMBOLS: &[&str] = &{};
+ "#,
+         default_threshold,
+         tier1_symbols.iter().map(|s| format!("\"{}\"", s)).collect::<Vec<_>>().join(", ")
+     );
+ 
+     let out_dir = env::var("OUT_DIR").unwrap();
+     fs::write(Path::new(&out_dir).join("constants.rs"), output).unwrap();
+ }

# crates/opendeviationbar-core/src/lib.rs
+ include!(concat!(env!("OUT_DIR"), "/constants.rs"));
```

### Verification Commands

```bash
# Build the core crate - should trigger build.rs
cargo build --manifest-path crates/opendeviationbar-core/Cargo.toml

# Check generated constants
cargo run --manifest-path crates/opendeviationbar-core/Cargo.toml --example show_constants 2>/dev/null || \
  cargo test --manifest-path crates/opendeviationbar-core/Cargo.toml -- --nocapture 2>&1 | head -30

# Verify build reproducibility
cargo clean --manifest-path crates/opendeviationbar-core/Cargo.toml
cargo build --manifest-path crates/opendeviationbar-core/Cargo.toml
cargo build --manifest-path crates/opendeviationbar-core/Cargo.toml  # Should be no-op

# Run core tests
cargo test --manifest-path crates/opendeviationbar-core/Cargo.toml
```

---

## Phase 4: Registry Enhancement (Advanced)

**Timeline:** Week 4+ (ongoing)  
**Effort:** Medium  
**Impact:** Enable future use cases, complete integration

### Objectives

1. Add metadata section to symbols.toml
2. Add asset-class-specific thresholds
3. Add priority symbols support
4. Add phase threshold definitions

### Milestones

| # | Milestone | Deliverable |
|---|-----------|-------------|
| 4.1 | Add meta section to symbols.toml | Default thresholds, crypto min, etc. |
| 4.2 | Add get_default_threshold_for_asset_class() | Per-asset-class defaults |
| 4.3 | Add get_phase_thresholds() | For population.py phases |
| 4.4 | Add get_priority_symbols() | For backfill.py priority list |
| 4.5 | Deprecate hardcoded values | Add warnings for direct access |

### File Changes

```diff
# python/opendeviationbar/data/symbols.toml
+ [meta]
+ # Global defaults
+ default_threshold = 250
+ crypto_min_threshold = 1000
+ 
+ [meta.default_thresholds]
+ crypto = 250
+ forex = 50
+ equities = 25
+ commodities = 100
+ 
+ [meta.phase_thresholds]
+ phase_1 = [1000]
+ phase_2 = [250]
+ phase_3 = [500, 750]
+ phase_4 = [100]
+ 
+ [meta.priority_symbols]
+ # Symbols that should always be processed first
+ priority = ["BTCUSDT", "ETHUSDT", "SOLUSDT"]

# python/opendeviationbar/symbol_registry.py
+ def get_default_threshold_for_asset_class(asset_class: str) -> int:
+     """Get default threshold for a specific asset class."""
+     # Read from meta.default_thresholds
+ 
+ def get_phase_thresholds(phase: str) -> list[int]:
+     """Get thresholds for a specific population phase."""
+     # Read from meta.phase_thresholds
+ 
+ def get_priority_symbols() -> list[str]:
+     """Get priority symbols from registry."""
+     # Read from meta.priority_symbols
```

### Verification Commands

```bash
# Verify metadata section exists and is parseable
python -c "
import toml
data = toml.load('python/opendeviationbar/data/symbols.toml')
assert 'meta' in data, 'symbols.toml should have [meta] section'
print('✓ symbols.toml has [meta] section')

# Test new functions
from opendeviationbar.symbol_registry import (
    get_default_threshold_for_asset_class,
    get_phase_thresholds,
    get_priority_symbols
)

# Asset class thresholds
crypto_threshold = get_default_threshold_for_asset_class('crypto')
print(f'Crypto threshold: {crypto_threshold}')

# Phase thresholds
phase2 = get_phase_thresholds('phase_2')
print(f'Phase 2 thresholds: {phase2}')

# Priority symbols
priority = get_priority_symbols()
print(f'Priority symbols: {priority}')
"

# Test full integration chain
python -c "
from opendeviationbar.symbol_registry import get_symbol_entries

# Verify all symbols can be queried
entries = get_symbol_entries()
print(f'Loaded {len(entries)} symbols from registry')

# Verify metadata is accessible
for entry in entries[:3]:
    print(f'  {entry.symbol}: threshold={entry.threshold}, enabled={entry.enabled}')
"
```

---

## Timeline Summary

| Phase | Duration | Effort | Impact (Files Changed) |
|-------|----------|--------|------------------------|
| Phase 1: Quick Wins | Week 1 (1-2 days) | Low | 3-5 files |
| Phase 2: Rust Tools | Week 2 (1-2 days) | Low | 3-4 files |
| Phase 3: Build-Time | Week 3 (2-3 days) | Medium | 2-3 files + auto-gen |
| Phase 4: Enhancement | Week 4+ (ongoing) | Medium | 2-3 files |

**Total Estimated Effort:** 1 developer week  
**Total Files Changed:** 10-15  
**Total Hardcoded Values Eliminated:** 45+

---

## Rollback Strategy

If issues arise during implementation:

| Phase | Rollback Command |
|-------|------------------|
| Phase 1 | `git checkout python/opendeviationbar/constants.py python/opendeviationbar/symbol_registry.py` |
| Phase 2 | `git checkout tools/chart/src/main.rs .mise.toml` |
| Phase 3 | `git checkout crates/opendeviationbar-core/build.rs crates/opendeviationbar-core/src/lib.rs` |
| Phase 4 | `git checkout python/opendeviationbar/data/symbols.toml python/opendeviationbar/symbol_registry.py` |

---

## Testing Strategy

### Unit Tests (Per Phase)

```bash
# Phase 1: Python unit tests
cd python && python -m pytest tests/unit/test_symbol_registry.py -v

# Phase 2: Rust integration tests
cargo test --manifest-path tools/chart/Cargo.toml

# Phase 3: Build verification
cargo test --manifest-path crates/opendeviationbar-core/Cargo.toml

# Phase 4: End-to-end tests
cd python && python -m pytest tests/ -v -k "integration"
```

### Regression Testing

```bash
# Full test suite
mise run check  # or equivalent: pytest + cargo test

# Smoke test key paths
python -c "from opendeviationbar import main; print('Import OK')"
cargo build --manifest-path crates/opendeviationbar-core/Cargo.toml
```

---

## Success Criteria

| Criterion | Metric |
|-----------|--------|
| No hardcoded threshold=250 in Python | `grep -r "= 250" python/opendeviationbar/*.py | grep -v test | wc -l` should be 0 |
| Registry functions return correct values | All verification commands pass |
| Tests pass | `mise run check` exits 0 |
| Build succeeds | `cargo build --all` exits 0 |

---

## Next Steps

1. **Create Phase 1 sub-tasks** for parallel implementation
2. **Assign owners** to each phase
3. **Schedule review** after each phase completion
4. **Update tracking** in project management tool

---

*Implementation plan generated for task-31: Draft implementation plan*
