# Task for crew-worker

# Task Assignment

**Task ID:** task-15
**Task Title:** Task 6: Backfill and Population Pipeline Audit
**PRD:** /Users/terryli/eon/opendeviationbar-py/PRD.md


## Your Mission

Implement this task following the crew-worker protocol:
1. Join the mesh
2. Read task spec to understand requirements
3. Start task and reserve files
4. Implement the feature
5. Commit your changes
6. Release reservations and mark complete

## Progress from Prior Attempts

[2026-03-12T20:42:57.817Z] (system) Assigned to crew-worker (attempt 1)
[2026-03-12T20:48:03.016Z] (system) Assigned to crew-worker (attempt 1)

## Concurrent Tasks

These tasks are being worked on by other workers in this wave. Discover their agent names after joining the mesh via `pi_messenger({ action: "list" })`.

- task-12: Task 3: ClickHouse Cache Layer Audit
- task-13: Task 4: Configuration Modules Audit
- task-14: Task 5: Scripts Directory Audit
- task-16: Task 7: Rust Crates Audit

## Recent Activity

13:54 YoungViper completed task-19 — Created proposal for adding asset-class default thresholds to symbols.toml. Proposed adding [meta.default_thresholds] section, get_default_threshold_for_asset_class() and get_effective_default_threshold() functions to symbol_registry.py. Documented update path for constants.py and threshold.py.
13:54 YoungViper started task-20 — Document hardcoded threshold locations needing updates
13:55 YoungViper completed task-20 — Documented 76+ files with hardcoded threshold=250 values across core library, examples, scripts, and tests. Categorized by priority based on impact. Created detailed migration strategy and recommended helper functions needed.
13:55 YoungViper started task-21 — Audit config files for registry integration
13:56 YoungViper completed task-21 — Audited config files (algorithm.py, threshold.py, population.py, cli_models.py). Proposed registry integration: add [meta.default_thresholds] to symbols.toml, add get_default_threshold_for_asset_class() helper, update config classes with computed properties.
13:56 YoungViper ✦ Completed task-17: Symbol Registry Integration Audit. Key findings: (1) Registry has 16 fields in SymbolEntry, covers thresholds via min_threshold/thresholds. (2) 76+ files have hardcoded 250 dbps....
13:56 SageKnight started task-10 — Task 1: Python Constants Module Audit
13:56 JadeFalcon started task-11 — Task 2: CLI/Tool Entry Points Audit

## Task Specification

# Task 6: Backfill and Population Pipeline Audit

Investigate data pipeline code for hardcoded thresholds and symbol lists. Create scaffolding directory at /tmp/crew-perspective-6/. Check python/opendeviationbar/backfill.py for hardcoded thresholds; audit population logic in python/opendeviationbar/population.py; review checkpoint logic (checkpoint.py) for threshold handling; check sidecar streaming (sidecar.py) for threshold validation vs registry. Use Dynamic Task Creation: create sub-tasks to document pipeline touchpoints that need registry integration. Broadcast key findings to all peers.


## Plan Context

Now I have a complete picture. Let me produce the task breakdown.

## 1. PRD Understanding Summary

The PRD requires auditing the codebase to identify all hardcoded values that should come from the Symbol Registry (symbols.toml). Key findings:

- **Symbol Registry**: Exists at `python/opendeviationbar/data/symbols.toml` with threshold fields, listing dates, asset class, exchange, enabled status
- **Hardcoded Values Found**:
  - Python: `constants.py` has `THRESHOLD_PRESETS` (250, 500, 750, 1000), multiple config files with `default_threshold_decimal_bps = 250`
  - Rust: `tools/chart/src/main.rs` has `default_value_t = 250`, test files with hardcoded thresholds
  - ClickHouse: `cache.py` has hardcoded "BTCUSDT" and threshold values in queries
  - Scripts: Many scripts with hardcoded threshold values

## 2. Relevant Code/Docs/Resources Reviewed

- **PRD.md**: Full specification with 9 investigative perspectives
- **Symbol Registry**: `python/opendeviationbar/symbol_registry.py` - exposes `get_thresholds_for_symbol()`, `get_symbol_entries()`, etc.
- **Constants**: `python/opendeviationbar/constants.py` - `THRESHOLD_PRESETS`, `TIER1_SYMBOLS`
- **CLI**: `tools/chart/src/main.rs` - default threshold 250
- **Config**: Multiple files in `python/opendeviationbar/config/` with hardcoded defaults
- **ClickHouse**: `python/opendeviationbar/clickhouse/cache.py` - hardcoded BTCUSDT and thresholds
- **Crates**: `crates/opendeviationbar-core/tests/` - hardcoded thresholds in tests

## 3. Sequential Implementation Steps

1. **Tasks 1-8 (Parallel)**: Each worker investigates their assigned code area
   - Create scaffolding directory `/tmp/crew-perspective-<N>/`
   - Systematically search for hardcoded thresholds, symbol names, dates
   - Use grep/ripgrep for empirical validation
   - Broadcast findings to peers

2. **Task 9**: Cross-Cutting Synthesis (after Tasks 1-8 complete)
   - Aggregate all hardcoded values found
   - Categorize by type: thresholds, symbols, dates
   - Propose r

[Spec truncated - read full spec from .pi/messenger/crew/plan.md]
## Coordination

**Message budget: 10 messages this session.** The system enforces this — sends are rejected after the limit.

**Broadcasts go to the team feed — only the user sees them live.** Other workers see your broadcasts in their initial context only. Use DMs for time-sensitive peer coordination.

### Announce yourself
After joining the mesh and starting your task, announce what you're working on:

```typescript
pi_messenger({ action: "broadcast", message: "Starting <task-id> (<title>) — will create <files>" })
```

### Coordinate with peers
If a concurrent task involves files or interfaces related to yours, send a brief DM. Only message when there's a concrete coordination need — shared files, interfaces, or blocking questions.

```typescript
pi_messenger({ action: "send", to: "<peer-name>", message: "I'm exporting FormatOptions from types.ts — will you need it?" })
```

### Responding to messages
If a peer asks you a direct question, reply briefly. Ignore messages that don't require a response. Do NOT start casual conversations.

### On completion
Announce what you built:

```typescript
pi_messenger({ action: "broadcast", message: "Completed <task-id>: <file> exports <symbols>" })
```

### Reservations
Before editing files, check if another worker has reserved them via `pi_messenger({ action: "list" })`. If a file you need is reserved, message the owner to coordinate. Do NOT edit reserved files without coordinating first.

### Questions about dependencies
If your task depends on a completed task and something about its implementation is unclear, read the code and the task's progress log at `.pi/messenger/crew/tasks/<task-id>.progress.md`. Dependency authors are from previous waves and are no longer in the mesh.

### Claim next task
After completing your assigned task, check if there are ready tasks you can pick up:

```typescript
pi_messenger({ action: "task.ready" })
```

If a task is ready, claim and implement it. If `task.start` fails (another worker claimed it first), check for other ready tasks. Only claim if your current task completed cleanly and quickly.

## Available Skills

Read any skill that matches what you're implementing.

  ralph-loop — Use ralph_loop tool to run agent tasks in a persistent loop until a goal or condition is met. Load this skill whenever the user mentions "ralph loop", "keep trying", "retry until", "loop until", "run until done", or wants something to keep going across sessions.
    /Users/terryli/.pi/agent/skills/ralph-loop/SKILL.md

To load a skill: read({ path: "<skill-path>" })
