# Research: Issues #21 & #22 Implementation Map
**Date**: 2026-04-08
**Issues**:
- #21: Add per-developer AI tool usage tracking (Claude Code, Cursor, Copilot)
- #22: Add 14-day code churn rate tracking

---

## 1. Database Schema

### `daily_metrics` table
**File**: `src/gitflow_analytics/models/database_commit_models.py` — class `DailyMetrics`

Current columns:
```
id, date, developer_id, project_key,
developer_name, developer_email,
feature_commits, bug_fix_commits, refactor_commits, documentation_commits,
maintenance_commits, test_commits, style_commits, build_commits, other_commits,
total_commits, files_changed, lines_added, lines_deleted, story_points,
tracked_commits, untracked_commits, unique_tickets,
merge_commits, complex_commits,
created_at, updated_at
```

**Gaps for Issue #21**: No AI tool columns exist (`ai_assisted_commits`, `claude_code_commits`, `cursor_commits`, `copilot_commits`).
**Gaps for Issue #22**: No churn columns exist (`churn_rate`, `lines_churned`). The `lines_added` and `lines_deleted` are present per-day; churn would need to be computed over a 14-day rolling window.

### `weekly_trends` table
**File**: `src/gitflow_analytics/models/database_commit_models.py` — class `WeeklyTrends`

Current columns:
```
id, week_start, week_end, developer_id, project_key,
total_commits, feature_commits, bug_fix_commits, refactor_commits,
total_commits_change, feature_commits_change, bug_fix_commits_change, refactor_commits_change,
days_active, avg_commits_per_day,
calculated_at
```

**Gaps for Issue #21**: No AI tool usage columns.
**Gaps for Issue #22**: No churn rate columns.

### Migration pattern
The project uses `ALTER TABLE … ADD COLUMN` inside `Database._apply_migrations()` in
`src/gitflow_analytics/models/database.py`. Each new column gets its own try/except block
to be idempotent. The pattern:

```python
# In Database._apply_migrations():
result = conn.execute(text("PRAGMA table_info(daily_metrics)"))
columns = {row[1] for row in result}
if "ai_assisted_commits" not in columns:
    conn.execute(text("ALTER TABLE daily_metrics ADD COLUMN ai_assisted_commits INTEGER DEFAULT 0"))
    conn.commit()
```

There is a `SchemaVersion` ORM model (`schema_version` table) that records version strings
like `"2.0"`. `CURRENT_SCHEMA_VERSION = "2.0"` in `Database`. New columns do NOT
require bumping this version — only breaking schema changes (like the v1→v2 timezone
migration) do. Additive migrations live in `_apply_migrations()` only.

---

## 2. Current AI Detection

### What exists today
The project currently detects AI involvement **only** for co-author attribution purposes
and for filtering out noise in classification — not for AI tool usage tracking.

**File**: `src/gitflow_analytics/utils/commit_utils.py`
- Function `extract_co_authors(message)` — extracts all `Co-authored-by: Name <email>` trailers
- Regex: `r"^Co-authored-by:\s*(.+?)\s*<([^>]+)>"` (MULTILINE | IGNORECASE)
- Returns `[{"name": ..., "email": ...}]`
- Used in `_analyze_commit()` to populate `commit_data["co_authors"]`

**File**: `src/gitflow_analytics/extractors/tickets.py` — function `filter_git_artifacts()`
- Strips `Co-authored-by:` lines generically (line 39)
- Has a **Copilot-specific** strip: `r"^Co-authored-by:.*[Cc]opilot.*$"` (line 57)
- Also strips `"suggestion from @?copilot"` in the `chore` category pattern (line 371)

**No existing code** maps co-author emails to specific AI tools (Claude Code, Cursor, Copilot).

### Known AI tool co-author signatures
These are the email patterns emitted by each tool in `Co-authored-by:` trailers:
- **Claude Code**: `Co-authored-by: Claude <noreply@anthropic.com>` (or similar `@anthropic.com`)
- **GitHub Copilot**: `Co-authored-by: GitHub Copilot <copilot@github.com>` or `copilot[bot] <...>`
- **Cursor**: Does not emit co-author trailers — detection requires commit message heuristics

---

## 3. Commit Data Flow (raw git → `daily_metrics` row)

### Step 1: Raw git commit → commit dict
**File**: `src/gitflow_analytics/core/analyzer_commit.py` — `CommitAnalyzerMixin._analyze_commit()`

Produces a dict with:
```python
{
  "hash": str,
  "author_name": str,
  "author_email": str,
  "message": str,
  "timestamp": datetime (UTC-aware),
  "local_hour": int,
  "is_merge": bool,
  "branch": str,
  "inferred_project": str,
  "files_changed_count": int,
  "files_changed": list[str],        # file paths
  "insertions": int,                  # raw (all files)
  "deletions": int,
  "filtered_insertions": int,         # after exclusion list
  "filtered_deletions": int,
  "story_points": int|None,
  "ticket_references": list[dict],    # [{platform, id, full_id}]
  "co_authors": list[dict],           # [{name, email}]
  "complexity_delta": float,
}
```

Raw stats come from `git diff --numstat` (both `_calculate_raw_stats` and
`_calculate_filtered_stats` parse `insertions\tdeletions\tfilename` output).

### Step 2: Commit dict → `cached_commits` row
**File**: `src/gitflow_analytics/core/cache.py` (and `cache_commits.py`)

Stores the commit dict into `CachedCommit` ORM. The `co_authors` field is NOT persisted
in `CachedCommit` — only: `commit_hash, repo_path, author_name, author_email, message,
timestamp, branch, is_merge, files_changed, insertions, deletions,
filtered_insertions, filtered_deletions, complexity_delta, story_points, ticket_references`.

### Step 3: Commit list → `daily_metrics` rows
**File**: `src/gitflow_analytics/core/metrics_storage.py` — `DailyMetricsStorage.store_daily_metrics()`
- Calls `_aggregate_commits_by_day()` which groups by `(developer_id, project_key)` for a target date
- The category comes from `commit.get("category", "other")` — set by `TicketExtractor.categorize_commit()`
- Uses `filtered_insertions` / `filtered_deletions` for line counts
- **AI tool fields do not exist** in this aggregation yet

For Issue #21, the co_authors list needs to be preserved through the pipeline (either
stored in `CachedCommit` or re-derived from `message`) and then checked in
`_aggregate_commits_by_day()`.

---

## 4. Weekly Trends Rollup

### `WeeklyTrends` ORM population
**File**: `src/gitflow_analytics/core/metrics_storage.py` — `DailyMetricsStorage.calculate_weekly_trends()`
- Queries `DailyMetrics` grouped by `func.strftime("%Y-%W", DailyMetrics.date)`
- Currently aggregates: `total_commits`, `feature_commits`, `bug_fix_commits`, `refactor_commits`
- Calculates week-over-week % change

**File**: `src/gitflow_analytics/reports/weekly_trends_writer.py` — `WeeklyTrendsWriter`
- Operates on raw commit lists (not the DB table) to produce CSV reports
- Groups by developer/project and Monday-anchored ISO week (`_get_week_start()`)
- Outputs per-category `{category}_count` and `{category}_pct_change` columns
- Does not read from `WeeklyTrends` table — it re-derives from commits directly

For Issue #22, the churn rate calculation (14-day rolling window) would fit in
`calculate_weekly_trends()` or as a separate method in `DailyMetricsStorage`.

---

## 5. CTO / Executive Report — AI Adoption Section

### Current state
There is **no "§4 AI adoption" section** in any report file. Searching all report files
for "AI adoption", "ai_assisted", "ai_tool", "§4" returned zero matches.

The narrative report is generated by:
- **File**: `src/gitflow_analytics/reports/narrative_writer.py` — `NarrativeReportGenerator.generate_narrative_report()`
- Composed from mixins: `NarrativeExecutiveMixin`, `NarrativeClassificationMixin`,
  `NarrativeTeamMixin`, `NarrativeAnalysisMixin`, `NarrativeRecommendationsMixin`

Sections currently produced:
1. Executive Summary (`_write_executive_summary` in `narrative_executive.py`)
2. Team Composition / Developer Profiles (`_write_team_composition` in `narrative_team.py`)
3. Project Activity
4. PR Analysis
5. Recommendations

An "AI Adoption" section would be a new `_write_ai_adoption()` method, best added to
`narrative_analysis.py` or a new `narrative_ai.py` mixin.

The executive summary already writes Markdown bullet points like:
```
- **Total Commits**: N
- **Ticket Coverage**: N%
```
The AI section would follow the same pattern.

---

## 6. Key Extractor/Writer Files

### `src/gitflow_analytics/extractors/tickets.py`
- Class `TicketExtractor(TicketAnalysisMixin)`
- `extract_from_text(text)` — returns `[{platform, id, full_id}]`
- `categorize_commit(message)` — returns category string, checked in priority order
- `filter_git_artifacts(message)` — strips co-author lines, Copilot lines, merge artifacts
- `should_skip_commit(commit)` — respects `commit_filter` config
- Inherits `analyze_ticket_coverage()` from `TicketAnalysisMixin`

### `src/gitflow_analytics/extractors/tickets_analysis.py`
- Class `TicketAnalysisMixin`
- `analyze_ticket_coverage(commits, prs)` — main coverage analysis entry point
- Filters excluded repos and excluded authors
- Produces `untracked_commits` list with `category` field set

### `src/gitflow_analytics/reports/weekly_trends_writer.py`
- Class `WeeklyTrendsWriter`
- `generate_weekly_trends_reports(commits, output_dir, weeks, ...)` — main entry point
- `_generate_developer_weekly_trends()` — per-developer CSV
- `_generate_project_weekly_trends()` — per-project CSV
- `_get_commit_classification(commit, categorize_fn)` — priority: `predicted_class` > `category` > `classification` > `categorize_fn` > `"other"`
- Uses Monday-aligned ISO weeks (`_get_week_start()`)
- All 20 classification categories listed in `self.classification_categories`

---

## 7. Test Patterns

### Schema migration tests
Pattern from `tests/models/test_pull_request_cache_v4.py`:
```python
class TestMigrationAppliesV4Columns:
    def _create_v3_db(self, db_path):
        # Create table with old schema via raw SQL
        engine = create_engine(f"sqlite:///{db_path}")
        with engine.connect() as conn:
            conn.execute(text("CREATE TABLE ... (old columns only)"))
            conn.execute(text("INSERT INTO ... (legacy row)"))
            conn.commit()

    def test_migration_adds_all_v4_columns(self, tmp_path):
        cache_dir = tmp_path / ".gitflow-cache"
        cache_dir.mkdir()
        self._create_v3_db(cache_dir / "gitflow_cache.db")
        cache = GitAnalysisCache(cache_dir, ttl_hours=24)  # triggers migration
        with cache.db.engine.connect() as conn:
            result = conn.execute(text("PRAGMA table_info(pull_request_cache)"))
            actual_columns = {row[1] for row in result}
        assert "new_column" in actual_columns

    def test_migration_preserves_existing_rows(self, tmp_path): ...
    def test_migration_idempotent(self, tmp_path): ...
```

For Issues #21/#22, follow this pattern with `daily_metrics` and `weekly_trends` tables.

### AI detection / classification tests
Pattern from `tests/extractors/test_commit_categorization.py` and
`tests/test_classification_system.py`:
- Use `TicketExtractor().categorize_commit(message)` directly
- Assert returned category string

For Issue #21, tests would assert:
```python
def test_detects_claude_code_coauthor():
    commit = {"message": "feat: add feature\n\nCo-authored-by: Claude <noreply@anthropic.com>"}
    result = detect_ai_tool(commit)
    assert result["claude_code"] == True

def test_detects_copilot_coauthor():
    commit = {"message": "fix: bug\n\nCo-authored-by: GitHub Copilot <copilot@github.com>"}
    result = detect_ai_tool(commit)
    assert result["copilot"] == True
```

---

## 8. Existing Churn-Related Code

**No code churn tracking exists.** The closest things:
- `lines_added` / `lines_deleted` in `DailyMetrics` — raw additions/deletions per developer per day
- `filtered_insertions` / `filtered_deletions` in `CachedCommit` — after exclude_paths filtering
- `revision_count` in `PullRequestCache` — commits pushed after PR opened (PR-level rework, not code churn)
- `metrics/activity_scoring.py` docstring mentions "Code churn (deletions valued for refactoring)" but there is no churn rate calculation anywhere

**For Issue #22**, code churn rate over 14 days would be calculated as:
```
churn_rate = sum(deletions over 14 days) / sum(insertions over 14 days)
```
or alternatively:
```
churn_lines = sum(deletions) (lines removed = old code replaced)
churn_rate = churn_lines / total_lines_changed
```
Data source: query `DailyMetrics` with `date >= today - 14 days`, aggregate
`lines_added` and `lines_deleted` per developer.

---

## Implementation Recommendations

### Issue #21: Per-developer AI tool tracking

1. **Detection function** (new file or add to `commit_utils.py`):
   - Parse `co_authors` list for known AI tool email patterns
   - Claude Code: email contains `anthropic.com`
   - Copilot: email contains `copilot@github.com` or name matches `copilot[bot]`
   - Cursor: heuristic on commit message (no standard co-author trailer exists)

2. **Preserve co_authors through pipeline**:
   - Option A: Add `co_authors` JSON column to `CachedCommit`
   - Option B: Re-parse from `message` field in `_aggregate_commits_by_day()`
   - Option B is simpler (message is already stored)

3. **New columns in `daily_metrics`** (add via `_apply_migrations()`):
   - `ai_assisted_commits INTEGER DEFAULT 0`
   - `claude_code_commits INTEGER DEFAULT 0`
   - `cursor_commits INTEGER DEFAULT 0`
   - `copilot_commits INTEGER DEFAULT 0`

4. **Update `_aggregate_commits_by_day()`** in `metrics_storage.py` to count AI tool usage

5. **New `_write_ai_adoption()` method** in `narrative_analysis.py` or new `narrative_ai.py`

### Issue #22: 14-day code churn rate

1. **No new DB columns needed** for the metric itself — it is derived from existing
   `lines_added` / `lines_deleted` in `DailyMetrics`

2. **New method** in `DailyMetricsStorage`:
   ```python
   def calculate_churn_rate(self, developer_id, end_date, days=14) -> float:
       # Query daily_metrics for the 14-day window
       # Return deletions / (additions + deletions) or similar
   ```

3. **Optional**: Add `churn_rate_14d REAL` to `WeeklyTrends` for precomputed reporting

4. **Report integration**: Add to developer profile section in `narrative_team.py`
   and to developer CSV in `csv_reports_developer.py`
