# Commit Classification System: Adding `complexity` Rating Field

**Date**: 2026-02-27
**Scope**: End-to-end investigation of the LLM classification pipeline to support adding a `complexity` (1-5) field alongside `change_type`.

---

## 1. LLM Prompt Template (the exact text sent to Bedrock)

The active prompt version is **V3_CONTEXTUAL** (set in both `BedrockClassifier.__init__` and `OpenAIClassifier.__init__`).

**File**: `src/gitflow_analytics/qualitative/classifiers/llm/prompts.py` — `PromptGenerator.TEMPLATES[PromptVersion.V3_CONTEXTUAL]`

### System prompt (single commit)
```
You are a specialized git commit classifier with deep understanding
of software development patterns. Consider both the commit message and file context
to make accurate classifications.
```

### User prompt template (single commit)
```
Analyze this commit and classify it into the most appropriate category.

Categories (choose ONE):
{categories_desc}

Commit Details:
Message: "{message}"
{context_info}

Classification Rules:
1. Focus on the PRIMARY purpose of the commit
2. Consider file types and patterns for additional context
3. If multiple categories apply, choose the most significant one
4. Be confident in clear cases, conservative when ambiguous

Format: CATEGORY confidence reasoning
Response:
```

Where `{categories_desc}` expands to (7 categories):
```
- feature: New functionality, capabilities, enhancements, additions
- bugfix: Fixes, errors, issues, crashes, bugs, corrections
- maintenance: Configuration, chores, dependencies, cleanup, refactoring, updates
- integration: Third-party services, APIs, webhooks, external systems
- content: Text, copy, documentation, README updates, comments
- media: Video, audio, streaming, players, visual assets, images
- localization: Translations, i18n, l10n, regional adaptations
```

### Batch prompt (Bedrock native batching — the path actually used in production)

**File**: `src/gitflow_analytics/qualitative/classifiers/llm/bedrock_client.py` — `BedrockClassifier._BATCH_SYSTEM_PROMPT`

```
You are a git commit classifier. For each commit, classify it into exactly one category.

Categories: feature, bugfix, maintenance, integration, content, media, localization

Respond with a JSON array where each element has:
category, confidence (0.0-1.0), reasoning (max 10 words).

Example response for 3 commits:
[{"category":"bugfix","confidence":0.95,"reasoning":"fixes crash in login flow"},
{"category":"feature","confidence":0.90,"reasoning":"adds new authentication system"},
{"category":"maintenance","confidence":0.85,"reasoning":"updates dependency versions"}]

IMPORTANT: Return ONLY the JSON array. No other text.
```

User prompt is constructed at runtime:
```
Classify these {n} commits:
1. "{commits[0].message}"
2. "{commits[1].message}"
...
```

The Bedrock batch path is used by default when AWS credentials are available.
Sub-batch size is capped at **25 commits per API call** (`_API_BATCH_SIZE = 25`).

---

## 2. DB Schema for Classifications

### Primary classification table
**File**: `src/gitflow_analytics/models/database_metrics_models.py`
**Table**: `qualitative_commits`

```python
class QualitativeCommitData(Base):
    __tablename__ = "qualitative_commits"

    commit_id   = Column(Integer, ForeignKey("cached_commits.id"), primary_key=True)

    # Classification results
    change_type            = Column(String, nullable=False)
    change_type_confidence = Column(Float, nullable=False)
    business_domain        = Column(String, nullable=False)
    domain_confidence      = Column(Float, nullable=False)
    risk_level             = Column(String, nullable=False)
    risk_factors           = Column(JSON)

    # Intent and context
    intent_signals          = Column(JSON)
    collaboration_patterns  = Column(JSON)
    technical_context       = Column(JSON)   # used to store llm_category, llm_confidence, error

    # Processing metadata
    processing_method  = Column(String, nullable=False)   # 'nlp' | 'llm' | 'fallback'
    processing_time_ms = Column(Float)
    confidence_score   = Column(Float, nullable=False)
    analyzed_at        = Column(DateTime(timezone=True))
    analysis_version   = Column(String, default="1.0")
```

### Supporting tables that touch classification

| Table | Relevant columns | Purpose |
|---|---|---|
| `daily_commit_batches` | `classification_status`, `classified_at` | Tracks which days have been classified |
| `classification_batches` | `model_used`, `prompt_template`, token counts | Batch-level audit trail |
| `daily_metrics` | `feature_commits`, `bug_fix_commits`, etc. | Pre-aggregated counts by category |

### Migration pattern

The project uses a `SchemaVersion` SQLAlchemy model in `core/schema_version.py` that tracks field lists per component (`"qualitative"` component lists the current fields). SQLAlchemy `create_all()` creates missing columns — but does **not** ALTER existing ones. For adding a nullable column to an existing database, an Alembic migration (or a manual `ALTER TABLE` in `core/cache.py`'s startup migration code) is needed.

---

## 3. Classification Pipeline Flow

```
pipeline_classify.py::run_classify()
  └── BatchCommitClassifier.classify_date_range()          [batch_classifier.py]
        └── _classify_weekly_batches(weekly_batches)        [batch_classifier_impl.py]
              └── _classify_commit_batch_with_llm(commits, tickets)
                    └── LLMCommitClassifier.classify_commits_batch()  [llm_commit_classifier.py]
                          ├── [fast path] BedrockClassifier.classify_commits_batch()
                          │     └── _classify_api_batch(sub_batch)
                          │           builds: user_prompt = "Classify these N commits:\n1. '...'\n..."
                          │           sends:  _BATCH_SYSTEM_PROMPT + user_prompt → Bedrock API
                          │           parses: JSON array [{category, confidence, reasoning}, ...]
                          │           returns: [ClassificationResult, ...]
                          └── [fallback] BatchProcessor → classify_commit() per-commit
                                └── PromptGenerator.generate_prompt() + Bedrock/OpenRouter → ResponseParser

              └── _store_commit_classification(session, result)       [batch_classifier_impl.py]
                    ├── look up CachedCommit by commit_hash
                    ├── UPSERT QualitativeCommitData:
                    │     change_type = result["category"]
                    │     change_type_confidence = result["confidence"]
                    │     confidence_score = result["confidence"]
                    │     processing_method = result["method"]
                    │     technical_context = {llm_category, llm_confidence, error}
                    └── session.commit()
```

### Reclassify flag

`gfa classify --reclassify` exists. It maps to `force_reclassify=True` in `run_classify()`, which passes `force_reclassify=True` to `BatchCommitClassifier.classify_date_range()`.

In `_get_batches_to_process()`:
```python
if not force_reclassify:
    query = query.filter(
        DailyCommitBatch.classification_status.in_(["pending", "failed"])
    )
# With force_reclassify=True: no status filter — all batches are returned
```

In `_store_commit_classification()`: it always does an UPSERT (SELECT then UPDATE or INSERT), so reclassification overwrites existing `QualitativeCommitData` rows. It does **not** clear old rows first.

---

## 4. Report Consumption of Classification Data

Classification data is consumed via the `qualitative_commits` table in two main paths:

**Path A — Database report generator** (`reports/database_report_generator.py`): joins `cached_commits` with `qualitative_commits` on `commit_id`, surfacing `change_type` and `confidence_score` for CSV and narrative reports.

**Path B — Classification report writer** (`reports/classification_writer.py` + `classification_csv.py`): reads pre-joined dicts with keys:
- `predicted_class` (mapped from `change_type`)
- `classification_confidence` (mapped from `confidence_score`)
- `is_reliable_prediction` (computed: confidence >= threshold)

**CSV output columns** (`classification_detailed_*.csv`):
`commit_hash, date, author, canonical_author, repository, predicted_class, confidence, is_reliable, message_preview, files_changed, insertions, deletions, lines_changed, primary_language, primary_activity, is_multilingual, branch, project_key, ticket_references`

**Narrative reports** (`narrative_classification.py`, `narrative_writer.py`) read `change_type` counts grouped by developer/week to generate text summaries.

The `QualitativeCommitData` Python dataclass in `qualitative/models/schemas.py` is used by the older NLP path; the LLM batch path writes directly to the SQLAlchemy ORM model in `database_metrics_models.py`.

---

## 5. How to Add `complexity` (1-5) Field End-to-End

### Files to change (in order)

#### Step 1: Add DB column
**File**: `src/gitflow_analytics/models/database_metrics_models.py`

```python
class QualitativeCommitData(Base):
    ...
    change_type            = Column(String, nullable=False)
    change_type_confidence = Column(Float, nullable=False)
    complexity             = Column(Integer, nullable=True)   # ADD: 1-5 scale, NULL if not assessed
    complexity_confidence  = Column(Float, nullable=True)     # ADD: optional confidence for complexity
    ...
```

Nullable is important — existing rows have no complexity value yet.

#### Step 2: Add DB migration
The project uses ad-hoc `ALTER TABLE` migrations in `src/gitflow_analytics/core/cache.py` (look for the `_apply_migrations` method or similar). Add:
```python
# In the migration list:
("ALTER TABLE qualitative_commits ADD COLUMN complexity INTEGER", "add complexity rating"),
("ALTER TABLE qualitative_commits ADD COLUMN complexity_confidence REAL", "add complexity confidence"),
```

Also update `SchemaVersionManager.CURRENT_SCHEMAS["qualitative"]["fields"]` in `core/schema_version.py` to include `"complexity"` and `"complexity_confidence"`.

#### Step 3: Update the batch prompt
**File**: `src/gitflow_analytics/qualitative/classifiers/llm/bedrock_client.py`

Update `_BATCH_SYSTEM_PROMPT` to ask for a `complexity` field:
```python
_BATCH_SYSTEM_PROMPT: str = (
    "You are a git commit classifier. For each commit, classify it into exactly one category "
    "and rate its complexity.\n\n"
    "Categories: feature, bugfix, maintenance, integration, content, media, localization\n\n"
    "Complexity: integer 1-5 where:\n"
    "  1=trivial (1-line fix, typo, rename)\n"
    "  2=simple (small focused change, 1-2 files)\n"
    "  3=moderate (multi-file change with clear intent)\n"
    "  4=complex (cross-cutting concern, significant logic change)\n"
    "  5=very complex (architectural change, multiple subsystems affected)\n\n"
    "Respond with a JSON array where each element has: "
    "category, confidence (0.0-1.0), reasoning (max 10 words), complexity (1-5).\n\n"
    "Example response for 3 commits:\n"
    '[{"category":"bugfix","confidence":0.95,"reasoning":"fixes crash in login flow","complexity":2},'
    '{"category":"feature","confidence":0.90,"reasoning":"adds new authentication system","complexity":4},'
    '{"category":"maintenance","confidence":0.85,"reasoning":"updates dependency versions","complexity":1}]\n\n'
    "IMPORTANT: Return ONLY the JSON array. No other text."
)
```

Also update the single-commit prompt in `prompts.py` (V3_CONTEXTUAL template) to request `complexity` in the response format:
```
Format: CATEGORY confidence reasoning complexity
```
And update `ResponseParser` to parse the additional field.

#### Step 4: Update JSON parsing / response parser
**File**: `src/gitflow_analytics/qualitative/classifiers/llm/bedrock_client.py` — `_validate_batch_items()`
**File**: `src/gitflow_analytics/qualitative/classifiers/llm/response_parser.py` — `parse_response()`

In `_validate_batch_items`, extract `complexity` from each item:
```python
def _validate_batch_items(self, items):
    valid = []
    for item in items:
        category = item.get("category", "maintenance")
        confidence = float(item.get("confidence", 0.5))
        reasoning = item.get("reasoning", "")
        complexity = item.get("complexity")          # ADD
        if complexity is not None:
            complexity = max(1, min(5, int(complexity)))  # clamp to 1-5
        valid.append({
            "category": category,
            "confidence": confidence,
            "reasoning": reasoning,
            "complexity": complexity,               # ADD
        })
    return valid
```

#### Step 5: Thread complexity through the result dict
**File**: `src/gitflow_analytics/classification/batch_classifier_impl.py` — `_classify_commit_batch_with_llm()`

When building `processed_results`, forward the `complexity` field:
```python
processed_results.append({
    "commit_hash": commit["commit_hash"],
    "category": predicted_category,
    "confidence": confidence,
    "complexity": llm_result.get("complexity"),    # ADD
    "method": "llm",
    "batch_id": batch_id,
})
```

#### Step 6: Store complexity in DB
**File**: `src/gitflow_analytics/classification/batch_classifier_impl.py` — `_store_commit_classification()`

```python
category   = classification_result["category"]
confidence = float(classification_result["confidence"])
method     = classification_result.get("method", "unknown")
complexity = classification_result.get("complexity")         # ADD

if qualitative:
    qualitative.change_type            = category
    qualitative.change_type_confidence = confidence
    qualitative.confidence_score       = confidence
    qualitative.processing_method      = method
    qualitative.analyzed_at            = datetime.now(timezone.utc)
    qualitative.complexity             = complexity           # ADD
else:
    qualitative = QualitativeCommitData(
        ...
        complexity=complexity,                               # ADD
        ...
    )
```

#### Step 7: Update the Python dataclass (NLP path)
**File**: `src/gitflow_analytics/qualitative/models/schemas.py`

```python
@dataclass
class QualitativeCommitData:
    ...
    change_type: str
    change_type_confidence: float
    complexity: Optional[int] = None          # ADD: 1-5 rating
    complexity_confidence: Optional[float] = None  # ADD
    ...

    def to_dict(self):
        return {
            ...
            "change_type": self.change_type,
            "change_type_confidence": self.change_type_confidence,
            "complexity": self.complexity,            # ADD
            "complexity_confidence": self.complexity_confidence,  # ADD
            ...
        }
```

#### Step 8: Expose complexity in CSV reports
**File**: `src/gitflow_analytics/reports/classification_csv.py` — `generate_detailed_csv_report()`

Add `complexity` to the headers list and the row-building loop:
```python
headers = [
    ...
    "predicted_class",
    "confidence",
    "complexity",          # ADD
    ...
]
...
row = [
    ...
    commit.get("predicted_class", ""),
    f"{commit.get('classification_confidence', 0):.3f}",
    commit.get("complexity", ""),                # ADD
    ...
]
```

---

## 6. Summary Table — Files That Need Changing

| File | Change |
|---|---|
| `models/database_metrics_models.py` | Add `complexity` (Integer) and `complexity_confidence` (Float) columns to `QualitativeCommitData` |
| `core/schema_version.py` | Add `"complexity"` to `CURRENT_SCHEMAS["qualitative"]["fields"]` |
| `core/cache.py` (or migration module) | Add `ALTER TABLE qualitative_commits ADD COLUMN complexity INTEGER` migration |
| `qualitative/classifiers/llm/bedrock_client.py` | Update `_BATCH_SYSTEM_PROMPT` to request `complexity`; update `_validate_batch_items()` to extract it |
| `qualitative/classifiers/llm/prompts.py` | Update V3_CONTEXTUAL template user prompt format line to include `complexity` |
| `qualitative/classifiers/llm/response_parser.py` | Update `parse_response()` to return `complexity` from single-commit responses |
| `classification/batch_classifier_impl.py` | Thread `complexity` from LLM result dict into `processed_results`; store it in `_store_commit_classification()` |
| `qualitative/models/schemas.py` | Add `complexity: Optional[int]` field to `QualitativeCommitData` dataclass |
| `reports/classification_csv.py` | Add `complexity` column to detailed CSV report |
| `reports/narrative_classification.py` | (Optional) Include complexity distribution in narrative text |

---

## 7. Key Design Notes

- **Nullable by default**: `complexity` should be `nullable=True` in SQLAlchemy so existing rows and fallback-classified commits (which go through rule-based fallback, not LLM) are unaffected.
- **Reclassify to backfill**: After adding the column and updating the prompt, run `gfa classify --reclassify` to backfill complexity ratings for already-classified commits. The UPSERT pattern in `_store_commit_classification()` handles this correctly.
- **Fallback path**: Rule-based fallback (`_fallback_classify_commit()`) does not call an LLM, so it will produce `complexity=None`. Accept this for now; a simple heuristic (based on files_changed count) could be added later.
- **Batch JSON parsing**: The LLM sometimes returns malformed JSON. `_parse_batch_json()` has two recovery strategies; the item validator must handle missing `complexity` keys gracefully (default to `None`).
- **Token budget**: Adding one integer field to a 25-commit batch response adds ~75 tokens. The current `batch_max_tokens = min(4096, n * 80)` formula should be updated to `n * 100` to accommodate the extra field.
