# v0.22.0 Community Posts

*Council Deliberation for Verification*

---

## Hacker News

### Show HN Post

**Title:** Show HN: LLM Council – Multi-model deliberation for code verification

**URL:** https://github.com/amiable-dev/llm-council

**Text (optional, for text posts):**

> We built an open-source system where multiple LLMs deliberate to verify code changes, inspired by academic peer review.
>
> The problem: Single-model code review has blind spots. One model says "looks good" while missing 2 of 4 issues.
>
> Our approach uses 3-stage deliberation:
>
> 1. **Stage 1**: Multiple models independently review the code
> 2. **Stage 2**: Anonymous peer ranking (models see "Response A, B, C" not "GPT-4, Claude, Gemini")
> 3. **Stage 3**: Chairman synthesizes a binary verdict (APPROVED/REJECTED) with confidence score
>
> The anonymization prevents prestige bias and self-promotion. Confidence is calculated from reviewer agreement—low agreement triggers "UNCLEAR" for human review instead of guessing.
>
> Exit codes (0/1/2) integrate directly into CI/CD pipelines.
>
> Technical details:
> - Async subprocess with streaming reads (DoS protection)
> - Borda count aggregation for rankings
> - Complete audit trail in `.council/logs/`
> - Configurable confidence threshold (default 0.7)
>
> Just released v0.22.0 with full council integration. Would love feedback from anyone doing AI-assisted code review.
>
> GitHub: https://github.com/amiable-dev/llm-council
> PyPI: `pip install llm-council-core`

---

## Reddit

### r/LocalLLaMA

**Title:** Multi-model deliberation for code verification - like peer review but with LLMs

**Body:**

> I've been working on LLM Council, an open-source project that uses multiple models to verify code changes through a deliberation process.
>
> **The problem I was trying to solve:**
>
> Single-model code review is unreliable. Different models have different blind spots—one catches SQL injection but misses XSS, another spots race conditions but overlooks CSRF. And they're often overconfident.
>
> **The solution: 3-stage deliberation**
>
> ```
> Stage 1: Parallel Reviews    → Multiple models review independently
> Stage 2: Anonymous Ranking   → Models rank reviews without knowing who wrote them
> Stage 3: Chairman Synthesis  → Final APPROVED/REJECTED verdict with confidence
> ```
>
> The key insight is **anonymous peer ranking**. Models see "Response A, B, C" instead of model names. This prevents:
> - Deferring to "prestigious" models
> - Self-promotion (ranking your own response higher)
> - Provider cliques
>
> **CI/CD integration:**
> - Exit 0 = PASS
> - Exit 1 = FAIL
> - Exit 2 = UNCLEAR (low confidence, needs human review)
>
> **Example usage:**
> ```bash
> pip install llm-council-core
> llm-council verify $(git rev-parse HEAD) --focus security
> ```
>
> Works with OpenRouter so you can mix models from different providers (GPT-4, Claude, Gemini, Llama, etc.).
>
> Just released v0.22.0 with the full council deliberation integration. Curious what you all think—anyone else experimenting with multi-model consensus for code tasks?
>
> GitHub: https://github.com/amiable-dev/llm-council

---

### r/MachineLearning

**Title:** [P] LLM Council: Ensemble deliberation for code verification with anonymous peer ranking

**Body:**

> **TL;DR:** Open-source system using multi-model deliberation (inspired by academic peer review) for code verification. Key innovation is anonymous peer ranking to prevent model bias.
>
> ---
>
> **Motivation**
>
> Single LLM evaluations suffer from:
> - Model-specific blind spots
> - Overconfidence despite uncertainty
> - Inconsistency across similar inputs
>
> Ensemble methods help, but naive voting doesn't capture the nuance of code review where different aspects matter (security, performance, correctness).
>
> **Architecture**
>
> We implemented a 3-stage deliberation pipeline:
>
> 1. **Stage 1 (Generation)**: N models independently review code changes
> 2. **Stage 2 (Evaluation)**: Each model ranks all reviews on a multi-dimensional rubric (accuracy, completeness, clarity, conciseness, relevance) - **crucially, reviews are anonymized as "Response A, B, C"**
> 3. **Stage 3 (Aggregation)**: Chairman model synthesizes rankings into binary verdict with confidence score
>
> **Why anonymization matters**
>
> Without it, we observed:
> - Models deferring to "GPT-4" or "Claude" regardless of response quality
> - Self-preference bias (models ranking their own responses higher)
> - Provider clustering effects
>
> With anonymization, evaluation correlates with actual response quality rather than model reputation.
>
> **Confidence calculation**
>
> Rather than arbitrary thresholds, confidence is derived from reviewer agreement:
> ```
> confidence = 1.0 - (pstdev(rubric_scores) / 4.5)
> ```
> Low agreement → low confidence → "UNCLEAR" verdict requiring human review.
>
> **Results**
>
> Anecdotally, the deliberation catches issues that individual models miss. We're working on formal benchmarks.
>
> **Links**
> - Paper/blog: https://github.com/amiable-dev/llm-council/blob/master/docs/blog/13-council-deliberation-verification.md
> - Code: https://github.com/amiable-dev/llm-council
> - PyPI: `pip install llm-council-core==0.22.0`
>
> Would appreciate feedback on the approach, especially from anyone working on LLM evaluation or ensemble methods.

---

### r/programming

**Title:** We built a code verification system where multiple AI models deliberate like academic peer reviewers

**Body:**

> Instead of asking one AI "is this code good?", we built a system where multiple models review code, anonymously rank each other's reviews, then synthesize a consensus verdict.
>
> **Why?**
>
> Single-model code review is like having one reviewer—you get one opinion with one set of blind spots. Model A catches the SQL injection but misses the race condition. Model B spots the memory leak but overlooks the auth bypass.
>
> **How it works:**
>
> 1. Multiple models review your code in parallel
> 2. Each model ranks the reviews (anonymized as "Response A, B, C" to prevent bias)
> 3. A "chairman" model synthesizes everything into APPROVED/REJECTED with a confidence score
>
> **CI/CD integration:**
>
> ```yaml
> - name: Verify code
>   run: |
>     llm-council verify ${{ github.sha }}
>     # Exit 0 = pass, 1 = fail, 2 = needs human review
> ```
>
> If models disagree significantly, it returns exit code 2 ("unclear") instead of guessing—flags for human review.
>
> **The anonymization is key.** Without it, models tend to defer to "prestigious" model names or rank their own responses higher. With anonymization, they evaluate based on actual quality.
>
> Open source, works with any OpenRouter-compatible models.
>
> GitHub: https://github.com/amiable-dev/llm-council
>
> Curious if anyone else is doing multi-model consensus for dev tooling?

---

### r/devops / r/DevOps

**Title:** AI code verification with proper CI/CD exit codes (0=pass, 1=fail, 2=unclear)

**Body:**

> Released v0.22.0 of LLM Council with a feature I think DevOps folks will appreciate: **structured exit codes for AI verification.**
>
> **The problem with most AI code review tools:**
> - Binary pass/fail when the AI isn't confident
> - No audit trail
> - Single model = single point of failure
>
> **Our approach:**
>
> Multiple models deliberate (like peer review), then return:
> - `Exit 0`: PASS - models agree, high confidence, ship it
> - `Exit 1`: FAIL - blocking issues found
> - `Exit 2`: UNCLEAR - low confidence, **needs human review**
>
> That exit code 2 is crucial. Instead of the AI guessing when it's uncertain, it explicitly flags for human review.
>
> **GitHub Actions example:**
>
> ```yaml
> - name: Verify code changes
>   run: |
>     set +e
>     llm-council verify ${{ github.sha }} --focus security
>     exit_code=$?
>     set -e
>
>     case $exit_code in
>       0) echo "Passed" ;;
>       1) echo "Failed"; exit 1 ;;
>       2) gh pr comment --body "Needs human review"; exit 0 ;;
>     esac
> ```
>
> **Audit trail:**
> ```
> .council/logs/2025-01-15T10-30-00-abc123/
> ├── request.json
> ├── stage1.json   # Individual reviews
> ├── stage2.json   # Peer rankings
> ├── stage3.json   # Final synthesis
> └── result.json   # Verdict + confidence
> ```
>
> Every decision is logged and reproducible.
>
> *Note: The `gh` command requires [GitHub CLI](https://cli.github.com/) to be installed.*
>
> GitHub: https://github.com/amiable-dev/llm-council
> `pip install llm-council-core`

---

## Cross-posting Notes

1. **Wait 24-48 hours between posts** to different subreddits to avoid spam flags
2. **Engage with comments** - answer questions, take feedback seriously
3. **Don't be defensive** - if someone points out a flaw, acknowledge it
4. **HN timing**: Post between 6-9am PT for best visibility
5. **Reddit timing**: Post during US business hours (9am-12pm ET)

## Subreddit-specific tips

| Subreddit | Tone | What they care about |
|-----------|------|---------------------|
| r/LocalLLaMA | Friendly, practical | Self-hosting, model comparisons, real-world usage |
| r/MachineLearning | Academic, technical | Novel methods, benchmarks, reproducibility |
| r/programming | Skeptical, pragmatic | Does it actually work? Show don't tell |
| r/devops | Ops-focused | Integration, reliability, observability |
| HN | Technical, concise | Interesting problems, elegant solutions |
