Python library for profile-conditioned RAG readiness

Correct memories are not always transferable.

HandoverGap detects missing tacit context in otherwise correct RAG memories, checks the requested profile and task context, and blocks unsafe transfer with clarification questions.

HandoverGap Streamlit demo showing Japanese default UI, local sample mode, and Live OpenAI plus TiDB option

Install And First Run

1. Install

Use the latest tested release

The core package has no TiDB, OpenAI, or Streamlit runtime requirement.

pip install handovergap==0.1.8
2. Inspect one case

Run the built-in demo

See a valid memory that is unsafe to use without the selected profile's required context.

handovergap demo
3. Measure behavior

Compare baselines

Run the bundled synthetic benchmark against Naive, Hybrid, and HandoverGap RAG.

handovergap evaluate --compare

Core Concepts

HandoverGap is a gate that runs before a RAG answer, agent action, or operational decision. It checks whether the selected profile has enough context to safely use a retrieved memory.

Concept Meaning Example
memory The retrieved note, decision, runbook line, CRM note, or agent memory. Use CSV for this release; API support is deferred.
profile The role or operating mode that wants to use the memory. CS, Engineer, Sales, IncidentCommander, LegalReviewer, AI Agent.
slot A required piece of context for that profile. authority, fallback_plan, escalation_path, customer_message.
gap A missing or weak slot that makes transfer unsafe or incomplete. Customer-safe wording is not available.
question The follow-up question that turns a blocked answer into an actionable next step. What customer-facing message should be used?

The built-in handover examples are only presets. The reusable mechanism is profile-conditioned readiness: define what a role must know, then block or ask when those requirements are missing.

CLI Usage

Detect gaps for a profile

`detect` checks a built-in fictional scenario against a profile and task context. `CS`, `Engineer`, and `Sales` are packaged presets, not a fixed industry taxonomy.

handovergap detect --scenario S001 --profile CS

Evaluate slot-filling sensitivity

The holdout stress profiles simulate how semantic slot filling can under-fill or over-fill required context.

handovergap evaluate --dataset holdout --stress-filling

Retrieve slot evidence

Hybrid retrieval combines vector-style similarity and full-text matching before the gate decides whether the profile has enough context.

handovergap retrieve-evidence \
  --scenario S001 \
  --profile CS \
  --slot communication_status \
  --mode hybrid

Generate a report

Produce a reproducible Markdown report for the bundled mini, holdout, adversarial, and sanitized splits.

handovergap report --dataset all \
  --output reports/evaluation-latest.md

Custom Profiles

Built-in profiles include CS, Engineer, and Sales. They are examples, not a fixed industry taxonomy. Use a YAML profile file when your team needs a different gate, such as incident response, legal review, renewal handoff, or an autonomous agent action check.

Define required slots

profiles:
  IncidentCommander:
    required_slots:
      - slot_name: blast_radius
        gap_type: blast_radius_gap
        description: Impacted users, systems, and time window are not explicit.
        question: Which users, systems, and time window are impacted?
        severity: HIGH
        high_risk: true
      - slot_name: rollback_owner
        gap_type: rollback_owner_gap
        description: The person or team authorized to trigger rollback is not explicit.
        question: Who is authorized to trigger rollback?
        severity: HIGH
        high_risk: true
      - slot_name: customer_message
        gap_type: customer_message_gap
        description: Customer-safe incident wording is not available.
        question: What customer-facing message should be used?
        severity: MEDIUM

Use the profile from CLI

handovergap detect \
  --scenario S001 \
  --profile IncidentCommander \
  --profile-file examples/profiles/incident_readiness.yml

The same profile can be used in Python with TransferabilityGate.from_profile_file(...).

What It Adds To RAG

Approach Optimizes Misses
Naive RAG Relevant memory retrieval Whether a profile can safely act
Hybrid RAG Evidence and risk warnings Profile-specific missing context
Context engineering Better prompt and context packaging A durable audit trail for withheld answers
HandoverGap Profile-required slot checks before answering Production use still needs organization-specific annotation

Local And Live Demo

Default

Local bilingual Streamlit demo

The UI opens in Japanese by default and can switch to English. Local-sample mode runs the real deterministic detector on fictional operational cases.

pip install "handovergap[demo]"
handovergap serve
Optional

Live OpenAI + TiDB audit demo

Live OpenAI + TiDB mode asks the selected OpenAI model to fill profile-required slots, runs HandoverGap on those slots, and persists audit rows to TiDB.

pip install "handovergap[live]"
export OPENAI_API_KEY="..."
export TIDB_HOST="..."
export TIDB_USER="..."
export TIDB_PASSWORD="..."
handovergap serve

Live dependencies are optional by design. First-run usage, CI, and package import do not require an OpenAI key or TiDB account.

TiDB Audit Query

TiDB is used as more than a vector store. HandoverGap stores the decision path so a blocked answer can be explained across SQL rows, vector-backed evidence, JSON retrieval metadata, gaps, and questions.

handovergap audit-sql

The generated SQL joins transfer_assessments, memory_items, context_gaps, slot_fill_attempts, source_events, and clarification_questions. It answers: which required slot was missing, what evidence was checked, and what should be asked before the answer is used.

Live TiDB validation Observed
Dataset sanitized
Persisted scenarios 6
Audit query rows 7
p50 latency 48.408 ms
p95 latency 1510.413 ms

This is a 10-iteration TiDB Cloud validation result for the audit query path, not a load-test claim. The p95 includes cold or variable cloud latency.

Generated workload on TiDB Observed
Generated scenarios 100
Slot-fill attempts 567
Context gaps / questions 254 / 254
Audit query rows 254
p50 / p95 latency 38.818 ms / 574.713 ms

Evaluation Snapshot

These are deterministic results from the bundled synthetic 20-scenario mini dataset. They demonstrate reproducible behavior, not production accuracy.

Method Tacit Gap Recall Unsafe Transfer Prevention Question Coverage Safe Transfer Allowance Blocked Precision
naive_rag 0.00 0.00 0.00 1.00 0.00
hybrid_rag 0.21 0.59 0.21 0.67 0.91
handovergap 1.00 1.00 1.00 1.00 1.00

Live gpt-5-mini with the tuned gpt5_strict prompt reached tacit gap recall 1.00, unsafe transfer prevention 1.00, safe transfer allowance 1.00, and blocked precision 1.00 on the holdout protocol. See results notes.

The adversarial split keeps recall low at 0.38 but reduces false clarification to 0.00 after evidence-backed slot reconciliation. The sanitized split uses field-realistic anonymized CRM, incident, runbook, and deal-review style notes without real company data.

Python API

Run the gate in code

from handovergap import TransferabilityGate

gate = TransferabilityGate()
result = gate.check(
    memory="Use CSV for this release; API support is deferred.",
    profile="CS",
    task_context="Answer customer questions about the workaround.",
    evidence=["CSV workaround approved for the release."],
    provided_slots=["scope"],
    evidence_slots=["scope"],
)

print(result.transferability_status)
print(result.gaps)
print(result.questions)

Run with a custom profile

from handovergap import TransferabilityGate

gate = TransferabilityGate.from_profile_file(
    "examples/profiles/incident_readiness.yml"
)
result = gate.check(
    memory="The checkout incident is mitigated by disabling the new queue worker.",
    profile="IncidentCommander",
    task_context="Decide whether customer-facing mitigation is complete.",
    evidence=["Rollback owner is on-call SRE. Blast radius is checkout only."],
    provided_slots=["rollback_owner"],
    evidence_slots=["blast_radius"],
)

for question in result.questions:
    print(question.question)

Prepare TiDB schema

Print the bundled schema locally or create it through the optional TiDB store.

pip install "handovergap[tidb]"
handovergap schema --dialect tidb

Stable API Contract

Surface Stable fields
TransferabilityGate.check(...) inputs memory, profile, task_context, evidence, provided_slots, evidence_slots, scenario_id, and memory_type.
DetectionResult outputs transferability_status, transferability_score, gaps, questions, scenario_id, profile, memory, and task_context.
Status values transferable, needs_clarification, and blocked.

The core API does not require OpenAI or TiDB. Those integrations are optional paths for semantic slot filling and audit persistence.

RAG Integration Pattern

Place HandoverGap after retrieval and before answer generation. Let your normal retriever collect candidate memory and evidence, then let the gate decide whether to answer, ask, or block.

Before answering

from handovergap import TransferabilityGate

gate = TransferabilityGate()

retrieved_memory = retriever.search(user_question)
evidence = retriever.search_evidence(user_question)

result = gate.check(
    memory=retrieved_memory.text,
    profile="CS",
    task_context="Answer the customer's question without overpromising.",
    evidence=[item.text for item in evidence],
    provided_slots=retrieved_memory.slots,
    evidence_slots=[slot for item in evidence for slot in item.slots],
)

if result.transferability_status != "transferable":
    return {
        "status": result.transferability_status,
        "reason": [gap.description for gap in result.gaps],
        "questions": [q.question for q in result.questions],
    }

return llm.answer(user_question, context=retrieved_memory.text)

What the gate returns

Field Use it for
transferability_status Route the flow: answer, ask for clarification, or block.
transferability_score Show a readiness score or monitor profile tuning.
gaps Explain which profile-required context is missing.
questions Create the next action instead of hallucinating missing context.

This pattern works for human handoff, support replies, incident response, sales renewal notes, and agent memory checks. The domain changes; the gate contract stays the same.

What This Page Helps Validate

PyPI first-run experience

Users can install the core package and run the MVP commands without secrets.

Demo clarity

The page shows the actual Japanese-default demo and its optional live path.

TiDB-specific learning

TiDB is used as a single audit store for slot attempts, evidence, gaps, questions, and transfer decisions.