Correct memories are not always transferable.

Install And First Run

pip install handovergap==0.1.15

handovergap demo

handovergap evaluate --compare

Core Concepts

HandoverGap is a gate that runs before a RAG answer, agent action, or operational decision. It checks whether the selected profile has enough context to safely use a retrieved memory.

CLI Usage

handovergap detect --scenario S001 --profile CS

handovergap evaluate --dataset holdout --stress-filling

handovergap retrieve-evidence \
  --scenario S001 \
  --profile CS \
  --slot communication_status \
  --mode hybrid

handovergap report --dataset all \
  --output reports/evaluation-latest.md

Custom Profiles

Built-in profiles include CS, Engineer, and Sales. They are examples, not a fixed industry taxonomy. Use a YAML profile file when your team needs a different gate, such as incident response, legal review, renewal handoff, or an autonomous agent action check.

profiles:
  IncidentCommander:
    required_slots:
      - slot_name: blast_radius
        gap_type: blast_radius_gap
        description: Impacted users, systems, and time window are not explicit.
        question: Which users, systems, and time window are impacted?
        severity: HIGH
        high_risk: true
      - slot_name: rollback_owner
        gap_type: rollback_owner_gap
        description: The person or team authorized to trigger rollback is not explicit.
        question: Who is authorized to trigger rollback?
        severity: HIGH
        high_risk: true
      - slot_name: customer_message
        gap_type: customer_message_gap
        description: Customer-safe incident wording is not available.
        question: What customer-facing message should be used?
        severity: MEDIUM

handovergap profiles validate \
  examples/profiles/incident_readiness.yml

handovergap detect \
  --scenario S001 \
  --profile IncidentCommander \
  --profile-file examples/profiles/incident_readiness.yml

What It Adds To RAG

Local And Live Demo

pip install "handovergap[demo]"
handovergap serve

pip install "handovergap[live]"
export OPENAI_API_KEY="..."
export TIDB_HOST="..."
export TIDB_USER="..."
export TIDB_PASSWORD="..."
handovergap serve

TiDB Audit Query

TiDB is used as more than a vector store. HandoverGap stores the decision path so a blocked answer can be explained across SQL rows, vector-backed evidence, JSON retrieval metadata, gaps, and questions.

handovergap audit-sql

The generated SQL joins transfer_assessments, memory_items, context_gaps, slot_fill_attempts, source_events, and clarification_questions. It answers: which required slot was missing, what evidence was checked, and what should be asked before the answer is used.

This is a 10-iteration TiDB Cloud validation result for the audit query path, not a load-test claim. The p95 includes cold or variable cloud latency.

Evaluation Snapshot

These are deterministic results from the bundled synthetic 20-scenario mini dataset. They demonstrate reproducible behavior, not production accuracy.

Live gpt-5-mini with the tuned gpt5_strict prompt reached tacit gap recall 1.00, unsafe transfer prevention 1.00, safe transfer allowance 1.00, and blocked precision 1.00 on the holdout protocol. See results notes.

The adversarial split keeps recall low at 0.38 but reduces false clarification to 0.00 after evidence-backed slot reconciliation. The sanitized split uses field-realistic anonymized CRM, incident, runbook, and deal-review style notes without real company data.

Python API

from handovergap import TransferabilityGate

gate = TransferabilityGate()
result = gate.check(
    memory="Use CSV for this release; API support is deferred.",
    profile="CS",
    task_context="Answer customer questions about the workaround.",
    evidence=["CSV workaround approved for the release."],
    provided_slots=["scope"],
    evidence_slots=["scope"],
)

print(result.transferability_status)
print(result.gaps)
print(result.questions)

from handovergap import TransferabilityGate

gate = TransferabilityGate.from_profile_file(
    "examples/profiles/incident_readiness.yml"
)
result = gate.check(
    memory="The checkout incident is mitigated by disabling the new queue worker.",
    profile="IncidentCommander",
    task_context="Decide whether customer-facing mitigation is complete.",
    evidence=["Rollback owner is on-call SRE. Blast radius is checkout only."],
    provided_slots=["rollback_owner"],
    evidence_slots=["blast_radius"],
)

for question in result.questions:
    print(question.question)

pip install "handovergap[tidb]"
handovergap schema --dialect tidb

Stable API Contract

The core API does not require OpenAI or TiDB. Those integrations are optional paths for semantic slot filling and audit persistence.

Evidence-To-Slot Mapping

Raw retrieved evidence is not automatically treated as complete context. Integrations should pass provided_slots for context explicit in the memory and evidence_slots for context supported by retrieved evidence.

from handovergap import TransferabilityGate, map_evidence_slots_by_keywords

evidence = [
    "Customer notice was sent on Monday with approved wording.",
    "Support can answer standard questions, but must not promise API dates.",
    "If CSV import fails, use the manual upload fallback and escalate in the support channel.",
]
slot_keywords = {
    "communication_status": ["notice was sent", "customer notice"],
    "authority": ["can answer", "must not promise"],
    "fallback_plan": ["fallback", "manual upload"],
    "escalation_path": ["escalate", "support channel"],
    "customer_facing_wording": ["approved wording"],
}

result = TransferabilityGate().check(
    memory="Use CSV for this release; API support is deferred.",
    profile="CS",
    task_context="Answer customer questions without overpromising.",
    evidence=evidence,
    provided_slots=["scope"],
    evidence_slots=map_evidence_slots_by_keywords(evidence, slot_keywords),
)

Manual review, deterministic rules, and optional LLM slot filling can all feed the same evidence_slots contract. The core API does not require an LLM.

Slot Filling Modes

HandoverGap is a slot-based readiness gate, not an LLM-only extractor. Use reviewed slots, deterministic rules, or optional LLM extraction depending on the trust boundary of your workflow.

handovergap evaluate --compare --slot-fill-mode user_provided
handovergap evaluate --slot-fill-mode optional_llm --model gpt-example --prompt-profile strict

RAG Integration Pattern

Place HandoverGap after retrieval and before answer generation. Let your normal retriever collect candidate memory and evidence, then let the gate decide whether to answer, ask, or block.

== first retrieval ==
provided_slots=scope
evidence_slots=communication_status,authority,customer_facing_wording
status=blocked action=block
gaps=fallback_plan,escalation_path
questions:
- 想定外の場合の代替手段は何ですか？
- 問題が起きた場合のエスカレーション先は誰ですか？
safe_context=withheld

== after retrieving runbook evidence ==
provided_slots=scope
evidence_slots=communication_status,authority,fallback_plan,escalation_path,customer_facing_wording
status=transferable action=answer
gaps=none
questions=none
safe_context=available

OpenAI slot filling can replace deterministic keyword mapping, and TiDB can persist the route audit trail. Neither is required for the core runtime.

python examples/end_to_end_integration.py

python examples/langchain_gate.py

python examples/llamaindex_gate.py

Full copyable recipes are in docs/30_rag_integration_recipes.md.

from handovergap import TransferabilityGate

gate = TransferabilityGate()

retrieved_memory = retriever.search(user_question)
evidence = retriever.search_evidence(user_question)

result = gate.check(
    memory=retrieved_memory.text,
    profile="CS",
    task_context="Answer the customer's question without overpromising.",
    evidence=[item.text for item in evidence],
    provided_slots=retrieved_memory.slots,
    evidence_slots=[slot for item in evidence for slot in item.slots],
)

if result.transferability_status != "transferable":
    return {
        "status": result.transferability_status,
        "reason": [gap.description for gap in result.gaps],
        "questions": [q.question for q in result.questions],
    }

return llm.answer(user_question, context=retrieved_memory.text)

Product Routing Semantics

from handovergap import TransferabilityGate, route_transferability_result

result = TransferabilityGate().check(
    memory="Use CSV for this release; API support is deferred.",
    profile="CS",
    task_context="Answer customer questions without overpromising.",
    provided_slots=["scope"],
)

route = route_transferability_result(result, safe_context=result.memory)

return {
    "status": route.status,
    "action": route.action,
    "reason": route.reason,
    "questions": route.questions,
    "safe_context": route.safe_context,
}

safe_context is only returned for transferable results. Ask and block routes omit it so applications do not expose context that is not ready to use.

Actionable Errors

HandoverGap reports common configuration mistakes with fix-oriented messages while avoiding raw evidence payloads in exception text.

What This Page Helps Validate