Context Engineering Multi-Agent Framework

CEMAF Architecture Atlas

A ground-truth tour of the framework — from the dependency tiers down to the single node hot-path where every feature is a station. Each view is drawn from src/cemaf/, not the ideal.

40 modules 2 layers, one-way arrow 2 dispatch seams 23 RuntimeServices deps protocol-first · BYO-X

Dependency tiers

Modules placed by measured import fan-in — the real coupling, not the org chart. Each tier depends only on the tiers below it. Hover a tier to focus it.

Foundation Shared fabric Capabilities Orchestration Self-hosting (Layer 2)
Layer 2 · Self-HostingCEMAF's first client — consumes the base via contracts, never imported back
metaauditknowledge
▲   consumes via Protocol only (one-way)   ▲
Tier 3 · Orchestrationtop of Layer 1 · fan-out 14
orchestrationresolvers/interceptorsbootstrapRuntimeServices
Tier 2 · Capabilitieswork units · quality/safety · integrations
agentstoolsskillssandboxblueprintevalscounciliterationresiliencecitationmoderationvalidationgenerationstreamingmcprlmcachepersistenceingestionstatecatalogsecuritytrustimprovementschedulerreplaydocs_api
Tier 1 · Shared fabricthe substrate every capability builds on
context (10)observability (10)events (9)llm (8)retrieval (7)memory (7)
Tier 0 · Foundationimported by nearly everything
core (fan-in 38)config (fan-in 18)

Result[T] · NewType ids · enums · utc_now · Settings · provider registry

The one-way boundary, precisely

No Layer-2 package is a structural dependency of the base. Three import sites cross down, with differing mechanisms: orchestration → knowledge.protocols.KnowledgeGraph (runtime Protocol), security → audit.protocols.AuditLog (TYPE_CHECKING only), and security → audit.models (a lazy, in-method concrete import that dodges a parse-time cycle). No base module imports meta at all.

The node hot-path

Every node travels one path: ContextNodeExecutor.execute_node. Features aren't branches — they're stations on this line. An unwired interceptor pipeline is a no-op; behaviour is byte-identical to running without it.

1

Resolve

resolvers/ · first-match-wins
CouncilResolver → NodeComplete (verdict steers DAG, agent never runs) Auction / Static → RunAgent
2

Prep context

recall global_memory (MemoryManager) → compile under TokenBudget (ContextCompiler)
3

PRE interceptors · run_pre

enrich context, or short-circuit
ACCEPT → continueREJECT → fail, agent never runs
4

Recovery loop  ≤ max_recovery_attempts (default 2)

inject last MAX_VISIBLE_HINTSagent.run → record ProvenanceLink → ingest to session memory → POST
ACCEPT → success REJECT → gate_rejected RECOVER + budget → re-run with hint ↺ RECOVER + spent → downgrade to REJECT
5

Emit

DAGExecutor → TASK_COMPLETED / TASK_FAILED · payload carries recovery_attempts, gate_rejected

The two dispatch seams

All extensibility lives here. Both are split single-method @runtime_checkable Protocols — a POST-only interceptor needn't implement pre.

Seam 1 · NodeResolver chain orchestration/resolvers/

First matches() wins → ResolveOutcome = RunAgent | NodeComplete. Add a node kind = register a resolver, not edit code.

CouncilResolvermatches config["council"] → deliberate, vote, return verdictSPEC-10
AuctionResolvermatches config["capability"] only when an agent_selector is wired; else falls throughSPEC-09
StaticRefResolveralways matches → RunAgent(node.ref_id) · universal fallbackbase

Seam 2 · Interceptor spine interceptors/ SPEC-01a

Ordered PRE → execute → POST. Empty = no-op.

PreInterceptorpre() → ACCEPT (enrich ctx) | REJECTPRE
PostInterceptorpost() → ACCEPT | REJECT | RECOVERPOST
GateEvalInterceptorruns Evaluators on output · on_failure = REJECT (block) | RECOVER (retry + hint)gate

Agentic behaviours

An Agent[GoalT, ResultT] is a typed protocol resolved from the registry. Single agents, competitive auctions, and deliberative councils are three ways the engine decides who answers — plus the self-hosting meta-agents that build the framework itself.

Built-in agentsagents/

Librarian · Researcher · Summarizer · Writer — each a typed Agent[Goal,Result], registered in the AgentRegistry.

Meta-agents (Layer 2)meta/agents.py

ArchitectAgent · AgentSynthesizer · AuditAgent · KnowledgeGraphAgent · DreamAgent · SolutionDesignerAgent — standard Agent[Goal,Result] impls that introspect & extend CEMAF.

Auction selection SPEC-09

N agents bid on a capability; deterministic max-bid wins. Capabilities: RESEARCH · SUMMARIZE · WRITE · LIBRARY · QUALITY.

score = 0.5·match + 0.3·(1−load) + 0.2·budget_headroom

Tie-break is deterministic; absent a selector, nodes resolve statically.

Agent council SPEC-10

N members deliberate on one question; a pluggable VoteAggregator decides. Methods: MAJORITY · WEIGHTED · QUORUM · UNANIMOUS.

Multi-round (rounds=N): round 2+ broadcasts prior opinions; members may revise; early-stops when the tally settles. The verdict becomes the node output and steers the DAG.

The capability substrate

Agents act through Tools (@tool, schema-validated) and Skills (multi-tool kits). The polyglot coding kit runs inside a ShellSandbox (cwd-confined, time/output-bounded, network-screened). A bounded IterationLoop (SPEC-08) parses pytest/ruff/mypy failures into FailureSignals and re-attempts until tests pass — feedback as a first-class control structure.

Context lifecycle

Context is the asset; tokens are the cost; provenance is the audit trail. The Context object is immutable (copy-on-write) — every mutation carries a typed ContextPatch source.

① Ingest
Adapters format / compress / prioritise raw data into ContextSources.
ingestion/
② Recall
MemoryManager pulls relevant items by scope + topic before the agent runs.
memory/
③ Compile
ContextCompiler selects under TokenBudget — by priority, not recency.
context/compiler.py
④ Use
The compiled slice is what agent.run sees; a ProvenanceLink records it.
orchestration/
⑤ Ingest result
Output written back to session memory; events fire.
memory/session.py
⑥ Extract
On dispose, distil SESSION → PROJECT for the next run.
memory/extraction

Selection algorithms

Greedy · Knapsack · Optimal — drop from the bottom of the priority order when over budget. Exclusions are tracked with a reason (BUDGET_EXCEEDED, LOW_PRIORITY).

Memory scopes & tiers

Scopes: SESSION · PROJECT · USER · GLOBAL. Tiered store L0/L1/L2 progressive recall; semantic + episodic; dedup on write.

Context types

RESOURCE · MEMORY · SKILL — each with behaviour rules (cacheable, shareable, compressible, TTL, default priority).

Provenance

Every patch records source · reason · correlation_id; every node run records a ProvenanceLink. Silent mutation is forbidden.

Runtime & the reactive plane

One composition root wires ~23 injectable dependencies; the executor runs the DAG and emits events. Quality, audit, and learning subscribe asynchronously — off the hot path.

create_executor() base root

Builds the ContextNodeExecutor (threads all 23 RuntimeServices deps incl. max_recovery_attempts), assembles the resolver chain [Council, Auction?, Static], subscribes online-eval + quality-police when an event bus is present, wraps in an InstrumentedDAGExecutor when a tracer is present.

create_meta_executor() Layer-2 root

Auto-builds AuditTrail from the EventBus and a KnowledgeGraph from the MemoryManager, registers the meta-agents/tools, then delegates to create_executor — so Layer 2 inherits every base primitive (incl. the RECOVER budget) for free.

DAGExecutor  —emits→  EventBus  (pub/sub)
TASK_STARTED · TASK_COMPLETED · TASK_FAILED · DAG_STARTED · DAG_COMPLETED · DAG_CHECKPOINT · SYSTEM_ERROR

OnlineEvalPipeline

consumes task events, scores output, re-emits EVAL_COMPLETED. evals/online.py

QualityPolice

subscribes to EVAL_COMPLETED (not raw task events); z-score anomaly + halt gate. evals/police.py

AuditTrail

EventBus → AuditEntry; feeds the KnowledgeGraph. audit/subscriber.py

BlueprintHarvester

distils high-scoring runs into reusable blueprints. blueprint/harvest.py

Memory subscriber

persists memory events. events/memory_subscriber.py

Two-hop chain: the executor emits task events → OnlineEvalPipeline re-emits EVAL_COMPLETED → QualityPolice subscribes to that. recovery_attempts and gate_rejected ride the task payload so audit & harvest correlate recovery without re-walking results.

The architecture in five sentences

If you remember nothing else.

1 Protocol-first

Every integration point (LLMClient, VectorStore, MemoryManager, Evaluator, NodeResolver, Interceptor, KnowledgeGraph, AuditLog) is a @runtime_checkable Protocol; implementations are structural — bring your own.

2 One hot-path, many stations

Features are PRE/POST interceptor stations and resolver entries on a single execute_node flow — not bespoke branches.

3 Immutable data + provenance

Context is copy-on-write; every mutation carries a ContextPatch source; every node run records a ProvenanceLink.

4 Budgeted, priority-selected context

TokenBudget + ContextCompiler drop by priority, not recency, before each agent call.

5 Strict layering

Layer 2 self-hosting consumes the base only through contracts; the base never imports it — so CEMAF introspects itself without a cycle.