{% extends "base.html" %} {% block title %}Observability & Monitoring - Memorizz{% endblock %} {% block head %}{% endblock %} {% block content %}
{% if ui_read_only %}Read-only provider{% else %}Writable provider{% endif %} Content mode: {{ trace_content_mode or 'full' }} {% if trace_query_metadata %} Query: {% if trace_query_metadata.provider_native %}provider-native{% else %}compatibility fallback{% endif %} {% if trace_query_metadata.truncated %}Window truncated{% endif %} {% endif %}
Incident Finder

Incident Finder

{% if trace_permissions and trace_permissions['account.resolve'] %}
{% endif %}

Search uses bounded, authorized event pages. Account addresses are resolved transiently, not stored in traces.

Host integration capabilities · {% if not trace_capabilities['account.resolve'].configured or not trace_capabilities['artifact.lookup'].configured or not trace_capabilities['replay.create'].configured %}setup required{% else %}configured{% endif %} {% for permission, capability in trace_capabilities.items() %}

{{ capability.setup }}: {{ 'Available' if capability.available else 'Not configured' if not capability.configured else 'Not permitted for this role' }}.{% if not capability.configured %} Supply a tenant-aware {{ capability.setup }} callback to create_app. No host lookup will run until configured.{% endif %}

{% endfor %}

Host source/artifact/parser/delivery recording is separate from these lookup hooks. Missing historical links cannot be reconstructed as facts.

{% if error %}
! {{ error }}
{% endif %}
Agent and thread navigator

Agents

{% if selected_agent_id %} {% for key, value in selection_urls.params.items() %}{% if key != 'agent_id' %}{% endif %}{% endfor %} {% endif %}
{% if agent_rows %}
{% for row in agent_rows %} {% set is_parent_expanded = (row.agent_id == selected_agent_id) %} {% for thread_row in row.thread_rows %} {% endfor %} {% endfor %}
Agent ID Mode Memories Tools Trace Events Last Activity
{{ row.name }}
{{ row.agent_id[:10] }}... {{ row.mode }} {{ row.memory_count }} {{ row.tool_count }} {{ row.event_count }} {{ row.last_activity }}
Thread {{ thread_row.thread_id|truncate(24) }}
{{ thread_row.thread_id|truncate(18) }}
thread
{{ thread_row.memory_id|truncate(16) }}
-
{{ thread_row.event_count }}
{{ thread_row.last_activity }}
{% else %}
TRACE

No matching agents

Try a different search query.

{% endif %}
{% if selected_agent %}
Agent {{ selected_agent_name or 'Agent' }}
Agent ID {{ selected_agent.agent_id }}
{% if selected_thread_row %}
Thread ID {{ selected_thread_row.thread_id }}
Thread Memory {{ selected_thread_row.memory_id }}
{% endif %} {% if trace_summary %}
Trace Events {{ trace_summary.event_count }} events · {{ trace_summary.bundle_count }} bundles
Latest {{ trace_summary.latest_timestamp }}
{% endif %}
{% if trace_analysis %}

Trace Insights

Read-only analysis

{{ trace_analysis.methodology }}

{% if trace_analysis_export_url %}
Export JSON {% if not ui_read_only %}
{% if selected_thread_id %}{% endif %} {% if selected_thread_memory_id %}{% endif %}
{% endif %}
{% endif %}
{{ trace_analysis.coverage.headline }}

Stored-event read: {{ trace_analysis.coverage.read_completeness }} · Instrumentation: {{ trace_analysis.coverage.instrumentation_coverage }} · Outcome evidence: {{ trace_analysis.coverage.outcome_verification }}.

{{ trace_analysis.coverage.bundle_count if trace_analysis.coverage.bundle_count is not none else 'Unknown' }} bundle records · {{ trace_analysis.summary.events_analyzed }} normalized events · {{ trace_analysis.coverage.normalization_errors or 0 }} normalization errors · Duplicates removed: {{ trace_analysis.coverage.duplicates_removed if trace_analysis.coverage.duplicates_removed is not none else 'unknown in selected scope' }}

{% if trace_analysis.coverage.coverage != 'complete' %}

Counts are observed lower bounds. Missing evidence does not establish zero activity or task success.

{% endif %} {% if not trace_analysis.coverage.profiles_declared %}

Coverage describes loaded events. The host has not declared required artifact or delivery stages.

{% endif %}

Stores: {{ trace_analysis.coverage.store_selection|join(', ') }} · Source rows by store: {{ trace_analysis.coverage.source_counts|tojson }}. Counts describe recorded evidence, not task-level activity.

{% if trace_analysis.coverage.parent_query_metadata %}

Deduplication and query diagnostics apply to the parent snapshot; the selected-root source totals use recorded provenance.

{% endif %} {% for missing in trace_analysis.coverage.missing_stages %}

Missing {{ missing.stage }} ({{ missing.profile }})

{% endfor %}
Trace volume {{ trace_analysis.summary.events_analyzed }} Events analyzed {{ trace_analysis.summary.threads_analyzed }} thread{% if trace_analysis.summary.threads_analyzed != 1 %}s{% endif %} · {{ trace_analysis.summary.context_provenance_events }} provenance events
Verified outcomes {% if trace_analysis.summary.verified_outcomes %}{{ trace_analysis.summary.verified_outcome_successes }}/{{ trace_analysis.summary.verified_outcomes }}{% else %}No recorded evidence{% endif %} Successful outcomes {{ trace_analysis.summary.verified_outcome_failures }} failed · {{ trace_analysis.summary.verified_negative_feedback }} negative feedback
Context integrity {{ trace_analysis.summary.page_context_mismatches }} Page / thread mismatches {{ trace_analysis.summary.missing_current_page_grounding }} missing grounding · {{ trace_analysis.summary.unverified_page_ownership }} unverified ownership
Memory supply {{ trace_analysis.summary.memory_supplied }} Items supplied to the model {{ trace_analysis.summary.memory_candidates }} retrieved candidates · {{ trace_analysis.summary.memory_explicit_references }} explicit references · {{ trace_analysis.summary.memory_degraded_turns }} degraded
Tool health {% if trace_analysis.metric_states.tool_calls.state == 'unknown' %}Unknown{% else %}{{ trace_analysis.summary.tool_calls }}{% if trace_analysis.metric_states.tool_calls.state == 'partial' %}+{% endif %}{% endif %} Tool calls {{ trace_analysis.summary.tool_failures }} failures · {{ trace_analysis.summary.tool_fallbacks }} fallback · {{ trace_analysis.summary.tool_degraded_results }} degraded
Semantic cache {{ trace_analysis.summary.cache_hits }} Validated hits {{ trace_analysis.summary.cache_misses }} miss · {{ trace_analysis.summary.cache_bypasses }} bypass · {{ trace_analysis.summary.cache_rejections }} rejected
Continual learning {{ trace_analysis.summary.learning_candidates }} Learning candidates {{ trace_analysis.summary.priority_counts.P0 }} P0 · {{ trace_analysis.summary.priority_counts.P1 }} P1 · {{ trace_analysis.summary.priority_counts.P2 }} P2 findings
Memory-first observability

What the model could actually use

Counts and hashed source references only. Raw profile facts, conversation excerpts, preferences, and writing samples are never copied into the trace.

Content-safe
1 · Retrieved {{ trace_analysis.summary.memory_candidates }} candidates considered
2 · Supplied {{ trace_analysis.summary.memory_supplied }} {{ trace_analysis.summary.memory_injected_chars }} context characters
3 · Referenced {{ trace_analysis.summary.memory_explicit_references }} conservative exact overlaps
{% if trace_analysis.memory_health %}
{% for source in trace_analysis.memory_health %}
{{ source.source|replace('_', ' ')|title }} {{ source.supplied }} {{ source.explicit_references }} explicit reference{% if source.explicit_references != 1 %}s{% endif %}
{% endfor %}
{% else %}

No memory-supply events are available in this trace window yet.

{% endif %} {% if trace_analysis.summary.memory_behavioral_sources_not_measured %}

{{ trace_analysis.summary.memory_behavioral_sources_not_measured }} style/preference signal{% if trace_analysis.summary.memory_behavioral_sources_not_measured != 1 %}s were{% else %} was{% endif %} supplied but not labelled “referenced”: behavioral application requires outcome evaluation, not string matching.

{% endif %}
{% if trace_analysis.insights %}
{% for insight in trace_analysis.insights %} {% set saved = persisted_recommendations.get(insight.id) %}
{{ insight.priority }} {{ insight.component }}

{{ insight.title }}

{{ insight.confidence }} confidence {{ insight.evidence_count }} signal{% if insight.evidence_count != 1 %}s{% endif %} {{ insight.effort }} effort

{{ insight.finding }}

{% if insight.evidence_anchors %}

Evidence: {% for anchor in insight.evidence_anchors %}Event {{ loop.index }}{% if not loop.last %} · {% endif %}{% endfor %}

{% endif %}
Recommended change

{{ insight.recommendation }}

{{ insight.target }}
{% if saved %} {{ saved.review_status|replace('_', ' ')|title }} · revision {{ saved.revision }} {% if not ui_read_only %}
{% endif %} {% else %} Not yet saved for review {% endif %}
{% endfor %}
{% else %}
OK

No actionable signals in the available evidence

Broaden the trace window or add explicit outcome feedback before changing the agent.

{% endif %}

Continual-learning handoff

{% if trace_analysis.learning_candidates %}

{{ trace_analysis.learning_candidates|length }} recurring trajectory candidate{% if trace_analysis.learning_candidates|length != 1 %}s{% endif %} found. Review outcomes before promotion; the existing diversity, success-rate, recency, and shadow-readiness gates still apply.

{% else %}

No repeated multi-step trajectory is strong enough in this window. Memorizz will keep collecting evidence without changing the agent.

{% endif %}
Review Workflow Memory
{% endif %} {% include "trace_waterfall.html" %} {% include "trace_inspectors.html" %} {% if selected_agent_id and trace_waterfall and trace_permissions and trace_permissions['replay.create'] %}

Safe Evalground replay draft

Freeze content-free references after current access checks. This does not run models, tools, or production side effects.

{% if not ui_read_only %}{% else %}

Draft persistence is disabled in read-only mode.

{% endif %} {% if not trace_capabilities['replay.create'].configured %}

Replay plans and drafts are unavailable until resource_authorizer is configured. No execution is enabled by configuring it.

{% endif %}

{% endif %}

Trace Timeline

{% if trace_events %}
{% for event in trace_events %}
{% if event.role == 'user' %}U{% elif event.role == 'assistant' %}A{% elif event.role == 'tool' %}T{% else %}S{% endif %}
{{ event.title or event.role|default('Event')|title }}
{% if event.kind %}{{ event.kind|replace('_', ' ')|title }}{% endif %} {% if event.outcome %} {% if event.outcome == 'success' %}Completed{% elif event.outcome == 'empty' %}Completed · no results{% elif event.outcome == 'fallback' %}Completed via fallback{% elif event.outcome == 'degraded' %}Completed with limitations{% elif event.outcome == 'provider_error' %}Provider error{% else %}Failed{% endif %} {% endif %} {% if event.tool_provider %}Provider {{ event.tool_provider }}{% endif %} {% if event.model %}{{ event.provider or 'Provider unknown' }} · {{ event.model }}{% endif %} {% if event.input_tokens is defined %}{{ event.input_tokens }} input tokens · {{ event.output_tokens|default('unknown') }} output tokens{% endif %} {% if event.finish_reason %}Finish: {{ event.finish_reason }}{% endif %} {% if event.max_output_tokens is defined %}Output limit: {{ event.max_output_tokens }}{% endif %} {% if event.response_chars is defined %}{{ event.response_chars }} characters · {{ event.response_bytes }} bytes{% endif %} {% if event.duration_ms is defined %}{{ event.duration_ms }} ms{% endif %} {% if event.status %}{{ event.status }}{% endif %} {% if event.outcome_reason_code %}Reason {{ event.outcome_reason_code|replace('_', ' ') }}{% endif %} {% if event.canonical_page_type %}Page {{ event.canonical_page_type }} · {{ event.canonical_page_id|default('—')|truncate(12) }}{% endif %} {% if event.thread_binding_status %}Binding {{ event.thread_binding_status|replace('_', ' ') }}{% endif %} {% if event.ownership_verified is defined and event.ownership_verified is not none %}Ownership {% if event.ownership_verified %}verified{% else %}unverified{% endif %}{% endif %} {% if event.grounding_status %}Grounding {{ event.grounding_status|replace('_', ' ') }}{% if event.grounding_source %} · {{ event.grounding_source|replace('_', ' ') }}{% endif %}{% endif %} {% if event.cache_decision %}Cache {{ event.cache_decision }}{% endif %} {% if event.kind == 'memory_context' %}{{ event.memory_supplied_count|default(0) }} supplied · {{ event.memory_candidate_count|default(0) }} candidates{% endif %} {% if event.kind == 'memory_reference' %}{{ event.memory_referenced_count|default(0) }} explicitly referenced{% endif %} {% if event.canonical_title_fingerprint or event.client_title_fingerprint %}Content fp {{ (event.canonical_title_fingerprint or event.client_title_fingerprint)|truncate(14) }}{% endif %} {% if event.request_context_fingerprint %}Context fp {{ event.request_context_fingerprint|truncate(14) }}{% endif %} {% if event.trace_id %}Trace {{ event.trace_id|truncate(14) }}{% endif %} {% if event.run_id %}Run {{ event.run_id|truncate(12) }}{% endif %} {% if event.span_id %}Span {{ event.span_id|truncate(12) }}{% endif %} Memory {{ event.memory_id|truncate(12) }} Thread {{ event.thread_id|default('—')|truncate(12) }} {{ event.timestamp }}
{% if event.input_refs or event.output_refs %}
Memory lineage and artifact provenance

Producer span: {{ event.parent_span_id or 'Not observed' }}

{% for ref in event.input_refs or [] %}{% endfor %} {% for ref in event.output_refs or [] %}{% endfor %}
DirectionResourceReferenceRoleVersionProvenance
Input{{ ref.resource_type }}{{ ref.ref }}{{ ref.role or 'Unspecified' }}{{ ref.version or 'Unknown' }}{{ ref.provenance_status }}
Output{{ ref.resource_type }}{{ ref.ref }}{{ ref.role or 'Unspecified' }}{{ ref.version or 'Unknown' }}{{ ref.provenance_status }}
{% if event.kind == 'artifact_persisted' %}

Persistence verified: {{ event.persistence_verified }} · Sources: {{ event.input_refs|length }} · Task: {{ event.task_id or 'Unspecified' }}

{% endif %}
{% endif %} {% if event.kind in ['output_contract', 'ui_delivery'] %}
Output contract: {{ event.contract_name }} {% if event.kind == 'output_contract' %}

Opening marker: {{ event.opening_marker_found }} · Closing marker: {{ event.closing_marker_found }} · Parser: {{ event.parser_status }} · Recovery: {{ event.recovery_status or ('used' if event.recovery_used else 'not used') }} · Nodes: {{ event.nodes|default('unknown') }} · Edges: {{ event.edges|default('unknown') }}

{% endif %} {% if event.kind == 'ui_delivery' %}

Emitted: {{ event.emitted }} · Browser acknowledged: {{ event.acknowledged|default('unknown', true) }}

{% endif %}
{% endif %} {% if trace_permissions and trace_permissions['trace.reveal'] and trace_content_mode != 'metadata' %}{% endif %}
Normalized metadata JSON
{{ event|tojson(indent=2) }}
{% endfor %}
{% else %}
TRACE

{% if trace_analysis and trace_analysis.coverage.normalization_errors %}Trace evidence is unavailable. Inspect the coverage errors above.{% else %}No traces recorded yet{% endif %}

Run the agent in Playground to generate trace logs.

{% endif %}
{% endif %} {% endblock %}