{% if bws_enabled %} {% endif %} {% if mace_enabled %} {% endif %} {% if embedding_viz_enabled %} {% endif %}

Agreement

Full report →

Loading…

{{ ui_lang.admin_section_system_info|default('System Information') }}

{{ ui_lang.admin_th_user_id|default('User ID') }} {{ ui_lang.admin_th_phase|default('Phase') }} {{ ui_lang.admin_th_annotations|default('Annotations') }} {{ ui_lang.admin_th_working_time|default('Working Time') }} {{ ui_lang.admin_th_avg_time_per_annotation|default('Avg Time/Annotation') }} {{ ui_lang.admin_th_speed_per_hour|default('Speed (per hour)') }} {{ ui_lang.admin_th_completion_pct|default('Completion %') }} {{ ui_lang.admin_th_max_instances|default('Max Instances') }} {{ ui_lang.admin_th_last_activity|default('Last Activity') }}
{{ ui_lang.admin_th_instance_id|default('Instance ID') }} {{ ui_lang.admin_th_text_preview|default('Text Preview') }} {{ ui_lang.admin_th_annotations|default('Annotations') }} {{ ui_lang.admin_th_completion_pct|default('Completion %') }} {{ ui_lang.admin_th_most_frequent_label|default('Most Frequent Label') }} {{ ui_lang.admin_th_disagreement|default('Disagreement') }} {{ ui_lang.admin_th_avg_time|default('Avg Time') }} {{ ui_lang.admin_th_annotators|default('Annotators') }} {{ ui_lang.admin_th_num_ai_used|default('Num AI Used') }}

{{ ui_lang.admin_section_devices|default('Annotator Devices') }}

{{ ui_lang.admin_th_user_id|default('User ID') }} {{ ui_lang.admin_th_last_device|default('Last Device') }} {{ ui_lang.admin_th_mobile_visits|default('Mobile Visits') }} {{ ui_lang.admin_th_tablet_visits|default('Tablet Visits') }} {{ ui_lang.admin_th_desktop_visits|default('Desktop Visits') }} {{ ui_lang.admin_th_pocket_visits|default('Mobile UI Visits') }} {{ ui_lang.admin_th_last_seen|default('Last Seen') }}

{{ ui_lang.admin_section_ai_usage|default('AI Assistance Usage') }}

{{ ui_lang.admin_section_quality|default('Quality Indicators') }}

{{ ui_lang.admin_th_user_id|default('User ID') }} {{ ui_lang.admin_th_instances|default('Instances') }} {{ ui_lang.admin_th_avg_time_s|default('Avg Time (s)') }} {{ ui_lang.admin_th_interactions|default('Interactions') }} {{ ui_lang.admin_th_changes|default('Changes') }} {{ ui_lang.admin_th_ai_requests|default('AI Requests') }} {{ ui_lang.admin_th_ai_accept_rate|default('AI Accept Rate') }} {{ ui_lang.admin_th_suspicion|default('Suspicion') }}
{% if embedding_viz_enabled %}

Loading...

INSTANCE PREVIEW

Hover over a point to see details

SELECTION QUEUE

Use lasso or box selection to select points

Selected items will be prioritized for annotation

CONTROLS

{% endif %} {% if bws_enabled %}

Loading BWS scoring data...

{{ ui_lang.admin_section_bws_scores|default('BWS Item Scores') }}

Compute item scores from best-worst annotations. Counting is the default method (no dependencies). Bradley-Terry and Plackett-Luce require the choix package.

{{ ui_lang.admin_th_rank|default('Rank') }} {{ ui_lang.admin_th_item_id|default('Item ID') }} {{ ui_lang.admin_th_text|default('Text') }} {{ ui_lang.admin_th_score|default('Score') }} {{ ui_lang.admin_th_best_count|default('Best Count') }} {{ ui_lang.admin_th_worst_count|default('Worst Count') }} {{ ui_lang.admin_th_appearances|default('Appearances') }}
Click "Generate Scores" to compute BWS scores
{% endif %} {% if mace_enabled %}

Loading MACE data...

{{ ui_lang.admin_section_competence|default('Annotator Competence') }}

MACE estimates each annotator's reliability (0 = random guessing, 1 = always correct).

{{ ui_lang.admin_th_annotator|default('Annotator') }} {{ ui_lang.admin_th_competence|default('Competence') }} {{ ui_lang.admin_th_reliability|default('Reliability') }}
Click "Run MACE" to compute scores

{{ ui_lang.admin_section_predicted_labels|default('Predicted Labels') }}

{{ ui_lang.admin_mace_predicted_help|default("MACE's best estimate of the true label for each item, weighted by annotator competence.") }}

{{ ui_lang.admin_mace_run_predictions|default('Run MACE to see predictions.') }}

{% endif %}

{{ ui_lang.admin_section_system_config|default('System Configuration') }}