| English-Egyptian Arabic | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| System Name | LP Supported | Params. (B) | Humeval? | AutoRank | CometKiwi-XL | GEMBA-ESA-CMDA | GEMBA-ESA-GPT4.1 | MetricX-24-Hybrid-XL | XCOMET-XL |
| Shy-hunyuan-MT | ✓ | 7 | ✓ | 1.0 | 0.658 | 76.3 | 75.0 | -5.7 | 0.388 |
| Wenyiil | ✓ | 14 | ✓ | 2.5 | 0.65 | 79.2 | 73.3 | -6.4 | 0.337 |
| Algharb | ✓ | 14 | ✓ | 2.6 | 0.645 | 80.0 | 73.9 | -6.5 | 0.328 |
| GemTrans | ✓ | 27 | ✓ | 3.4 | 0.644 | 73.0 | 69.6 | -6.0 | 0.345 |
| CommandA-WMT | ✓ | 111 | ✓ | 4.0 | 0.621 | 77.8 | 75.4 | -7.0 | 0.311 |
| UvA-MT | ✓ | 12 | ✓ | 4.1 | 0.637 | 74.4 | 73.4 | -7.1 | 0.325 |
| Yolu | ✓ | 14 | ✓ | 5.4 | 0.658 | 67.8 | 63.9 | -6.6 | 0.323 |
| Gemini-2.5-Pro | ✓ | ? | ✓ | 5.6 | 0.552 | 79.5 | 84.5 | -7.6 | 0.267 |
| ONLINE-B | ✓ | ? | ✓ | 6.4 | 0.627 | 70.4 | 67.4 | -7.1 | 0.288 |
| GPT-4.1 | ✓ | ? | ✓ | 6.5 | 0.534 | 78.4 | 84.1 | -7.8 | 0.265 |
| DeepSeek-V3 | ? | 671 | ✓ | 6.9 | 0.573 | 74.2 | 75.7 | -7.7 | 0.273 |
| Mistral-Medium | ✓ | ? | ✓ | 7.5 | 0.586 | 71.7 | 71.0 | -7.8 | 0.274 |
| English-Czech | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| System Name | LP Supported | Params. (B) | Humeval? | AutoRank | CometKiwi-XL | GEMBA-ESA-CMDA | GEMBA-ESA-GPT4.1 | MetricX-24-Hybrid-XL | XCOMET-XL |
| Shy-hunyuan-MT | ✓ | 7 | ✓ | 1.0 | 0.658 | 83.7 | 89.4 | -5.5 | 0.639 |
| Gemini-2.5-Pro | ✓ | ? | ✓ | 3.4 | 0.633 | 83.8 | 91.5 | -6.2 | 0.574 |
| CommandA-WMT | ✓ | 111 | ✓ | 3.5 | 0.645 | 81.3 | 86.2 | -6.0 | 0.594 |
| GPT-4.1 | ✓ | ? | ✓ | 3.9 | 0.63 | 84.2 | 89.7 | -6.6 | 0.576 |
| Wenyiil | ✓ | 14 | ✓ | 4.4 | 0.645 | 79.4 | 86.3 | -6.4 | 0.586 |
| DeepSeek-V3 | ? | 671 | ✓ | 5.0 | 0.628 | 81.4 | 87.0 | -6.5 | 0.565 |
| GemTrans | ✓ | 27 | ✓ | 5.0 | 0.636 | 76.6 | 81.8 | -5.8 | 0.596 |
| Algharb | ✓ | 14 | ✓ | 6.2 | 0.627 | 79.4 | 85.0 | -6.9 | 0.552 |
| Yolu | ✓ | 14 | ✓ | 6.3 | 0.651 | 74.6 | 78.6 | -6.5 | 0.582 |
| UvA-MT | ✓ | 12 | ✓ | 6.4 | 0.637 | 77.3 | 82.9 | -6.9 | 0.562 |
| Mistral-Medium | ? | ? | ✓ | 7.0 | 0.621 | 78.4 | 84.4 | -7.1 | 0.547 |
| SRPOL | ✓ | 12 | ✓ | 8.6 | 0.641 | 72.9 | 76.2 | -7.3 | 0.552 |