Language Model Benchmark Results
We evaluated several language models across standard NLP benchmarks to assess their relative performance. The following tables summarize our findings from extensive testing conducted over six months of rigorous evaluation and comparison.
Performance Comparison
The table below shows accuracy scores on the GLUE benchmark for each model variant tested in our comprehensive study.
| Model | MNLI | QQP | SST-2 | Average |
|---|---|---|---|---|
| BERT-base | 84.6 | 71.2 | 93.5 | 83.1 |
| RoBERTa | 87.6 | 74.3 | 94.8 | 85.6 |
| DeBERTa | 90.1 | 76.8 | 96.2 | 87.7 |
Training Efficiency
Beyond raw performance, we also compared training efficiency. Models were evaluated on training time relative to performance gains, providing a cost-effectiveness analysis for practitioners and researchers with limited computational budgets.
Conclusion
DeBERTa consistently outperforms both BERT and RoBERTa across all evaluated benchmarks, though at a higher computational cost. For resource-constrained settings, RoBERTa provides the best performance-to-cost ratio based on our extensive evaluation.