HyperAIHyperAI

Command Palette

Search for a command to run...

Console

Question Answering On Squad11 Dev

Metrics

EM
F1

Results

Performance results of various models on this benchmark

Paper Title
XLNet+DSC89.7995.77Dice Loss for Data-imbalanced NLP Tasks
T5-11B90.0695.64Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
XLNet (single model)89.795.1XLNet: Generalized Autoregressive Pretraining for Language Understanding
LUKE 483M-95LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
T5-3B88.5394.95Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
T5-Large 770M86.6693.79Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
BERT-LARGE (Ensemble+TriviaQA)86.292.2BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
T5-Base85.4492.08Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
BERT-LARGE (Single+TriviaQA)84.291.1BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BART Base (with text infilling)-90.8BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
BERT large (LAMB optimizer)-90.584Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
BERT-Large-uncased-PruneOFA (90% unstruct sparse)83.3590.2Prune Once for All: Sparse Pre-Trained Language Models
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)83.2290.02Prune Once for All: Sparse Pre-Trained Language Models
BERT-Base-uncased-PruneOFA (85% unstruct sparse)81.188.42Prune Once for All: Sparse Pre-Trained Language Models
BERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)80.8488.24Prune Once for All: Sparse Pre-Trained Language Models
TinyBERT-6 67M79.787.5TinyBERT: Distilling BERT for Natural Language Understanding
BERT-Base-uncased-PruneOFA (90% unstruct sparse)79.8387.25Prune Once for All: Sparse Pre-Trained Language Models
T5-Small79.187.24Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
R.M-Reader (single)78.9 86.3Reinforced Mnemonic Reader for Machine Reading Comprehension
DensePhrases78.386.3Learning Dense Representations of Phrases at Scale
0 of 55 row(s) selected.
Question Answering On Squad11 Dev | SOTA | HyperAI