paper:doi-10-18653-v1-d16-1264SQuAD: 100,000+ Questions for Machine Comprehension of Text
Original abstract (expand)
We present the Stanford Question Answering Dataset (SQuAD), a new reading comprehension dataset consisting of 100,000+ questions posed by crowdworkers on a set of Wikipedia articles, where the answer to each question is a segment of text from the corresponding reading passage. We analyze the dataset to understand the types of reasoning required to answer the questions, leaning heavily on dependency and constituency trees. We build a strong logistic regression model, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%). However, human performance (86.8%) is much higher, indicating that the dataset presents a good challenge problem for future research. The dataset is freely available at https://stanford-qa.com
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 72%
- CoMMET: To What Extent Can LLMs Perform Theory of Mind Tasks?Weifeng Jiang, Chengwei Qin, Cheston Tan Ruirui Chen2026≈ 70%
- Comparing Specialised Small and General Large Language Models on Text Classification: 100 Labelled Samples to Achieve Break-Even PerformanceIvan Srba, Maria Bielikova Branislav Pecher2026≈ 70%
- ≈ 69%
- Silicon Minds versus Human Hearts: The Wisdom of Crowds Beats the Wisdom of AI in Emotion RecognitionVinayaka Gude, Haya Ajjan Mustafa Akben2025≈ 69%
- An Evaluation on Large Language Model Outputs: Discourse and MemorizationXun Wang, Alex Sokolov, Qilong Gu and Si-Qing Chen Adrian de Wynter2026≈ 69%
- Belief in the Machine: Investigating Epistemological Blind Spots of Language ModelsTayfun Gur, Federico Bianchi, Daniel E. Ho, Thomas Icard, Dan Jurafsky, James Zou Mirac Suzgun2024≈ 69%
- An agentic system with reinforcement-learned subsystem improvements for parsing form-like documentsSaurav Sthapit, Tahir Qasim Syed Ayesha Amjad2025≈ 69%
- Grammar Search for Multi-Agent SystemsVikas Yadav, Shiva Krishna Reddy Malay, Shravan Nayak, Sai Rajeswar, Sathwik Tejaswi Madhusudhan, Eduardo Blanco Mayank Singh2025≈ 69%
- StreetMath: Study of LLMs' Approximation BehaviorsSomshubhra Roy, Maisha Thasin, Danyang Zhang, and Blessing Effiong Chiung-Yi Tseng2025≈ 69%
- When Do You Need Billions of Words of Pretraining Data?Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman Yian Zhang2020≈ 69%
- Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of ScoresVenkata S. Govindarajan, Joe Barrow, Jiuding Sun, Alexa F. Siu, Byron C. Wallace, Ani Nenkova Chantal Shaib2026≈ 69%
- Probing the Robustness of Theory of Mind in Large Language ModelsLaura Schrewe, Lucie Flek Christian Nickel2024≈ 68%
- MMToM-QA: Multimodal Theory of Mind Question AnsweringYutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer Ullman, Antonio Torralba, Joshua B. Tenenbaum, Tianmin Shu Chuanyang Jin2024≈ 68%
- Collective Intelligence in Human-AI Teams: A Bayesian Theory of Mind ApproachSamuel Westby and Christoph Riedl2023≈ 68%
- ≈ 66%
- ≈ 66%
- ≈ 66%
- ≈ 66%
- ≈ 65%
- Evaluating Language Model Character Traitsin corpus2024≈ 65%
- Model Alignment Searchin corpus2025≈ 65%
- Collective intelligence: A unifying concept for integrating biology across scales and substratesin corpus2024≈ 64%
- Opening the Hood of a Word Processorin corpus1984≈ 64%
- ≈ 64%
- ≈ 64%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 64%
- ≈ 64%
- ≈ 64%
Similar preprints — Semantic Scholar
Cited by (2)
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire
A sparse autoencoder trained on layer-19 residual-stream activations of Llama-3-8B-Instruct, using LMSYS-Chat-1M, yields features that can be labeled automatically, ranked by causal relevance, and use
- Interpreting Language Model Parameters
VPD (adVersarial Parameter Decomposition) decomposes weight matrices directly into rank-one interpretable subcomponents rather than decomposing activations as sparse autoencoders (SAEs) do, flipping t