Language Models Generate Multiple-Choice Questions with Artifacts
Atrey Desai, Nishant Balepur, Rachel Rudinger
MASC-SLL, 2025
NLPBenchmarking
research
Filter:
publications
2025
2022
Reinforcement Learning As End-User Trigger-Action Programming
Chace Hayhurst, Hyojae Park, Atrey Desai, Suheidy De Los Santos, Michael Littman
AAAI IML Workshop 2022, RLDM 2022, 2022
Reinforcement LearningHuman-AI Interaction
preprints
BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks
Nishant Balepur, Bhavya Rajasekaran, Jane Oh, Michael Xie, Atrey Desai, Jordan Boyd-Graber
Under Review at ACL, 2025 preprint
NLPBenchmarking
A Preview of Computational Animal Linguistics
Atrey Desai, Tirza Panunto, Lindsay Pike, Theron S. Wang, Tuan M. Dang, Hridayesh Lekhak, Kenny Q. Zhu
Under Review at Computational Linguistics, 2025 preprint
NLPAnimal Communication
Emergence of Filler–Gap Mechanisms in Developmentally Constrained Language Models
Atrey Desai, Sathvik Nair
Under Review at TLS, ACL, 2025 preprint
NLPLinguistics
Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers
Nishant Balepur, Atrey Desai, Rachel Rudinger
Under Review at ACL Rolling Review, 2025 preprint
TL;DR: While choices-only success is often deemed problematic, reasoning traces reveal that LLMs use less problematic strategies like inferring missing questions, challenging claims that partial-input success is always a flaw. Consequently, reasoning traces could help separate problematic data from less problematic reasoning.
NLPReasoning
talks & presentations
Language Models Generate Multiple-Choice Questions with Artifacts
University of Maryland, College Park · Jul 2025 poster
Language Models Generate Multiple-Choice Questions with Artifacts
University of Maryland, College Park · Apr 2025 poster
Adaptor Grammars and Neural Networks for Feline Lexical Discovery (adapted)
University of Maryland, College Park · Nov 2024 research talk
Adaptor Grammars and Neural Networks for Feline Lexical Discovery
The University of Texas at Arlington · Jul 2024 research talk