Harder Benchmarks Are Not Always Better: Evaluating LLM Distractors for Multiple-Choice Benchmarks
Atrey Desai, Nishant Balepur, Rachel Rudinger
Under ACL ARR Review, 2026 preprint
NLPEvaluation
research
10 entries
publications
Test-Time Reasoners Are Strategic Multiple-Choice Test-Takers
Nishant Balepur, Atrey Desai, Rachel Rudinger
ACL Main, 2026
NLPReasoning
Asking About Occluded Content Marks Adversarial Multimodal Questions
HyoJung Han*, Atrey Desai*, Ishan Upadhyay, Tianyi Zheng, Eve Fleisig, Aparna Garimella, Marine Carpuat, Jordan Boyd-Graber
Under ACL ARR Review, 2026 preprint
NLPMultimodalEvaluation
Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?
Atrey Desai, Sathvik Nair
ACL Findings; TLS, 2026 Oral (TLS)
NLPLinguistics
BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarks
ACL Main, 2026
NLPEvaluation
A Preview of Computational Animal Linguistics
Tuan M. Dang, Atrey Desai, Hridayesh Lekhak, Tirza Panunto, Lindsay Pike, Theron S. Wang, Kenny Q. Zhu
ACM Computing Surveys, 2026
NLPLinguistics
Language Models Generate Multiple-Choice Questions with Artifacts
Atrey Desai, Nishant Balepur, Rachel Rudinger
MASC-SLL, 2025
NLPEvaluation
Reinforcement Learning As End-User Trigger-Action Programming
AAAI IML Workshop 2022, RLDM 2022, 2022
RLHCI
class projects
talks
Filler-Gap Dependencies under Developmental Constraints in LMs
The University of Texas at Austin, Feb 2026
Adaptor Grammars and Neural Networks for Feline Lexical Discovery
University of Maryland, College Park, Nov 2024
The University of Texas at Arlington, Jul 2024
presentations
Harder, but Are They Better? Measuring the Quality of LLM-Generated MCQ Distractors
University of Maryland, College Park, May 2026
Language Models Generate Multiple-Choice Questions with Artifacts
University of Maryland, College Park, Jul 2025
University of Maryland, College Park, Apr 2025