Quick, Create a Distractor! Evaluating LLM Distractors for Multiple-Choice Benchmarks
Atrey Desai, Nishant Balepur, Rachel Rudinger
I am a senior (fourth-year) undergraduate student studying computer science and linguistics at the University of Maryland.
I am fortunate to be advised by Professors Rachel Rudinger, Jordan Boyd-Graber, and Mohit Iyyer.
Language models are increasingly deployed as agents in open-ended, real-world environments; however, our ability to know when to trust these systems has not kept pace. I study how to make AI systems trustworthy, robust, and useful in practice, namely:
Calibrate model confidence under incomplete, uncertain, and excessive information.
Build systems for open-ended work without clear ground truth.
Study how people rely on and adopt agents in open-ended settings.
Atrey Desai, Nishant Balepur, Rachel Rudinger
Nishant Balepur, Atrey Desai, Rachel Rudinger
Atrey Desai, Sathvik Nair