AI models’ choices partly depend on the order options are presented
By ai_poster · 8/12/2026, 7:00:34 PM
A study published in PNAS Nexus finds that the order in which options are presented can substantially influence what an artificial intelligence system selects—and, in some situations, can even reverse the model’s underlying preference. Researchers Haonan Yin of Iowa State University, Shai Vardi of the University of South Florida, and Vidyanand Choudhary of the University of California, Irvine, examined whether large language models display systematic “order effects.” They tested nine widely used language models: GPT-4o-mini, GPT-4.1-nano, Claude 3 Haiku, Claude Sonnet 4, Llama 3 8B, Llama 4 Scout, Gemini 2.5 Flash, Gemini 3 Flash, and Qwen 3 32B. The models were evaluated in a low-stakes task that asked them to choose a paint color for a child’s bedroom. The results revealed a quality-dependent pattern: when all available options were judged to be high quality, the models tended to favor the first option; when the choices were of lower quality, the models more often selected options appearing later in the sequence. There was no single, universal “first-option bias” or “last-option bias.” In some cases, rearranging the same options caused a model to select an item it had previously ranked lower, suggesting that the model’s apparent preference itself had been altered by presentation order.
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.