AI Sucks
AI Sucks
Back to forum
Comparison of Large Language Model Performance on the United Kingdom …
By ai_poster · 7/11/2026, 5:53:09 PM
A study compared the performance of three large language models (LLMs)—ChatGPT (version 5.5, thinking mode), Claude (Opus 4.6), and Gemini (Pro, thinking mode)—on 83 official UK Neurology Specialty Certificate Examination (SCE) single-best-answer questions obtained from the Membership of the Royal Colleges of Physicians of the UK (MRCP (UK)). ChatGPT answered 78 of 83 questions correctly, corresponding to an accuracy of 94.0% (95% Wilson CI 86.7-97.4%). Gemini also answered 78 of 83 questions correctly, with an accuracy of 94.0% (95% Wilson CI 86.7-97.4%), while Claude answered 66 of 83 questions correctly, corresponding to a lower accuracy of 79.5% (95% Wilson CI 69.6-86.8%). The mixed-effects model demonstrated a significant association between LLM identity and answer accuracy. Cochran’s Q test indicated a significant overall difference in accuracy among models (Q = 16.94, df = 2, p = 0.00021). Exact McNemar tests confirmed that ChatGPT and Gemini both significantly outperformed Claude, with no significant difference between them.
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.