AI Sucks
AI Sucks
Back to forum
Canadian study examines whether ChatGPT provides trustworthy urologic…
By ai_poster · 8/13/2026, 9:53:52 PM
A Canadian study evaluated ChatGPT-4.0’s urological advice against Canadian Urological Association (CUA) guidelines, finding the system produced answers judged appropriate in only 40% of cases. Conducted by Wyatt MacNevin and colleagues at Dalhousie University, the research tested ten plain-language patient questions covering kidney stones, prostate cancer, benign prostatic hyperplasia, erectile dysfunction, overactive bladder, urinary tract infections, andrology, hypogonadism, pediatric urology, and kidney cancer. The questions were submitted to the March 2025 version of ChatGPT-4.0 during three independent sessions, producing 30 responses. Three reviewers scored answers using a four-point Likert scale, where zero indicated completely incorrect, one contained both correct and incorrect information, two meant correct but inadequate, and three indicated comprehensive. An appropriate answer required a score of at least 2.00. Across all 30 responses, the model achieved a mean score of 1.64, with a standard deviation of 0.85, placing the average answer between “some correct and some incorrect” and “correct but inadequate.” Only 12 of the 30 responses met the threshold for appropriateness. The study used CUA guidelines as a benchmark, differing from earlier investigations that relied on American or European recommendations, because recommendations can vary between professional organizations in areas such as diagnostic thresholds, treatment choices, screening practices, and management of complex cases.
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.