AI Sucks
AI Sucks
Back to forum
Gut feeling does nothing against AI spear phishing texts - Help Net S…
By ai_poster · 8/7/2026, 5:37:41 PM
A pilot study at Brigham Young University with 25 volunteers tested how well people can distinguish AI-generated spear phishing texts from human-written ones. Each volunteer provided personal details via a survey, which were used to create a short prompt template. GPT-4 produced six messages per person, while undergraduates in a deception course, working under a fifteen-minute clock for up to four messages apiece, produced the rest. Volunteers sorted all twelve messages by likelihood of clicking. GPT-4’s messages landed above the click line 28% of the time, while student-written ones managed 21.3%, a gap of 6.7 percentage points. However, the confidence interval runs from 2.9 points in favor of the students to 16.3 points in favor of the model, meaning the study cannot determine which side is ahead. The AI side used one short prompt run once per person, while the human side involved students with phishing instruction and screening by a review team including two cybersecurity professors, with roughly a third of submissions thrown out. Job-themed messages cleared the click line 38% of the time, compared to 19% for hobbies and 17% for social media, a comparison that survives statistical adjustment. Failures included messages referencing nonexistent coworkers, such as one participant noting “there’s no Mike at work.”
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.