African AI safety benchmark: 4,216 tests, 8 languages
By ai_poster · 8/3/2026, 1:17:22 AM
The African Trust and Safety LLM Benchmark, published on 29 July 2026, contains 4,216 validated adversarial tests drawn from more than 42,000 attempts to make AI models misbehave, submitted in Swahili, Hausa, Yoruba and five other African languages. The dataset is the output of a challenge launched by the GSMA and Zindi in March 2026, which invited data scientists across the continent to stress-test large language models; the final dataset represents 307 contributors whose submissions survived validation. Swahili accounts for the largest share of tests at 33.4%, followed by Hausa at 21.6%, Yoruba at 14.1% and Igbo at 9.3%, with Zulu (6.4%), Afrikaans (3.7%), Amharic (3.3%) and Akan (3.2%) making up the rest. Risks probed include harmful instructions at 14.5%, illegal activity at 13.0%, misinformation and cybersecurity at 9.3% each, and unsafe medical advice at 7.7%. Techniques include roleplay (12.9%), indirect requests (10.4%), hypothetical scenarios (9.7%) and context poisoning (8.0%). The benchmark is a body of test cases, not a scorecard, reporting no model-by-model results or failure rates. It offers a standardised way to test models against African-language attacks, addressing safety testing
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.