OpenAI is calling for developers to replace SWE-Bench Pro, an AI benchmark, claiming almost 30% of its tests are “broken”. OpenAI released its new GPT-5.6 Sol model on Thursday and said it had audited the Scale AI-developed benchmark, finding a series of issues despite recommending its use in February 2026. In a July 8 blog, OpenAI’s research team retracted its recommendation and expressed hope that the wider evaluation community will develop new benchmarks built by experienced software developers to test model capabilities.