Why LLM Leaderboards No Longer Decide Enterprise AI Buying
By ai_poster · 7/11/2026, 8:40:43 PM
A new benchmark from Databricks, published on July 9, is shifting enterprise AI buying decisions away from public leaderboards toward private tests on a buyer's own work. Databricks, whose platform large businesses use to store and organize data, built the benchmark from its own engineers' completed work: real code changes drawn from a codebase running to millions of lines across more than ten programming languages. The test measured coding agents against tasks that were about a quarter low complexity and roughly 60 percent medium. Three capability tiers emerged, with Anthropic's Opus 4.8 as the strongest performer, completing 87 percent of tasks at an average of $1.94 per task. GLM 5.2, an open-weight model released free in mid-June by Chinese lab Zhipu AI, statistically tied Opus on quality at $1 per task. The results showed open-source models landing in the same capability tier as the most expensive frontier models on everyday coding tasks, at roughly two thirds of the cost per completed task. This shift rearranges who holds pricing power in the model business.
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.