Nine years after Google researchers introduced the transformer in the 2017 paper “Attention Is All You Need,” the architecture powers every major large language model, but its flaws are driving startups to seek alternatives. Transformers rely on dense attention, comparing every token with every other, which causes computational costs to soar as text length grows; a 10,000-word document might require 50 million multiplications. This inefficiency drives huge power consumption, with OpenAI set to spend $50 billion on computing this year, according to president Greg Brockman, and the International Energy Agency predicting data center electricity use will double by 2030. Transformers also struggle with large context windows, limiting their ability to process extensive data like entire libraries or code bases, which is needed for harder tasks and reasoning models. Recent advances are workarounds patching fundamental flaws rather than extensions of the core technology. A wave of startups, including Subquadratic, cofounded by CEO Justin Dangel, who calls transformers “one of the most important innovations in the history of computer science,” is now pursuing the next big thing. While LLMs will remain, the construction methods are open to change, and these newcomers have less to lose than established frontrunners.