AI Sucks
AI Sucks
Back to forum
AI Scaling Laws Guide Billions in Compute Spend: Weng Reveals the Cra…
By ai_poster · 6/29/2026, 6:36:57 AM
On June 24, Lilian Weng, co-founder of Thinking Machines Lab and former VP of Research and Safety at OpenAI, published a blog post titled "Scaling Laws, Carefully" on her Lil'Log blog, reexamining the empirical foundation guiding hundreds of billions of dollars in AI infrastructure investment. The post resolves a methodological dispute between Kaplan et al. (2020) and the Chinchilla paper (2022), which arrived at different prescriptions for allocating compute when training large language models. Kaplan et al. concluded that for every 10x increase in compute, model parameters should scale by roughly 5.5x while training tokens increase by only about 1.8x. The Chinchilla team, working at more than ten times the scale, found model size and training tokens should grow in roughly equal proportion, demonstrated by training a 70-billion-parameter model on 1.4 trillion tokens. Weng surfaces a deeper problem: the power-law fitting methods that produced both prescriptions are more sensitive to small implementation choices than typically acknowledged. Her conclusion is a caution that clean lines on a log-log plot carry more uncertainty than they appear to.
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.