Filip Makraduli, with co-author Niels Graf, presented an optimization for transformer models on the AI Engineer podcast, targeting RMSNorm, the normalization step that does "quite small" arithmetic but can fire roughly 33 times in a single decoding step, incurring kernel launch overhead, memory traffic, and synchronization waits. The work offers two algebraic propositions plus a third for newer architectures: weightless normalization, which folds gain and weight into a single offline-computed matrix and requires no kernel work, applicable to any model via a repository called Transformer Tricks; and deferred normalization, which moves scalar division to execute in parallel with the matrix multiplication, requiring custom CUDA streams. A third, pre-normalization cancellation, drops one of two RMSNorms based on scale invariance and is architecture-dependent. Makraduli compared the approach to Flash Attention, reordering operations to limit launch and synchronization overhead rather than FLOPs. He also described a bug in his CUDA implementation of deferred normalization, which split work across tensor cores and CUDA cores using an implicit join; unit tests and perplexity testing passed, but over long generations the model produced a one-step lag, repeating earlier words, such as "because" in a test prompt ending with "because."