Apple researchers trained a 1.7B-parameter base flow model on 2.1T tokens and self-distilled it into a Categorical Flow Map (CFM) capable of generating diverse, high-quality text in as few as 4 inference steps while maintaining near-data-level token entropy. The work addresses the previously open question of scalability for CFMs, which had only been evaluated at scales below 1B parameters. The authors also introduced a likelihood bound for CFMs in the semi-discrete setting, demonstrating that these models can score standard language modelling benchmarks with results in the same range as discrete diffusion methods. Additionally, the article uncovers challenges arising from training such models at scale and provides prescriptive insights on loss weighting and time scheduling. The research builds on prior work showing that discrete data can be generated continuously via a flow matching process between a Gaussian and one-hot encoded data distributions, with CFMs enabling accelerated sampling and competitive sample quality in the few-step regime. The authors are Oscar Davis (University of Oxford), Anastasiia Filippova, Victor Turrisi, Amitis Shidani, Pierre Ablin, Marco Cuturi, and Louis Béthune.