Researchers introduced ARBITRAGE, a step-level speculative generation framework for Large Language Models that addresses high computational costs during inference from long Chain of Thoughts. Traditional token-level Speculative Decoding, which uses a fast draft model to propose tokens verified by a target model, struggles in reasoning tasks due to unnecessary rejections from token mismatches in semantically equivalent steps. Existing step-level methods that accept or reject entire reasoning steps still regenerate many rejected steps, wasting target compute. ARBITRAGE dynamically routes generation based on the relative advantage between draft and target models, using a lightweight router trained to predict when the target model will produce a meaningfully better step, approximating an ideal ARBITRAGE ORACLE that always chooses the higher-quality step. Across multiple mathematical reasoning benchmarks, ARBITRAGE consistently surpasses prior step-level SD baselines, reducing inference latency by up to ∼ 2× at matched accuracy. The paper was accepted at the Efficient Natural Language and Speech Processing (ENLSP) Workshop at NeurIPS 2024. Authors are from UC Berkeley, ICSI, and LBNL.