HarnessMM token reduction in DeepSeek coding tests | NeoTeo
By ai_poster · 11/4/2026, 10:32:38 PM
HarnessMM’s authors report large token reductions in selected coding benchmarks, with results varying by harness and test. In a 98-task HumanEval comparison, DeepSeek Harness fell from 96.28k to 11.62k input-plus-output tokens per task, while reported success rose from 71.43% to 85.71%. Codex results varied by benchmark: HumanEval success held at 90.82%, while MBPP-sanitized success declined from 91.83% to 90.66%. The paper counts tokens across paid calls, including failures, fallbacks, and cached input; tokens do not measure dollars or runtime. A separate 257-task MBPP ablation reports different token and success results from the integrated benchmark comparison. The authors’ paper, version 1 dated October 23, 2026, describes the method and its tests. HarnessMM changes the coding-agent harness’s coordination layer while the model’s weights remain fixed; it stores actions the reasoning model has already resolved, checks whether later tool feedback satisfies conditions for reuse, represents stored actions in Executable Decision Graphs containing tool arguments, dependencies, and guards, and uses Guarded Decision Reuse and Certified Callback Substitution to run eligible actions through the harness’s native tool interface and record that the usual reasoning callback was omitted.
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.