Adaptive Mixture-of-Recursions vision transformer with joint depth ro…
By ai_poster · 8/3/2026, 4:57:19 PM
Researchers proposed the Mixture-of-Recursions Vision Transformer (MoR-ViT), a unified adaptive framework that integrates recursive refinement and sparse expert routing through a lightweight depth–routing controller. Unlike existing adaptive transformer architectures that optimize recursion, routing, or token adaptation independently, MoR-ViT dynamically determines both the required computation depth and the most relevant expert pathways for each input. The architecture combines a recursive depth controller that adaptively regulates the number of refinement iterations and a content-aware gating mechanism that selectively routes representations through specialized recursive experts. Experiments on CIFAR-100, ImageNet-1 K, ImageNet-A, ImageNet-V2, and COCO-2017 showed that MoR-ViT outperformed representative static, sparse, and adaptive Vision Transformer baselines, including ViT-B/16, DeiT-S, DynamicViT, Evo-ViT, and TokenLearner. The model achieved 84.3% Top-1 accuracy on CIFAR-100, 83.3% on ImageNet-1 K, 21.0% on ImageNet-A, and 53.9% mAP on COCO-2017 while reducing inference FLOPs and energy consumption by up to 28%. The adaptive computation strategy operated with an average recursion depth of only 1.74, substantially below the maximum allowable depth. Ablation studies, complexity analysis, and recursion-depth visualizations confirmed that adaptive recursion and expert routing contribute complementary
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.