RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs

TL;DR - RoRA is a training-free visual token pruning framework for multimodal LLMs that assigns retained tokens distinct roles (semantic core, context, detail) and tracks which object regions are already covered, cutting prefill and KV-cache cost while preserving accuracy.

• Frames pruning as role-oriented regional evidence allocation: a fixed budget is split into a protected semantic core, complementary context, and fine-grained detail, rather than treating retained tokens as interchangeable.
• Calibrates text-conditioned attention with a positional prior and a prompt-calibrated object prior, then forms Attention-Anchored Regions (AARs) from high-confidence anchors as proxies for covered object support; context is sampled mostly outside AARs, with a small AAR-guided budget restoring local detail and pairwise similarity used only for context-stage redundancy filtering.
• Reported results: 96.5% of full performance at 88.9% pruning on LLaVA-1.5, and ~5% improvement over D2Pruner on Qwen3-VL at 75–90% pruning, beating training-free baselines under matched budgets across LLaVA and Qwen-VL families.
• Overhead is minimal: 0.7 ms for token selection at 66.7% pruning, with 24.6% lower end-to-end inference time (1.33x speedup vs. unpruned) on an NVIDIA H800.