An AI4AI Framework for Visual Token Pruning
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - AutoPrune is a training-free "AI4AI" framework where an LLM automatically designs visual-token pruning policies for multimodal LLMs, replacing handcrafted heuristics and expert trial-and-error. It matters because inference cost of MLLMs is dominated by visual tokens, and the design space for pruning is expanding faster than manual tuning can cover.
- Introduces TPDSL, a Token Pruning Domain-Specific Language with 131 reusable atoms covering budget control, token scoring, selection constraints, and token reassembly.
- Key design choice: each search state is expressed as a residual modification of a strong base policy, narrowing the search space and focusing the LLM on the highest-impact policy components.
- Evaluated on 14 multimodal benchmarks and three MLLM backbones, showing effectiveness, efficiency, and transferability across settings.
- At 94.4% visual-token removal, retains >99% of full-token performance while cutting FLOPs 9.9x and prefill latency 6.4x.
Sources (1)
An AI4AI Framework for Visual Token Pruning
TL;DR - AutoPrune is a training-free "AI4AI" framework where an LLM automatically designs visual-token pruning policies for multimodal LLMs, replacing handcrafted heuristics and expert trial-and-error. It matters because inference cost of MLLMs is dominated by visual tokens, and the design space for pruning is expanding faster than manual tuning can cover.
- Introduces TPDSL, a Token Pruning Domain-Specific Language with 131 reusable atoms covering budget control, token scoring, selection constraints, and token reassembly.
- Key design choice: each search state is expressed as a residual modification of a strong base policy, narrowing the search space and focusing the LLM on the highest-impact policy components.
- Evaluated on 14 multimodal benchmarks and three MLLM backbones, showing effectiveness, efficiency, and transferability across settings.
- At 94.4% visual-token removal, retains >99% of full-token performance while cutting FLOPs 9.9x and prefill latency 6.4x.