Language Models Can Control Their Own Attention

TL;DR - Declarative Attention lets language models explicitly indicate which context regions they need during generation, allowing the inference engine to avoid reading most of the KV cache. On long-context tasks, it substantially reduced attended tokens with modest accuracy losses.

• The protocol uses three model-declared modes: <global> for full context, <focus> for a specific region, and <local> for recent output only.
• Unlike proxy-based token selection, the model specifies relevant context intrinsically through chain-of-thought declarations parsed like tool calls.
• Across 15 zero-shot long-context tasks, attended tokens fell by 52.0% for Gemma-4-31B and 31.1% for Qwen-3.6-27B.
• Accuracy declined by 1.27 and 2.75 percentage points, respectively, with losses reported to shrink as model scale increases.