Language Models Can Control Their Own Attention
Ranking
Overall
87
Content
95
Popularity
68
Observed public metrics from 1 member.
Merged summary
TL;DR - Declarative Attention lets language models explicitly indicate which context regions they need during generation, allowing the inference engine to avoid reading most of the KV cache. On long-context tasks, it substantially reduced attended tokens with modest accuracy losses.
- The protocol uses three model-declared modes:
<global>for full context,<focus>for a specific region, and<local>for recent output only. - Unlike proxy-based token selection, the model specifies relevant context intrinsically through chain-of-thought declarations parsed like tool calls.
- Across 15 zero-shot long-context tasks, attended tokens fell by 52.0% for Gemma-4-31B and 31.1% for Qwen-3.6-27B.
- Accuracy declined by 1.27 and 2.75 percentage points, respectively, with losses reported to shrink as model scale increases.
Sources (1)
Language Models Can Control Their Own Attention
Public signals
Hugging Face upvotes 78
TL;DR - Declarative Attention lets language models explicitly indicate which context regions they need during generation, allowing the inference engine to avoid reading most of the KV cache. On long-context tasks, it substantially reduced attended tokens with modest accuracy losses.
- The protocol uses three model-declared modes:
<global>for full context,<focus>for a specific region, and<local>for recent output only. - Unlike proxy-based token selection, the model specifies relevant context intrinsically through chain-of-thought declarations parsed like tool calls.
- Across 15 zero-shot long-context tasks, attended tokens fell by 52.0% for Gemma-4-31B and 31.1% for Qwen-3.6-27B.
- Accuracy declined by 1.27 and 2.75 percentage points, respectively, with losses reported to shrink as model scale increases.