Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generation
Ranking
Overall
78
Content
95
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - PASK uses parser-state information to preserve schema-critical KV-cache entries during structured LLM generation. On Qwen3-4B, it improves function-calling accuracy under compression while reducing memory use and accelerating serving.
- Converts parser-derived structural signals into layer-group-specific KV persistence policies.
- Calibrates policies offline using task-error sensitivity and attention-output distortion, requiring only lightweight lookups online.
- At a 0.33 target KV budget, surpasses the strongest compressed baseline by an average of 17.39 percentage points across eight BFCL subcategories.
- Achieves up to 2.2× higher throughput, 3.3× lower time per output token, and 0.53× the peak GPU memory of full KV caching.
Sources (1)
Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generation
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - PASK uses parser-state information to preserve schema-critical KV-cache entries during structured LLM generation. On Qwen3-4B, it improves function-calling accuracy under compression while reducing memory use and accelerating serving.
- Converts parser-derived structural signals into layer-group-specific KV persistence policies.
- Calibrates policies offline using task-error sensitivity and attention-output distortion, requiring only lightweight lookups online.
- At a 0.33 target KV budget, surpasses the strongest compressed baseline by an average of 17.39 percentage points across eight BFCL subcategories.
- Achieves up to 2.2× higher throughput, 3.3× lower time per output token, and 0.53× the peak GPU memory of full KV caching.