ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
Ranking
Overall
87
Content
95
Popularity
69
Observed public metrics from 1 member.
Merged summary
TL;DR - ReCache makes tool-augmented LLM agents more efficient by caching tool and skill schema representations independently, allowing reuse even when resources appear in different combinations or orders. It substantially reduces latency and KV-cache memory with minimal invocation-quality loss.
- Resource-wise attention creates composition-invariant KV blocks by removing cross-resource interactions and using resource-local positions.
- Selective layer/head-group routing and structural-semantic pruning retain only invocation-critical schema information.
- Resource-wise attention achieved 82.3% Inv-F1 versus 82.4% for dense invocation, with a 3.655× time-to-first-token speedup.
- The complete framework reduced allocated KV-tensor memory by 92.43% and accelerated attention by 1.423×.
Sources (1)
ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents
Public signals
Semantic Scholar citations 1 · Semantic Scholar influential citations 0
TL;DR - ReCache makes tool-augmented LLM agents more efficient by caching tool and skill schema representations independently, allowing reuse even when resources appear in different combinations or orders. It substantially reduces latency and KV-cache memory with minimal invocation-quality loss.
- Resource-wise attention creates composition-invariant KV blocks by removing cross-resource interactions and using resource-local positions.
- Selective layer/head-group routing and structural-semantic pruning retain only invocation-critical schema information.
- Resource-wise attention achieved 82.3% Inv-F1 versus 82.4% for dense invocation, with a 3.655× time-to-first-token speedup.
- The complete framework reduced allocated KV-tensor memory by 92.43% and accelerated attention by 1.423×.