ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents

TL;DR - ReCache makes tool-augmented LLM agents more efficient by caching tool and skill schema representations independently, allowing reuse even when resources appear in different combinations or orders. It substantially reduces latency and KV-cache memory with minimal invocation-quality loss.

• Resource-wise attention creates composition-invariant KV blocks by removing cross-resource interactions and using resource-local positions.
• Selective layer/head-group routing and structural-semantic pruning retain only invocation-critical schema information.
• Resource-wise attention achieved 82.3% Inv-F1 versus 82.4% for dense invocation, with a 3.655× time-to-first-token speedup.
• The complete framework reduced allocated KV-tensor memory by 92.43% and accelerated attention by 1.423×.