HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
Ranking
Overall
72
Content
85
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - HiKV is an algorithm-hardware co-design that compresses LLM KV caches using hierarchical importance filtering. It accelerates attention by up to 7.95× and reduces energy by 90% with negligible accuracy loss.
- First evicts unimportant tokens, then loads only significant elements from retained tokens.
- Uses a reconfigurable importance sorter to support both compression stages in one accelerator.
- Reduces external memory accesses by an additional 1.82–4.87× versus importance-based alternatives at equal accuracy.
- Adds 8% system area while keeping accuracy loss within 1%.
Sources (1)
HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - HiKV is an algorithm-hardware co-design that compresses LLM KV caches using hierarchical importance filtering. It accelerates attention by up to 7.95× and reduces energy by 90% with negligible accuracy loss.
- First evicts unimportant tokens, then loads only significant elements from retained tokens.
- Uses a reconfigurable importance sorter to support both compression stages in one accelerator.
- Reduces external memory accesses by an additional 1.82–4.87× versus importance-based alternatives at equal accuracy.
- Adds 8% system area while keeping accuracy loss within 1%.