挑战 1 比特！ETH Zürich 秦浩桐：如何把大模型「塞进」小设备？| IJCAI 2026
Ranking
Overall
71
Content
80
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - At an IJCAI 2026 workshop, ETH Zürich researcher Haotong Qin presented BiLLM and SqueezeLLM, post-training quantization methods designed to run large language models on memory-constrained devices. They push weights toward 1–2 bits without retraining while highlighting unresolved quality and hardware limitations.
- BiLLM isolates a small set of sensitive, channel-structured weights and binarizes most remaining weights, compressing models to about 1.08 bits per weight using roughly 30–60 minutes on one GPU.
- The article reports that BiLLM achieved 8.41 perplexity on WikiText-2 for LLaMA-2 70B, while retaining coherent text generation.
- SqueezeLLM dynamically allocates precision across groups of 128 weights according to sensitivity, improving 2-bit quantization without irregular bitmap storage or incompatible memory access.
- Key open problems include quality degradation on complex instructions below 2 bits, limited native INT2/1-bit hardware support, and quantizing activations and KV caches.
Sources (1)
挑战 1 比特！ETH Zürich 秦浩桐：如何把大模型「塞进」小设备？| IJCAI 2026
Public signals
N/A
TL;DR - At an IJCAI 2026 workshop, ETH Zürich researcher Haotong Qin presented BiLLM and SqueezeLLM, post-training quantization methods designed to run large language models on memory-constrained devices. They push weights toward 1–2 bits without retraining while highlighting unresolved quality and hardware limitations.
- BiLLM isolates a small set of sensitive, channel-structured weights and binarizes most remaining weights, compressing models to about 1.08 bits per weight using roughly 30–60 minutes on one GPU.
- The article reports that BiLLM achieved 8.41 perplexity on WikiText-2 for LLaMA-2 70B, while retaining coherent text generation.
- SqueezeLLM dynamically allocates precision across groups of 128 weights according to sensitivity, improving 2-bit quantization without irregular bitmap storage or incompatible memory access.
- Key open problems include quality degradation on complex instructions below 2 bits, limited native INT2/1-bit hardware support, and quantizing activations and KV caches.