挑战 1 比特！ETH Zürich 秦浩桐：如何把大模型「塞进」小设备？| IJCAI 2026

TL;DR - At an IJCAI 2026 workshop, ETH Zürich researcher Haotong Qin presented BiLLM and SqueezeLLM, post-training quantization methods designed to run large language models on memory-constrained devices. They push weights toward 1–2 bits without retraining while highlighting unresolved quality and hardware limitations.

• BiLLM isolates a small set of sensitive, channel-structured weights and binarizes most remaining weights, compressing models to about 1.08 bits per weight using roughly 30–60 minutes on one GPU.
• The article reports that BiLLM achieved 8.41 perplexity on WikiText-2 for LLaMA-2 70B, while retaining coherent text generation.
• SqueezeLLM dynamically allocates precision across groups of 128 weights according to sensitivity, improving 2-bit quantization without irregular bitmap storage or incompatible memory access.
• Key open problems include quality degradation on complex instructions below 2 bits, limited native INT2/1-bit hardware support, and quantizing activations and KV caches.