笔记本跑7000亿参数GLM！无GPU也行? SSD当显存用火爆GitHub

TL;DR - Colibrì is an open-source, dependency-free C inference framework that runs extremely large mixture-of-experts models on consumer computers by dynamically moving weights among NVMe SSD, RAM, and VRAM. This makes models as large as 744B—and reportedly 2.8T parameters—runnable without a GPU, though performance depends heavily on available memory and storage bandwidth.

• For GLM-5.2, roughly 17B dense parameters occupy about 9.9GB of RAM in INT4, while approximately 370GB of routed expert weights remain on SSD and are loaded only when selected.
• Colibrì reduces SSD traffic through LRU caching, expert-frequency tracking, and one-layer-ahead prefetching; the project reports 71.6% predictability for expert routing between adjacent layers.
• Reported GLM-5.2 throughput ranges from 0.05–0.1 tokens/s on a 12-core CPU with 25GB RAM under cold-cache conditions to about 1.8 tokens/s on a 128GB CPU system.
• The framework supports nine model configurations, including a 2.8T-parameter Kimi K3 setup requiring about 1.6TB of storage and at least 32GB RAM, with GPU acceleration optional.