JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management

TL;DR - JustFit is an MLX-based runtime that serves Qwen3.8-27B with more than 200K tokens of context on a 24 GiB MacBook by compressing and managing execution state just in time. It expands local long-context inference capacity without depending on additional model-weight quantization.

• Completed 196,608 input plus 16,384 output tokens in three runs, reaching 212,992 positions—6.93× the mlx-vlm baseline.
• Combines compressed KV execution, component residency swapping, and state-preserving serving transitions.
• Achieved 19.11 tokens/s on a 32K-input, 64-output test and a median 16,374 MiB peak footprint on a repeated 32K+6K workload.
• The integrated runtime correctly answered 29 of 30 AIME 2026 problems.