Kimi K3 技术报告公开了

TL;DR - Moonshot AI (月之暗面) open-sourced Kimi K3's weights and technical report: a 2.8T-parameter MoE model with native vision understanding and a 1M-token context window, reportedly ranking top-5 globally and #1 on WebDev Arena as the first open-weight model to do so, at a fraction of Claude Fable 5's per-task cost.

• Architecture: hybrid attention mixing KDA with Gated MLA at a 3:1 ratio, "Attention Residuals" replacing plain residual connections, and Stable LatentMoE with 896 experts (16 active per token) balanced via Quantile Balancing — ~3× K2's parameters with ~2.5× better training efficiency.
• Native multimodal: text/image/video share one backbone with no separate alignment stage; MoonViT-V2 vision encoder trained from scratch via next-token prediction, which the team says gave more stable gradients than SigLIP-initialized baselines at parity on vision benchmarks.
• Post-training: SFT cold-start agentic model → 9 expert models (general reasoning, general agent, coding agent × three reasoning-effort levels) → Multi-Teacher On-Policy Distillation into a single unified model.
• Infra open-sourced separately: MoonEP (redundant-expert EP communication with provable load balancing), FlashKDA (CUTLASS operator, 1.72–2.22× faster prefill than flash-linear-attention on H20, now an FLA backend), and AgentENV (Firecracker microVM RL sandboxes, 133ms checkpoint / 49ms resume; ~51.2M sandboxes created during training/eval).