Kimi K3 技术报告公开了
Ranking
Overall
75
Content
85
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - Moonshot AI (月之暗面) open-sourced Kimi K3's weights and technical report: a 2.8T-parameter MoE model with native vision understanding and a 1M-token context window, reportedly ranking top-5 globally and #1 on WebDev Arena as the first open-weight model to do so, at a fraction of Claude Fable 5's per-task cost.
- Architecture: hybrid attention mixing KDA with Gated MLA at a 3:1 ratio, "Attention Residuals" replacing plain residual connections, and Stable LatentMoE with 896 experts (16 active per token) balanced via Quantile Balancing — ~3× K2's parameters with ~2.5× better training efficiency.
- Native multimodal: text/image/video share one backbone with no separate alignment stage; MoonViT-V2 vision encoder trained from scratch via next-token prediction, which the team says gave more stable gradients than SigLIP-initialized baselines at parity on vision benchmarks.
- Post-training: SFT cold-start agentic model → 9 expert models (general reasoning, general agent, coding agent × three reasoning-effort levels) → Multi-Teacher On-Policy Distillation into a single unified model.
- Infra open-sourced separately: MoonEP (redundant-expert EP communication with provable load balancing), FlashKDA (CUTLASS operator, 1.72–2.22× faster prefill than flash-linear-attention on H20, now an FLA backend), and AgentENV (Firecracker microVM RL sandboxes, 133ms checkpoint / 49ms resume; ~51.2M sandboxes created during training/eval).
Sources (1)
Kimi K3 技术报告公开了
Public signals
N/A
TL;DR - Moonshot AI (月之暗面) open-sourced Kimi K3's weights and technical report: a 2.8T-parameter MoE model with native vision understanding and a 1M-token context window, reportedly ranking top-5 globally and #1 on WebDev Arena as the first open-weight model to do so, at a fraction of Claude Fable 5's per-task cost.
- Architecture: hybrid attention mixing KDA with Gated MLA at a 3:1 ratio, "Attention Residuals" replacing plain residual connections, and Stable LatentMoE with 896 experts (16 active per token) balanced via Quantile Balancing — ~3× K2's parameters with ~2.5× better training efficiency.
- Native multimodal: text/image/video share one backbone with no separate alignment stage; MoonViT-V2 vision encoder trained from scratch via next-token prediction, which the team says gave more stable gradients than SigLIP-initialized baselines at parity on vision benchmarks.
- Post-training: SFT cold-start agentic model → 9 expert models (general reasoning, general agent, coding agent × three reasoning-effort levels) → Multi-Teacher On-Policy Distillation into a single unified model.
- Infra open-sourced separately: MoonEP (redundant-expert EP communication with provable load balancing), FlashKDA (CUTLASS operator, 1.72–2.22× faster prefill than flash-linear-attention on H20, now an FLA backend), and AgentENV (Firecracker microVM RL sandboxes, 133ms checkpoint / 49ms resume; ~51.2M sandboxes created during training/eval).