Solar Open 2 Technical Report
Merged summary
TL;DR - Solar Open 2 is a 250B-parameter Mixture-of-Experts model with 15B active parameters, designed for long-horizon agentic tasks. It combines a 1M-token context window, compute-efficient pretraining, and multi-teacher distillation to achieve strong English and Korean benchmark results.
- Uses hybrid linear and softmax attention without positional encoding to support 1M-token contexts.
- Reuses Solar Open 1’s 5.69B-parameter shared skeleton and trains on a value-curated 10T-token mixture drawn from a 20T-token pool.
- Consolidates 12 domain specialists through Multi-teacher On-Policy Distillation.
- Leads comparably sized open-weight models on MMLU-Pro, LiveCodeBench, and APEX-Agents, while posting the highest reported Korean benchmark average among compared models.
Sources (1)
Solar Open 2 Technical Report
TL;DR - Solar Open 2 is a 250B-parameter Mixture-of-Experts model with 15B active parameters, designed for long-horizon agentic tasks. It combines a 1M-token context window, compute-efficient pretraining, and multi-teacher distillation to achieve strong English and Korean benchmark results.
- Uses hybrid linear and softmax attention without positional encoding to support 1M-token contexts.
- Reuses Solar Open 1’s 5.69B-parameter shared skeleton and trains on a value-curated 10T-token mixture drawn from a 20T-token pool.
- Consolidates 12 domain specialists through Multi-teacher On-Policy Distillation.
- Leads comparably sized open-weight models on MMLU-Pro, LiveCodeBench, and APEX-Agents, while posting the highest reported Korean benchmark average among compared models.