DeepSeek V4 多模态开源，我们把它的视觉链路拆了一遍
Ranking
Overall
78
Content
90
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - DeepSeek has open-sourced the weights and reference inference code for V4-Flash-Vision-Exp, revealing how vision is integrated directly into its long-context, MoE-based agent backbone. The design makes images native context for agent reasoning, but repeated visual processing may become a major efficiency bottleneck.
- A 32-layer ViT encodes images, while a 3×3 Aligner compresses visual features by roughly 9× and projects them into V4’s 4096-dimensional language space, with up to 384 visual tokens per image.
- Visual tokens share the main Transformer and expert pool with text, but receive image-aware attention visibility and distinct MoE routing biases.
- The architecture supports multimodal agent loops in which screenshots, instructions, tool outputs, and actions coexist in a long context exceeding one million tokens.
- Repeatedly encoding mostly unchanged screenshots could dominate multi-step agent workloads, motivating visual caching, frame differencing, and hybrid representations combining pixels with DOM or accessibility data.
Sources (1)
DeepSeek V4 多模态开源，我们把它的视觉链路拆了一遍
Public signals
N/A
TL;DR - DeepSeek has open-sourced the weights and reference inference code for V4-Flash-Vision-Exp, revealing how vision is integrated directly into its long-context, MoE-based agent backbone. The design makes images native context for agent reasoning, but repeated visual processing may become a major efficiency bottleneck.
- A 32-layer ViT encodes images, while a 3×3 Aligner compresses visual features by roughly 9× and projects them into V4’s 4096-dimensional language space, with up to 384 visual tokens per image.
- Visual tokens share the main Transformer and expert pool with text, but receive image-aware attention visibility and distinct MoE routing biases.
- The architecture supports multimodal agent loops in which screenshots, instructions, tool outputs, and actions coexist in a long context exceeding one million tokens.
- Repeatedly encoding mostly unchanged screenshots could dominate multi-step agent workloads, motivating visual caching, frame differencing, and hybrid representations combining pixels with DOM or accessibility data.