Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression
Ranking
Overall
67
Content
80
Popularity
36
Observed public metrics from 1 member.
Merged summary
TL;DR - SPIRAL is a self-supervised framework that fixes "cross-path inconsistency" in Vision-Text Compression, where text rendered into images is encoded by a ViT as glyphs/layout rather than linguistic semantics. It matters because it lets long texts be compressed into far fewer visual tokens while nearly matching native-text performance.
- Diagnoses cross-path inconsistency via rendering perturbation experiments, identifying it as the key VTC bottleneck since ViTs are pretrained on natural images.
- Uses only the model's own text-path behavior as supervision — no external teachers or extra annotations — at two granularities: token-level on-policy distillation (OPD) and sequence-level preference optimization (DPO).
- On VTCBench, lifts Qwen3-VL-8B from 35.10 to 54.02 overall, close to native text input (55.60) and beating models up to 30x larger.
- OPD is sample-efficient and strongest on retrieval; DPO is stronger on reasoning/memory and scales better with data; gains also transfer to out-of-domain benchmarks.
Sources (1)
Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - SPIRAL is a self-supervised framework that fixes "cross-path inconsistency" in Vision-Text Compression, where text rendered into images is encoded by a ViT as glyphs/layout rather than linguistic semantics. It matters because it lets long texts be compressed into far fewer visual tokens while nearly matching native-text performance.
- Diagnoses cross-path inconsistency via rendering perturbation experiments, identifying it as the key VTC bottleneck since ViTs are pretrained on natural images.
- Uses only the model's own text-path behavior as supervision — no external teachers or extra annotations — at two granularities: token-level on-policy distillation (OPD) and sequence-level preference optimization (DPO).
- On VTCBench, lifts Qwen3-VL-8B from 35.10 to 54.02 overall, close to native text input (55.60) and beating models up to 30x larger.
- OPD is sample-efficient and strongest on retrieval; DPO is stronger on reasoning/memory and scales better with data; gains also transfer to out-of-domain benchmarks.