‹ back to 2026-08-23

Latent Ordinal Evidence, Misaligned Outputs: Inference-Time Ordinal Lens Alignment for Multimodal LLMs

arXiv cs.CV Multimodal & Generative Haiming Li, Yingsheng Liu, Jingmin Zhu, Siyuan Yan, Xieji Li, Jiajun Sun, Zhen Yu, Zongyuan Ge 2026-08-21

TL;DR - Multimodal LLMs encode strong ordinal information for tasks such as age estimation and disease grading, but their digit-token outputs fail to reflect it. Ordinal Lens Alignment (OLA) recovers this evidence at inference time without modifying the model backbone.

Hidden states yielded ordinal labels with Spearman correlation up to 0.938 across four benchmarks and four MLLM backbones.
Native outputs lagged linear probes by 16–77 absolute accuracy points because the unembedding layer largely filtered out the ordinal direction.
OLA trains lightweight lenses on intermediate-to-deep decoder layers, fuses their predictions, and adjusts only digit-token logits during generation.
It outperformed the LoRA-tuned OrderChain baseline in most settings and improved over an offline lens in every evaluated setting.

view merged work →