Screening Is Effective for Visual Recognition
Merged summary
TL;DR — VisionScreen adapts the "Screening" mechanism from language modeling to Vision Transformers, letting each image patch independently include or reject other patches by query-key similarity instead of softmax competition. It matters because it offers an alternative attention paradigm that can filter irrelevant background/redundant patches in visual recognition.
- Standard ViT self-attention uses softmax-normalized relative weights across all patches, so it cannot independently judge patch relevance or explicitly reject background/redundant patches.
- Screening (from language models) evaluates each token's relevance via query-key similarity and excludes low-relevance tokens through thresholding, rather than competitive normalization.
- VisionScreen extends this to a 2D spatial grid, so each patch aggregates only content-wise and spatially relevant patches, using absolute rather than relative relevance estimation.
- On image classification benchmarks, the method reportedly outperforms conventional ViT; specific datasets and numerical gains are not provided in the content.
Sources (1)
Screening Is Effective for Visual Recognition
TL;DR — VisionScreen adapts the "Screening" mechanism from language modeling to Vision Transformers, letting each image patch independently include or reject other patches by query-key similarity instead of softmax competition. It matters because it offers an alternative attention paradigm that can filter irrelevant background/redundant patches in visual recognition.
- Standard ViT self-attention uses softmax-normalized relative weights across all patches, so it cannot independently judge patch relevance or explicitly reject background/redundant patches.
- Screening (from language models) evaluates each token's relevance via query-key similarity and excludes low-relevance tokens through thresholding, rather than competitive normalization.
- VisionScreen extends this to a 2D spatial grid, so each patch aggregates only content-wise and spatially relevant patches, using absolute rather than relative relevance estimation.
- On image classification benchmarks, the method reportedly outperforms conventional ViT; specific datasets and numerical gains are not provided in the content.