When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation

TL;DR - ResponseGuard is a 2B vision-language safety model that detects harmful streamed responses in one forward pass, avoiding costly chain-of-thought moderation. It outperforms a 3B reasoning-based guard on response harmfulness at roughly 150× lower time cost.

• Pools the request, response, and image into a single representation for classification.
• Supports sentence-by-sentence screening to stop harmful responses during generation.
• Reasoning retains an overall advantage for request harmfulness, with remaining gaps concentrated in image-only cases.
• The authors attribute image-related limitations partly to frozen vision encoders and release their code, models, and datasets.