When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
Merged summary
TL;DR - This paper shows that "physical danger" (harm arising when benign-sounding instructions are executed by embodied agents) is a distinct, separable signal in LLM hidden states from ordinary text-level content danger, and uses that to build a lightweight probe (PRISM) that detects physically unsafe tasks far more accurately than same-scale LLM judges.
- Via hidden-state direction analysis and random-split null tests across Qwen2.5 (3B–32B), Phi-3.5, and SmolLM2, content danger (CD) and physical danger (PD) form separable representational signals.
- PRISM, a single-layer L2-regularized logistic probe over full hidden states, hits 86.2–87.7% accuracy at 11.7–13.7% FPR on SafeAgentBench, while same-scale LLM judges over-block safe tasks at 24.7–39.0% FPR.
- On the new PhysicalSafetyBench-1K (1,000 contrastive physical-risk pairs without explicit harm keywords), PRISM reaches 99.6% accuracy / 0.7% FPR versus a Qwen2.5-3B judge rejecting 67.8% of safe tasks; results also replicate on SafeText and EARBench.
- Takeaway: representation-level probing is proposed as a more reliable approach to embodied/physical safety than text-based content moderation.
Sources (1)
When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
TL;DR - This paper shows that "physical danger" (harm arising when benign-sounding instructions are executed by embodied agents) is a distinct, separable signal in LLM hidden states from ordinary text-level content danger, and uses that to build a lightweight probe (PRISM) that detects physically unsafe tasks far more accurately than same-scale LLM judges.
- Via hidden-state direction analysis and random-split null tests across Qwen2.5 (3B–32B), Phi-3.5, and SmolLM2, content danger (CD) and physical danger (PD) form separable representational signals.
- PRISM, a single-layer L2-regularized logistic probe over full hidden states, hits 86.2–87.7% accuracy at 11.7–13.7% FPR on SafeAgentBench, while same-scale LLM judges over-block safe tasks at 24.7–39.0% FPR.
- On the new PhysicalSafetyBench-1K (1,000 contrastive physical-risk pairs without explicit harm keywords), PRISM reaches 99.6% accuracy / 0.7% FPR versus a Qwen2.5-3B judge rejecting 67.8% of safe tasks; results also replicate on SafeText and EARBench.
- Takeaway: representation-level probing is proposed as a more reliable approach to embodied/physical safety than text-based content moderation.