Not All Eval-Awareness Is Equal: Capabilities Framing Predicts Compliance

TL;DR - This paper finds that eval-awareness is not a behaviorally uniform signal: whether a model frames testing as a capabilities check or a safety-boundary check strongly predicts compliance. This matters because aggregate suppression metrics may obscure whether safety-relevant awareness actually changed.

• On Qwen3-32B and FORTRESS, capabilities-framed awareness predicted 24–46 percentage points more compliance than safety-framed awareness across all steering conditions.
• Chain-of-thought eval-awareness was classified as capabilities-flavored, safety-flavored, both, or neither.
• CoT-prefill experiments shifted compliance in the predicted direction for 10 of 11 prefills, suggesting a causal relationship.
• Identical aggregate eval-awareness suppression rates can correspond to qualitatively different safety outcomes.