Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
Ranking
Overall
76
Content
80
Popularity
65
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv paper identifying schema-formatted tool specifications as a primary cause of safety degradation in LLM agents, and proposing SafeKeep, an inference-time safeguard that restores refusal behavior without hurting task performance.
- White-box representation analysis shows JSON/schema-style tool specs weaken the model's internal refusal signals, contributing to unsafe tool execution.
- SafeKeep decouples safety judgment from execution: it evaluates requests against flattened textual tool specs while keeping the original schema-formatted specs for actual tool calls.
- Across two benchmarks and four LLMs (white-box and black-box), refusal rate on harmful requests rose from 23.8% → 70.6%, and attack success under observation-level prompt injection fell from 25.6% → 2.5%.
- Reported to outperform existing safeguards while preserving task-handling capability; code and data released at github.com/snowcatsmoking/SafeKeep.
Sources (1)
Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
Public signals
Hugging Face upvotes 2
TL;DR - An arXiv paper identifying schema-formatted tool specifications as a primary cause of safety degradation in LLM agents, and proposing SafeKeep, an inference-time safeguard that restores refusal behavior without hurting task performance.
- White-box representation analysis shows JSON/schema-style tool specs weaken the model's internal refusal signals, contributing to unsafe tool execution.
- SafeKeep decouples safety judgment from execution: it evaluates requests against flattened textual tool specs while keeping the original schema-formatted specs for actual tool calls.
- Across two benchmarks and four LLMs (white-box and black-box), refusal rate on harmful requests rose from 23.8% → 70.6%, and attack success under observation-level prompt injection fell from 25.6% → 2.5%.
- Reported to outperform existing safeguards while preserving task-handling capability; code and data released at github.com/snowcatsmoking/SafeKeep.