RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
Ranking
Overall
78
Content
95
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - RAG-Safety-Bench evaluates how retrieval-augmented generation affects LLM safety by isolating retrieval context from retriever quality. Results across five open-source LLMs indicate that existing safety guardrails may not reliably carry over to RAG deployments.
- Tests four settings: no RAG, oracle harmful-answer documents, related documents without answers, and random safe documents.
- Finds an inverse relationship between benign capability and unsafe capability.
- Shows that baseline model safeguards do not guarantee safe outputs after retrieval.
- Finds model-specific evidence that even benign retrieved documents can increase unsafe generation.
Sources (1)
RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - RAG-Safety-Bench evaluates how retrieval-augmented generation affects LLM safety by isolating retrieval context from retriever quality. Results across five open-source LLMs indicate that existing safety guardrails may not reliably carry over to RAG deployments.
- Tests four settings: no RAG, oracle harmful-answer documents, related documents without answers, and random safe documents.
- Finds an inverse relationship between benign capability and unsafe capability.
- Shows that baseline model safeguards do not guarantee safe outputs after retrieval.
- Finds model-specific evidence that even benign retrieved documents can increase unsafe generation.