Local Sparsity Enables Unsupervised LLM Safety Detection
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - This paper proposes an unsupervised LLM safety detector that models only safe activation patterns and flags anomalies using locally masked sparse autoencoders. It could improve detection of novel attacks and harms without requiring labeled unsafe training data.
- The method exploits local sparsity: nearby points in sparse-autoencoder concept space share a small active support.
- The authors provide theoretical justification for masking SAE features locally during anomaly detection.
- Evaluations span multiple model architectures and both capability-focused and safety-specific datasets.
- With 1% out-of-distribution data for calibration, locally sparse methods approach optimal performance while using only 1–2% of SAE neurons.
Sources (1)
Local Sparsity Enables Unsupervised LLM Safety Detection
Public signals
N/A
TL;DR - This paper proposes an unsupervised LLM safety detector that models only safe activation patterns and flags anomalies using locally masked sparse autoencoders. It could improve detection of novel attacks and harms without requiring labeled unsafe training data.
- The method exploits local sparsity: nearby points in sparse-autoencoder concept space share a small active support.
- The authors provide theoretical justification for masking SAE features locally during anomaly detection.
- Evaluations span multiple model architectures and both capability-focused and safety-specific datasets.
- With 1% out-of-distribution data for calibration, locally sparse methods approach optimal performance while using only 1–2% of SAE neurons.