Sound Probabilistic Safety Bounds for Large Language Models
Merged summary
TL;DR - This paper introduces a framework for computing formally sound probability bounds on harmful LLM outputs. It enables statistical safety certification, including useful lower bounds when harmful generations are extremely rare.
- Applies Clopper–Pearson confidence intervals to derive probably approximately correct (PAC) harm-probability bounds.
- Prioritizes risky branches of the autoregressive generation tree using latent-space features.
- Formally guarantees that computed lower bounds do not exceed the true harmful-output probability.
- Demonstrates non-trivial lower bounds for state-of-the-art LLMs.
Sources (1)
Sound Probabilistic Safety Bounds for Large Language Models
TL;DR - This paper introduces a framework for computing formally sound probability bounds on harmful LLM outputs. It enables statistical safety certification, including useful lower bounds when harmful generations are extremely rare.
- Applies Clopper–Pearson confidence intervals to derive probably approximately correct (PAC) harm-probability bounds.
- Prioritizes risky branches of the autoregressive generation tree using latent-space features.
- Formally guarantees that computed lower bounds do not exceed the true harmful-output probability.
- Demonstrates non-trivial lower bounds for state-of-the-art LLMs.