ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessions
Ranking
Overall
78
Content
95
Popularity
39
Observed public metrics from 1 member.
Merged summary
TL;DR - ASLEval is an authorization-aware framework for measuring privacy exposure across all visible exits in multi-step LLM agent sessions. It shows that evaluations focused on a single expected output can substantially underestimate leakage.
- Expected-outlet-only evaluation missed 46.9% of exposure captured by the union of visible exits.
- Attacker self-reports exhibited both omissions and high false-discovery rates, making them unreliable privacy proxies.
- Schema-aligned internal evidence generally appeared before visible exposure at the request or probe level, helping diagnose leakage paths.
- Restricting model-visible tool returns altered exposure pathways but could also eliminate successful completion of legitimate tasks, highlighting a privacy–utility tradeoff.
Sources (1)
ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessions
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - ASLEval is an authorization-aware framework for measuring privacy exposure across all visible exits in multi-step LLM agent sessions. It shows that evaluations focused on a single expected output can substantially underestimate leakage.
- Expected-outlet-only evaluation missed 46.9% of exposure captured by the union of visible exits.
- Attacker self-reports exhibited both omissions and high false-discovery rates, making them unreliable privacy proxies.
- Schema-aligned internal evidence generally appeared before visible exposure at the request or probe level, helping diagnose leakage paths.
- Restricting model-visible tool returns altered exposure pathways but could also eliminate successful completion of legitimate tasks, highlighting a privacy–utility tradeoff.