Beyond Component Testing: Validating Agentic AI Systems

TL;DR - A survey of 257 papers argues that agentic AI systems must be validated as multi-step trajectories in context, not as isolated components or one-shot input–output tests. It matters because current assurance practice leaves major gaps for safety-critical agent deployments.

• Proposes a five-dimension validation taxonomy: behavioral, safety, temporal, regulatory, and multi-agent concerns, used to map existing approaches and expose coverage gaps.
• Synthesizes literature across agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance.
• Finds behavioral evaluation comparatively mature, while temporal validity, runtime evidence maintenance, regulatory legibility, and open-ended multi-agent assurance remain under-developed.
• Case studies in medical care, industrial operations, and smart mobility motivate a lifecycle research agenda: bounded-autonomy specs, adversarial trajectory generation, runtime monitoring, and audit-ready evidence structures.