Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation

TL;DR — A conceptual/framework paper that reframes penetration testing for AI-enabled systems, arguing that security evaluation must move beyond infrastructure compromise to test whether adversaries can induce AI-governed behavior that violates operational objectives. It matters because agentic and LLM-driven systems introduce attack surfaces (prompts, retrieval, tools, memory) that traditional pentesting doesn't cover.

• Defines "AI-enabled penetration" as the feasible induction of AI-governed behavior that violates operational objectives under an explicit threat model, extending rather than replacing conventional pentesting.
• Enumerates adversarial pathways specific to AI systems: prompt injection (direct and indirect), data/retrieval poisoning, sensor manipulation, tool misuse, and agentic misalignment.
• Proposes a testing workflow: identify objectives → map AI-governed behavior → analyze adversarial influence surfaces → define behavioral failure criteria → run scenario-based tests → report evidence linking action to objective violation.
• Illustrated with a running example of an AI-enabled security operations center (SOC) assistant; this is a framework/definitions paper with no empirical results reported.