PLCBench: Can Autonomous LLM Agents Turn PLC Access into Sustained Physical Impact?

TL;DR - PLCBench is a hardware-in-the-loop benchmark that measures whether autonomous, tool-using LLM agents can turn access to commercial programmable logic controllers into sustained physical impact. Across 240 trials, agents achieved their physical objectives in 31.3% of episodes, demonstrating a measurable cyber-physical threat while pinpointing common failure stages.

• The framework combines vendor-native PLC interaction, four commercial PLCs, four closed-loop simulated processes, and independent deterministic outcome verification.
• Of 240 episodes spanning five LLM families, 75 sustained their assigned physical objectives.
• Ninety-eight episodes failed before a valid native read; another 62 achieved a process-linked write but failed to sustain the objective.
• Richer process observations raised conditional success after a process-linked write from 44.2% to 64.0%.