Adversarial Entropy Inflation Against Gumbel-Based Inference Verification

TL;DR - This paper shows that Gumbel-based defenses against LLM weight exfiltration weaken when attackers control prompts and deliberately increase output entropy. The attack roughly doubles leaked bits per token, indicating that verification thresholds should adapt to local token entropy rather than rely on benign-traffic calibration.

• Character- and script-level prompt disruptions break grammatical and subword structure, expanding the verifier’s admissible token set and covert-channel capacity.
• Across six instruction-tuned models from 1B to 32B parameters and three random seeds, the strongest attack leaked roughly twice as many bits per token as benign prompts.
• The reported slowdown for steganographic exfiltration fell from more than 200× under benign traffic to 60×–118× under adversarial prompts.
• The authors recommend dynamically calibrating jitter-forgiveness thresholds against local token entropy.