When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

TL;DR - This paper offers an information-bottleneck theory explaining when LLM multi-agent systems (MAS) outperform single-agent systems (SAS), framing multi-agent design as a compression trade-off over bounded inter-agent messages.

• Core insight: SAS keeps its full reasoning trace in one shared context, while MAS uses isolated local contexts linked by bounded "relay" messages; with infinite relay bandwidth, MAS can simply simulate SAS.
• MAS advantage emerges only under bounded relays, where compression removes redundant context (helpful) but risks losing task-relevant information — formalized as an information bottleneck governed by an effective parameter β tied to model capability.
• Validated with 18 controlled experiments across five benchmarks and three model scales: MAS helps most when relays are near-sufficient, especially for weaker models.
• MAS gains shrink or reverse when relays lose information, particularly for stronger models that already extract signal from redundant context, so compression yields little benefit.