‹ back to 2026-07-20

Scalable LLM Agent Tool Access in the Cloud

arXiv cs.DC LLM Agents Mingxin Li, Enge Song, Yueshang Zuo, Xiaodong Liu, Rong Wen, Qiang Fu, Gianni Antichi, Jian He, Jing Tie, Zhou Shao, Xiaobo Xue, Xiong Xiao, Luyao Zhong, Shaokai Zhang, Jiangu Zhao, Jianyuan Lu, Shize Zhang, Xiaoqing Sun, Changgang Zheng, Zihao Fan, Haonan Li, Tian Pan, Xiaomin Wu, Yang Song, Xing Li, Biao Lyu, Meng Li, Haipeng Dai, Guihai Chen, Shunmin Zhu 2026-07-17

TL;DR - This paper presents a production-deployed cloud gateway for scalable Model Context Protocol tool access. It enables agents to use 3,000+ tools while reducing selection latency and context-token costs.

The gateway handles legacy-service integration, MCP variant compatibility, access control, tool recommendation, and session-aware routing.
Hybrid retrieval achieves 98% Top-15 tool recall.
The system reduces tool-selection time by 8.9× and token usage by 23.8×.
Per-call overhead remains low and stable during scale-out.

view merged work →