VC-Tooler: Learning Compositional and Adaptive Visual Tool Use

TL;DR - VC-Tooler is a vision-language model trained to use visual tools compositionally and adaptively, rather than through fixed tool spaces and rigid invocation patterns. It matters because agentic multimodal reasoning depends on models that can chain tools across steps and adjust based on returned observations, not just ground a single tool call.

• Frames effective visual tool use as three capabilities: grounding tool calls in visual context, composing tools over multiple steps, and adapting reasoning to tool-returned observations — arguing prior work under-addresses the latter two.
• Builds a trajectory bank via a hierarchical synthesis pipeline spanning three levels: single-tool grounding, multi-tool composition, and diverse tool contexts/interfaces.
• Two-stage training: supervised cold start to establish the capabilities, then reinforcement learning rewarding accurate, efficient, and context-aware tool use.
• Reports state-of-the-art results among open-source models on general-purpose and agentic benchmarks, including 95.8% on V* and 35.3% on VTC-Bench, plus claimed transfer to richer inference-time tool settings.