TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
Merged summary
TL;DR - TRACE-Router uses a contextual bandit to assign an entire agentic task to one LLM and learns from delayed task-level rewards. This improves accuracy-latency trade-offs by aligning routing decisions with long-horizon workflow outcomes.
- Pins all LLM calls within a task to the model selected at admission.
- Optimizes routing from terminal rewards that jointly reflect accuracy and latency.
- Avoids explicit task-complexity estimation while adapting to changing workloads.
- Beats latency-matched model interpolation by 7–8 accuracy points on τ²-Bench; on Terminal-Bench, it gains 7.1 points over the strongest single model with 36% lower latency.
Sources (1)
TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
TL;DR - TRACE-Router uses a contextual bandit to assign an entire agentic task to one LLM and learns from delayed task-level rewards. This improves accuracy-latency trade-offs by aligning routing decisions with long-horizon workflow outcomes.
- Pins all LLM calls within a task to the model selected at admission.
- Optimizes routing from terminal rewards that jointly reflect accuracy and latency.
- Avoids explicit task-complexity estimation while adapting to changing workloads.
- Beats latency-matched model interpolation by 7–8 accuracy points on τ²-Bench; on Terminal-Bench, it gains 7.1 points over the strongest single model with 36% lower latency.