FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Verification
Ranking
Overall
75
Content
90
Popularity
40
Observed public metrics from 1 member.
Merged summary
TL;DR - FaithEyes is a multi-agent framework that trains vision-language models to use image-processing tools only when their outputs genuinely aid reasoning. It improves tool faithfulness while maintaining competitive or superior benchmark accuracy.
- A VLM judges whether each cropped or manipulated process image helps answer the question.
- Helpfulness judgments guide subsequent reasoning and scale rewards to discourage decorative or misaligned tool calls.
- At inference, the model acts as its own judging subagent, avoiding reliance on an external evaluator.
- Training uses a two-stage supervised fine-tuning and reinforcement learning pipeline on adapted open-source data.
Sources (1)
FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Verification
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - FaithEyes is a multi-agent framework that trains vision-language models to use image-processing tools only when their outputs genuinely aid reasoning. It improves tool faithfulness while maintaining competitive or superior benchmark accuracy.
- A VLM judges whether each cropped or manipulated process image helps answer the question.
- Helpfulness judgments guide subsequent reasoning and scale rewards to discourage decorative or misaligned tool calls.
- At inference, the model acts as its own judging subagent, avoiding reliance on an external evaluator.
- Training uses a two-stage supervised fine-tuning and reinforcement learning pipeline on adapted open-source data.