Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs

TL;DR - This study compares inference-time and lightweight training methods for reducing multilingual factual inconsistencies in Gemma 3 12B Instruct. Simple persona prompting performed best overall, suggesting language-dependent errors partly reflect knowledge selection rather than missing knowledge.

• Evaluates persona prompting, Contrastive Activation Addition (CAA), and two DPO adapter variants across English, German, Spanish, and Bulgarian.
• Persona prompting best balanced factual steering, safety, and out-of-domain cultural generalization.
• CAA produced strong benchmark shifts but was configuration-sensitive and could degrade knowledge.
• DPO adapters made persistent but narrower improvements with weaker transfer.