Move by Move: Measuring and Steering How LLMs Conduct Psychotherapy

TL;DR - This paper introduces a psychologist-validated ontology of ten therapeutic moves to measure how LLMs conduct psychotherapy. Frontier models differ substantially from human clinicians, but tool-based guidance improves their alignment without fine-tuning.

• Models use inquiry at up to three times the human rate while underusing psychoeducation.
• LLMs tend to continue strategies introduced by human clinicians rather than initiate those strategies independently.
• A judge-based annotation approach scales the ontology while matching expert agreement.
• Exposing therapeutic moves as tools roughly halves deviation from human move distributions and improves turn-level alignment by 7–9 percentage points.