S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?

TL;DR - S3Gym is an interactive benchmark testing whether LLM agents can explore their behavior, evaluate experience, and convert feedback into better decisions. Results show that self-improvement is task-dependent and can cause negative transfer rather than consistent gains.

• Evaluates Self-Testing, Self-Judging, and Self-Improvement across seven text-based games with executable verifiers.
• Compares raw interaction history, score-conditioned summary memory, and parameter training as experience-incorporation methods.
• Summaries help when experience compresses into reusable strategies, while raw history works better for precise, state-dependent decisions.
• Parameter training yields substantial gains on some tasks but unstable improvement and severe negative transfer on others.