Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks

TL;DR - Researchers introduce RRBench, an open-source framework for evaluating local, open-weight coding agents on longitudinal data preparation. Leading 31–35B models achieved up to 87.9% average task completion, suggesting privacy-preserving local agents may be practical for governance-restricted research.

• Benchmarks 20 tasks creating 102 variables from six waves of a British cohort study.
• Covers R-based workflows including category harmonization and multi-wave data merging.
• Automatically evaluates both generated R code and resulting datasets against curated ground truth.
• Tests models across consumer-grade deployment configurations without sending sensitive data to cloud services.