Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Ranking
Overall
69
Content
80
Popularity
42
Observed public metrics from 1 member.
Merged summary
TL;DR - Researchers introduce RRBench, an open-source framework for evaluating local, open-weight coding agents on longitudinal data preparation. Leading 31–35B models achieved up to 87.9% average task completion, suggesting privacy-preserving local agents may be practical for governance-restricted research.
- Benchmarks 20 tasks creating 102 variables from six waves of a British cohort study.
- Covers R-based workflows including category harmonization and multi-wave data merging.
- Automatically evaluates both generated R code and resulting datasets against curated ground truth.
- Tests models across consumer-grade deployment configurations without sending sensitive data to cloud services.
Sources (1)
Agentic coding without the cloud: evaluating open-weight large language models on longitudinal data preparation tasks
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Researchers introduce RRBench, an open-source framework for evaluating local, open-weight coding agents on longitudinal data preparation. Leading 31–35B models achieved up to 87.9% average task completion, suggesting privacy-preserving local agents may be practical for governance-restricted research.
- Benchmarks 20 tasks creating 102 variables from six waves of a British cohort study.
- Covers R-based workflows including category harmonization and multi-wave data merging.
- Automatically evaluates both generated R code and resulting datasets against curated ground truth.
- Tests models across consumer-grade deployment configurations without sending sensitive data to cloud services.