Training AI Scientists to Replicate Research
Ranking
Overall
88
Content
100
Popularity
59
Observed public metrics from 1 member.
Merged summary
TL;DR - Replica is a scalable benchmark and training environment for AI agents that reproduce published research. Post-trained on it, the 27B-parameter Faraday agent outperforms Claude Opus 4.8 and GPT-5.5 on held-out replication tasks.
- Replica frames paper replication as hypothesis-driven, long-horizon scientific work.
- An automatically generated rubric-based judge provides low-noise rewards aligned with human assessments.
- Faraday delegates coding work to coding agents used as tools.
- Rollout analysis suggests Faraday follows a more scientifically principled replication process.
Sources (1)
Training AI Scientists to Replicate Research
Public signals
Hugging Face upvotes 3
TL;DR - Replica is a scalable benchmark and training environment for AI agents that reproduce published research. Post-trained on it, the 27B-parameter Faraday agent outperforms Claude Opus 4.8 and GPT-5.5 on held-out replication tasks.
- Replica frames paper replication as hypothesis-driven, long-horizon scientific work.
- An automatically generated rubric-based judge provides low-noise rewards aligned with human assessments.
- Faraday delegates coding work to coding agents used as tools.
- Rollout analysis suggests Faraday follows a more scientifically principled replication process.