离开OpenAI和Google后，两位大模型核心负责人决定卷下一代架构

TL;DR - Jerry Tworek (former OpenAI RL lead) and Rohan Anil (former Google Gemini pretraining lead) have founded Core Automation to build a next-generation architecture beyond Transformers, arguing that large-scale RL alone won't reach AGI because today's models cannot keep learning after deployment.

• Core thesis: learning happens almost entirely pre-deployment. In-context learning is flexible but capacity-limited and lost on context compaction; continual fine-tuning risks catastrophic forgetting and needs too much data/compute. They want a third path — meta-learning that bakes "how to learn" into the architecture itself.
• RL is only one form of learning from experience: it relies on massive parallel rollouts with clear actions/rewards, unlike human learning of, say, mathematics via reading, comparison, and internal reasoning. They expect a broader learning algorithm rather than more RL scale.
• Rohan's second critique is compute depth: Transformers have only tens to ~100 layers of fixed forward-pass computation, so chain-of-thought and test-time scaling act as an architectural patch that buys depth one token at a time. He wants deeper computation in fewer steps, with pretraining and RL co-designed alongside kernels and hardware.
• They don't claim Transformers are done — scaling, MoE, sparsity, and speculative decoding still have room — but say scale alone yields a tool dependent on an external human retraining loop. Their AGI bar: a system that improves itself without human involvement. They're building an agent-heavy automated lab to raise architecture-experiment throughput toward tens or hundreds of runs per day.