Ai2 releases Olmo-core 3 to make developing large mixture-of-experts LLMs more efficient

🤖 AI/ML Daily Signal — Evening Edition 03 Oct 2026 · 16:36 UTC ──────────────────────────────── 🔥 1. How Far is Adam from Natural Gradient Descent? 🔬 arXiv cs.LG (Machine Learning) · Score: 9/10 Investigates the geometric relationship between Adam and natural gradient descent, resolving unresolved questions about a core optimizer used in all deep learning. Understanding this connection helps practitioners make principled choices about optimizer selection and hyperparameter tuning. Read more → ⭐ 2. MoRA: MoE Pruning via Router Bias Learning and Expert Approximation 🔬 arXiv cs.LG (Machine Learning) · Score: 8/10 Proposes MoRA, a pruning technique for Mixture-of-Experts models via router bias learning and expert approximation. This is immediately actionable for practitioners deploying large MoE systems with compute constraints. Read more → ⭐ 3. The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning 🔬 arXiv cs.LG (Machine Learning) · Score: 8/10 Presents a constrained RL method for distilling reasoning from LLMs into smaller students by optimizing for worst-case performance. Directly addresses the bottleneck in efficient LLM deployment. Read more