@dair_ai
Super interesting paper from Meta Superintelligence Labs. This work suggests complex reasoning and search capabilities can emerge solely through self-evolution, challenging the assumption that human supervision is necessary for advanced agent abilities. Let's break down the paper: Self-evolving LLMs can improve without human-curated data by generating their own training problems. However, existing data-free frameworks focus on narrow domains like math and coding. They struggle with open-domain search agents due to limited question diversity and the massive compute required for multi-step reasoning with tools. But what if search agents could evolve from scratch using only an external search engine? This new research introduces Dr. Zero (DeepResearch-Zero), a framework enabling search agents to self-evolve without any training data, demonstrations, or human annotations. The core design: a proposer-solver feedback loop where both models initialize from the same base LLM. The proposer generates diverse questions to train the solver. As the solver improves, it pushes the proposer to create harder yet still solvable queries, establishing an automated curriculum. Standard GRPO requires nested sampling, generating multiple queries each with multiple responses. This becomes computationally prohibitive for multi-turn search agents. Dr. Zero introduces Hop-Grouped Relative Policy Optimization (HRPO), which clusters structurally similar questions by their cross-hop complexity to construct group-level baselines. This eliminates nested sampling while maintaining stable training. The proposer reward balances verifiability and difficulty. If the solver gets everything right, the question is too easy. If it fails completely, too hard. The sweet spot maximizes the learning signal. Results: The data-free Dr. Zero matches or surpasses fully supervised search agents by up to 14.1% on complex QA benchmarks, including HotpotQA, 2WikiMQA, and MuSiQue. On Qwen2.5-7B, Dr. Zero achieves 0.372 average score compared to 0.347 for supervised Search-R1. Paper: https://t.co/CjkbRQNQIl Learn to build effective AI agents in our academy: https://t.co/zQXQt0PMbG