Dream-RSI from Google DeepMind lets AI agents test new strategies using past search histories, cutting down on attempts and computation in coding and GPU optimization. The method highlights the limits of overly precise instructions and is now open source.
Dream-RSI, a new tool from Google DeepMind, lets AI agents test thousands of strategies without burning through compute or resetting the model. Instead of starting over for each new approach, agents can replay their own search histories to try out different tactics-no restarts, no wasted cycles. In tests with Gemini 3.1 Pro and 3.7 Flash, Dream-RSI sped up code, improved optimization, and made GPU kernel design more efficient, all while reducing the number of attempts needed.
The main idea behind Dream-RSI is straightforward: the agent saves every attempt and result, then uses this archive to test alternative strategies. Researchers can see what would have happened if the agent had chosen different branches or dropped dead ends earlier, all within the same search tree. There's no need for new live executions, which saves time and resources.
Concrete results and the limits of instructions
In one test, Dream-RSI generated a program for a statistical calculation used in finance and genomics that beat both sklearn and glmnet across six datasets. With Gemini 3.1 Pro, average execution time dropped from 3,587 to 2,931 milliseconds, and the number of attempts fell from 550 to 317. Compared to SimpleTES, which needed 51,200 executions, Dream-RSI managed with just 317. For GPU tasks, executions were cut by up to 2.43 times, and in some cases, performance doubled at the same computational budget.
But not every change helps. When researchers turned search histories into explicit instructions for the agent, GPU task performance actually dropped. Too much guidance narrowed the search space and limited the agent's ability to find new solutions. Dream-RSI's replay system showed that strict instructions can backfire in open-ended tasks, making the agent less adaptable.
Adaptive strategy and comparison with other approaches
Dream-RSI adapts as it goes: it cuts back on attempts when progress is good, then explores more when things stall. This is different from fixed strategies that repeat dead ends or online adaptation that burns through compute to test alternatives. By replaying past searches, Dream-RSI avoids these problems and lets researchers compare thousands of policies without extra computational cost.
The method was tested on eight tasks in three domains, always against a baseline with the same starting conditions. Dream-RSI consistently came out ahead, especially in tasks that needed fast code or kernel optimization. While other projects focus on local AI hardware, Dream-RSI's breakthrough is in software strategy.
Overview of self-improvement techniques
Dream-RSI isn't the only project working on smarter self-improving agents. AlphaEvolve, also from DeepMind, combines Gemini Flash for code suggestions, Gemini Pro for evaluation, and an evolutionary algorithm to keep the best results. AutoTTS tests algorithms in simulated environments and outperforms hand-crafted methods with fewer resources. WikiSkill, another Google Research project, turns agent successes and failures into reusable instructions, though Dream-RSI's results suggest that explicit instructions can limit open-ended exploration. Meta's Hyperagents go further, letting agents change their own improvement methods. Across the field, one lesson stands out: using execution history is crucial, but how and when you intervene matters.
Self-improving agents could discover new algorithms, solve complex math, and write faster code by cycling through propose-evaluate-learn-retry loops. The main challenges are huge search spaces, expensive evaluations, and the risk of repeating mistakes. Dream-RSI's open-source release on GitHub gives researchers a way to reuse search data and benchmark strategies, which could speed up progress in AI optimization.
Dream-RSI's replay-based approach changes how AI agents are tested and improved. By showing the hidden costs of rigid instructions and making better use of past searches, DeepMind is pushing for more efficient AI development. The open release points to a clear trend: smarter reuse of search data, not brute-force repetition, will drive the next wave of AI optimization.
