01What Karpathy Auto Research Is and How It Works
Karpathy auto research is a method where an AI agent runs experiments on its own, without human help, to improve a system. Andre Karpathy, who co-founded OpenAI, shared the project in March 2026. The idea is simple: you set a goal, tell the AI what "better" looks like, and then walk away.
The AI tries things, checks if they work, keeps the wins, and throws out the losses. It repeats this loop over and over, all night long. So while you sleep, the AI might run hundreds of experiments.
When you wake up, you get a report of what improved.
In the episode, Karpathy's AI ran 126 experiments in one night and made his system 11% faster. The AI even found things that Karpathy had missed after 20 years of work. That is the power of auto research.
02Key Features of the Auto Research Pattern
The auto research pattern has three simple steps. First, try something. Second, measure if it works.
Third, keep it or throw it out and try something else. Humans have always done these steps, but the AI does them much faster.
- Speed: The AI runs about 12 experiments per hour on a single GPU, so about 100 while you sleep.
- Volume: It can run thousands of experiments in days. In one two-day run, it completed about 700 experiments.
- No tiredness: The AI doesn't get bored, distracted, or skip work on a Friday afternoon.
- Parallel power: With 16 GPUs, it can run over 100 experiments per hour at once, catching interactions between variables.
The goal, as Karpathy said, is not to replace one researcher. It is to emulate an entire research community. A swarm of agents works on your stuff while you sleep.
03How to Set Up Auto Research for Your Own Projects
Setting up auto research does not require a huge budget or a big team. The entire repository is just 630 lines of code. It runs on a single GPU, which you can rent by the hour for a few dollars.
- Get the auto research code from the public repository. It has over 42,000 GitHub stars.
- Set up a GPU. You can rent one from a cloud provider for a few dollars per hour.
- Define your goal clearly. Tell the AI what result you want to improve.
- Decide what "better" looks like. This could be a metric like speed, accuracy, or click-through rate.
- Launch the agent and let it run. It will start experimenting on its own.
- Check the results when you wake up or after a set time. The AI will show what worked.
You don't need to be a data scientist. You need a clear metric, a tool that can run experiments, and something to optimize.
04How to Use Auto Research in Your Business
You can use auto research for any task where you can measure results. For example, a freelancer might test different pitch emails to see which gets more replies. An agency owner might test onboarding sequences to see which keeps clients longer.
An e-commerce store might test product page layouts to see which drives more add-to-cart clicks.
The episode gives a real example: Auto Voice Evals used the loop to optimize prompts. In 20 automated iterations, it improved a scheduling agent's success rate from 25% to 100%. Interestingly, the final prompt was shorter than the original.
To use it, you set up the system with your goal and metric. The AI runs experiments, measures results, picks a winner, and starts the next round. You just define the direction and let the AI run the laps.
05Real Results and Tests Shown in the Episode
The episode shows several impressive results from auto research. First, in one overnight run, Karpathy's agent completed 126 experiments and made his system 11% faster. The next run, over two full days, involved about 700 experiments and found roughly 20 improvements.
Those improvements stacked to reduce training time from 2.02 hours to 1.8 hours.
Shopify's CEO, Toby Lütke, tried it on Shopify's internal systems. His agent found a 53% speedup in performance and used 61% fewer resources through 93 automated changes. It also optimized an AI model: after 37 experiments overnight, the model got 19% better.
The new model had 0.8 billion parameters and outperformed the old 1.6 billion parameter model.
Another test used 16 GPUs in parallel. Over eight hours, the agent submitted about 910 experiments and drove measurable improvements. With one GPU, it tested one idea at a time.
With 16, it ran grids of 10 to 13 experiments per wave, catching interactions that sequential testing missed.
06Tips, Limits, and Mistakes to Avoid
Karpathy himself noted that his original setup was a small project: 630 lines of code, one GPU, a contained system. He acknowledged that scaling to a frontier AI lab with millions of lines of code is much more complex. But he said it is just engineering and it will work.
One limit is that auto research is not magic at large scale yet. It is not replacing OpenAI's whole research team tomorrow. But the pattern already works on real business systems, as the Shopify example shows.
A common mistake is to ignore the pattern because it seems technical. Don't let that happen. The winning move is to understand the concept, know what to measure, and take action. The gap between businesses that run 30 tests a year and those that run 36,000 will grow fast.
07Who Auto Research Is For and Why It Matters
Auto research is for anyone who wants to improve something measurable without spending hours doing manual tests. That includes marketers, content creators, e-commerce owners, and agency leaders. You do not need a data science degree to apply the pattern.
Why does it matter? Because the pace of improvement is about to change. Karpathy said all LLM frontier labs will adopt auto research.
It is a "final boss battle." AI can now improve itself at machine speed, which is much faster than human speed.
The episode gives a powerful example: in 17 hours, AI agents using the pattern rediscovered machine learning milestones like RMS norm and tied embeddings. Human researchers took nearly eight years to formalize those ideas. The businesses that apply this loop to their marketing, offers, and processes will have a huge advantage over those that do not.
