01What Is ARC-AGI-3 and How Does It Work?
ARC-AGI-3 is a new AGI test that measures how well AI can learn and adapt to brand new tasks. It was created by Francois Chollet, the builder of Keras, and launched on March 25th. In this test, humans score 100% while AI scores less than 1%.
The test uses video game-like environments where agents must explore, learn, and plan. There are over 1,000 levels across 150 different environments. No instructions are given, so the AI must figure out the rules by itself.
The scoring is not just pass or fail. It measures how many actions the AI takes compared to a human. Humans quickly figure out the rules and play efficiently.
AI, even the best models like Gemini, GPT, and Claude, scores near zero.
This test is designed to be impossible to memorize. Every environment is handcrafted and novel, so the AI cannot rely on training data. That makes the less than 1% score a real, honest measure of capability.
02Why ARC-AGI-3 Is Different From Other Benchmarks
ARC-AGI-3 exposes a major problem with how AI is currently evaluated. Researchers found that selective model submissions inflated leaderboard scores by up to 100 points through cherry-picking. Meta tested 27 private variants of LLAMA4 before publishing, and the public version dropped from number two to positions 32 through 35.
Yann LeCun even confirmed to the Financial Times that some results were fudged a little bit. Documents from a former Liquid AI executive showed that models like LLAMA3, Google Gemma, and Mistral7b could reproduce benchmark test sets verbatim. They had seen the questions before.
On ARC itself, clean data scored 0.4555 versus contaminated data at 0.5632. That is a 24% inflation just from test leakage. These findings show why static benchmarks are broken and why ARC-AGI-3 is different.
ARC-AGI-3 cannot be studied for because every environment is new. There is no test set to leak, and the game changes as you act. This makes the low scores from AI real, not inflated.
03How to Get Started With ARC-AGI-3
ARC-AGI-3 is now a live API, so anyone can build an agent and test it. The environment engine runs at 2,000 frames per second, which allows researchers to test at scale. The benchmark is built to be attacked fast, and there is $2 million in prizes for the ARC Prize 2026.
- Go to the ARC Prize website and find the ARC-AGI-3 API documentation.
- Create an account and get your API key.
- Build an AI agent using your preferred framework or language.
- Connect your agent to the API so it can receive test environments.
- Run your agent against the levels and get your score.
- Iterate on your approach, focusing on adaptation and learning.
You do not need a massive model to start. The test rewards efficiency and adaptation, not just scale. That is why it is open to everyone, not just big labs.
04How to Use ARC-AGI-3 to Test Your AI Agent
Using ARC-AGI-3 means setting up your agent to explore and learn in a new environment. In the preview challenge, researcher Evgeny Rudakov built a training-free graph-based approach. It mapped every state and action like a flowchart, and it solved a median of 30 out of 52 levels.
This beat frontier LLM-based agents by using a better memory structure.
The test measures several key skills. Exploration is about gathering information from a new environment by deciding what to try. Humans poke around and build a mental map, but AI has no built-in strategy for that. Persistent memory is about remembering what you tried before. Even the best models fail basic recall tasks about one in every eight tries.
Goal acquisition is another core skill. The test gives you no objective, so you have to infer it. Humans do this naturally, but AI does not.
Finally, the test checks credit assessment: can the AI trace back which decision 10 steps ago caused a problem? For AI agents, this is almost impossible.
To use ARC-AGI-3 effectively, focus on these skills in your agent design. Build in loops that let the agent refine its own solutions based on feedback.
05Results and Comparisons From the Episode
The episode shows a huge gap between AI and human performance. On ARC-AGI-3, frontier models scored 0% while human participants scored 100%. That is the most honest picture of AI capability we have.
Compare that to previous versions. ARC-AGI-1 launched in 2019, and AI scored near zero for years. It eventually hit 87%, but only through brute force, running tasks thousands of times.
ARC-AGI-2 launched in 2025, and Claude Opus hit 37.6% for $2.20 per task. Gemini 3 DeepThink hit 84.6%, but those scores came from engineering around the benchmark, not true intelligence.
ARC-AGI-3 resets AI to less than 1%. This shows that each new version cracks faster but requires new ideas, not just more compute. Researcher Alexia Jalicka Martineau built a network with only 7 million parameters and zero pre-training, and it hit 45% on ARC-AGI-1.
That is competitive with GPT-4, which has hundreds of billions of parameters.
The key takeaway is that adaptation matters more than scale. The models that do well are the ones that can learn on the fly.
06Tips, Limits, and Mistakes to Avoid
Do not treat ARC-AGI-3 like other benchmarks. Avoid trying to memorize tasks because the test is designed to prevent that. The environments are new every time, so a cheat sheet will not help.
One big mistake is thinking that a larger model will automatically do better. The evidence shows that a 7-million-parameter network can outperform massive models if it adapts well. Focus on building better memory and exploration strategies instead.
Another mistake is ignoring the problem of error compounding. If an agent has 85% accuracy per action, a 10-step workflow only works about 20% of the time. Errors add up, so you need to keep accuracy high and build in feedback loops.
Finally, do not overhype your results. The labs have been caught cherry-picking and fudging scores. On ARC-AGI-3, a real score means your agent can learn new tasks, not just repeat old ones.
07Who Should Care About ARC-AGI-3 and Why
ARC-AGI-3 matters for anyone using AI or building AI systems. If you are a business owner, freelancer, or creator, this test shows the real limits of current AI. It tells you what your tools can actually do when they face new problems.
The episode stresses that all four major AI labs—Anthropic, Google DeepMind, OpenAI, and XAI—now officially report ARC-AGI performance on their model cards. That means every meaningful jump on this test reflects real capability improvement, not marketing.
Gartner predicts that 40% of agentic AI projects will fail this year due to brittle workflows, not model intelligence. Knowing the limitations helps you build better workflows and avoid these failures. Also, 42% of enterprises plan to deploy AI agents this year, so the investment is real.
If you start building your AI skills now, you will be ready when the gap closes. The people who wait will be left behind. This is not a test to ignore; it is a roadmap.
08The Future of AI According to ARC-AGI-3
Francois Chollet has cut his AGI timeline from 10 years to five. That is because of progress in test-time adaptation and refinement loops. These are systems that improve their own solutions using feedback from the environment.
The research direction is clear: teach AI to hypothesis test like a scientist, not pattern match like a search engine. The $2 million prize pool and the live API are pushing researchers to explore this.
And the breakthroughs will flow to everyone. ARC-AGI-3 is open source, so any improvement benefits the whole field. The trajectory shows that each version gets cracked faster, but always with new ideas, not just more compute.
So, the gap is real but closing. The AI tools you use today are already useful, even with a score of less than 1%. Imagine what they will do when they reach 5% or 20%.
Start building now.
