This New AGI Test Just Shocked The World

This new AGI test just shocked the world. AGI, the most hyped word in tech right now. Every lab, OpenAI, Google, Anthropic, they all say they're close. They're spending billions.

They all have benchmark scores to prove how smart their AI is. And then yesterday, a guy named Francois Chollet dropped a test that made every single one of them look broken. Humans score 100%, AI scores less than 1%. That's not a typo.

The test is called ARC-AGI-3. It launched March 25th. Over 1,000 levels across 150 environments where agents must explore, learn and adapt. And it just gave the most honest picture of where AI actually is versus where the labs claim it is.

Here's what makes this even more striking. Gemini 3 DeepThink, it recently hit 84.6% on the previous version of the test. Gemini 3.1 Pro was sitting at 77%. OpenAI has been telling everyone AGI is basically here.

The labs are flexing benchmark scores every single week. And then ARC-AGI-3 drops and frontier models scored 0% whilst human participants achieved 100%. That's a gap. We are actually dealing with.

So who is Francois Chollet and why does his test matter more than every other benchmark you've ever seen? Chollet built Keras, an open source deep learning library used by over two and a half million developers. He's not a blogger. He's one of the most respected people in AI research.

And for years, he's had the same argument. The industry is measuring intelligence wrong. The ARC Prize Foundation defines AGI as a system that can match the learning efficiency of humans. Not automate jobs, not pass exams, learn on the fly with something it's never seen before.

Here's how Chollet puts it. You can always achieve skill by memorization, effectively just storing a lookup table of everything you need to do. Intelligence is the efficiency with which you make sense of new things, of tasks you've never seen before. That's a gap.

AI has read everything. Brilliant tasks that exist in its training data. Walk into something genuinely new, however, and it breaks. ARC-AGI uses video game like environments where agents must perceive, plan, and act across multiple steps to achieve long horizon goals.

No instructions. You figure out the rules by yourself, then play efficiently. Every environment is handcrafted and novel. You can't just memorize your way through it.

And the scoring isn't just pass and fail. It measures how many actions it takes you to complete each level compared to a human. Now, humans sit down, explore for a few minutes, figure out the rules, start winning efficiently. That's fluid intelligence.

AI, less than 1%. Gemini, GPT, Claude, all of them, zero. And here's something most people covering this story are completely missing. This isn't just about one test being hard.

It's about the entire way AI benchmarks work being broken. And ARC-AGI3 being the only test that's actually cheap proof. Because here's what's been happening with every other benchmark. Researchers analyzing 2.8 million model comparison records found that selective model submissions inflated leaderboard scores by up to 100 points through cherry picking.

Meta tested 27 private variants before publishing LLAMA4. The public version dropped from number two to positions 32 through 35. Yann LeCun later confirmed to the Financial Times that the results were fudged a little bit. Documents from a former Liquid AI executive showed that multiple major models, including LLAMA3, Google, Gemma, and Mistral7b could reproduce benchmark test sets verbatim.

They'd seen the questions before. That's not intelligence, that's memorization. On ARC specifically, clean data scored 0.4555 versus contaminated data at 0.5632. A 24% inflation from test leakage alone.

Andy Konwinski, whose Lord Institute donated $25,000 to ARC Prize, said this benchmark goes at the heart of this gap between actually measuring for AGI and the standard set of benchmark suites that the big labs use in the rat race of getting 0.5% improvement over every state-of-the-art model for a week. ARC AGI3 can't be studied for. Every environment is brand new. There's no test set to leak.

The game changes as you act and you can't retry your way through it. So when AI scores less than 1% here, that score is real. That's the actual capability of the best AI systems on earth when the cheat sheet is removed. Let that sink in.

This is the open loop I want you to sit with because understanding what's actually missing in AI, not the marketing version, the real version, tells you exactly what's about to change. Quick one. If you want to stay ahead of this and know exactly which AI tools to build your business around right now, the AI Profit Boardroom is where 2,600 business owners, freelancers, and creators are doing that every single day. Four live coaching calls every week, daily tutorials, 30-day roadmaps, local map to find people in your city, and 24-7 support.

Link in the comments and description. Now, here's what's actually missing. ARC AGI3 tests for specific capabilities on the fly. Learning, exploration, memory, and goal acquisition.

Things that interactive benchmarks test that static benchmarks completely ignore. Exploration, for example. Can the AI officially gather information from a brand new environment by deciding what to try? Humans poke around.

They test things. They build a mental map. AI has no built-in strategy for, I don't know what's happening. Let me find out.

Persistent memory across actions. So can the AI remember what it tried five moves ago and use that to improve the next move? Even the best models fail. Basic recall tasks around one in every eight tries.

Applying information meaningfully across a long task chain stays deeply unreliable. Goal acquisition. ARC AGI3 gives you no objective. You infer it.

Now, of course, humans do this naturally. AI doesn't. And credit assessment and assignment over long horizons. So can you, for example, trace back which decision you made 10 steps ago, which is causing your current problem?

For AI agents, it's almost impossible. If an agent achieves 85% accuracy per action, a 10-step workflow only completes correctly about 20% of the time. Errors compound with every step. That's a real reason AI agents didn't take over in 2025 the way everyone predicted.

Gartner predicts that 40% of agentic AI projects this year will fail. Not because of model intelligence, but because of brittle workflows and missing foundations. Now, here's something that is fascinating about how researchers are attacking this. During the ARC AGI3 preview challenge in mid-2025, researcher Evgeny Rudakov built a training-free graph-based approach.

No AI, just a system that maps every state and action like a flowchart. And it substantially outperformed frontier LLM-based agents on the ARC AGI3 preview, solving a median of 30 out of 52 levels. Not by being smarter, just by having a better memory structure that LLMs currently lack. That tells you the problem isn't model size, it's architecture.

The approaches showing the most promises aren't bigger models. They're time, testing, and training, right? So researcher Alexia Jalicka Martineau built a network with only 7 million parameters, randomly initialized, zero pre-training. They hit 45% on ARC AGI1 by training specifically on the task in front of it, real-time.

GPT-4 has hundreds of billions of parameters. She hit competitive scores with 7 million by adapting on the fly. That's what the solution looks like. It's not scale, it's adaptation.

The defining theme in 2025 ARC prize research was refinement loops. Systems that iteratively improve their own solution using feedback from the environment. Teaching AI to hypothesis tests, like a scientist instead of pattern match like a search engine. That's the research direction that's working.

This is why Charlotte cut his AGI timeline from 10 years to five. Not because models got bigger, because test time adaptation, refinement loops, and separated knowledge from reasoning are getting genuine traction for the first time. Now look at the trajectory. ARC AGI1 launched 2019.

AI scored nearly zero for years. Eventually it hit 87%, but only through brute force techniques, running it thousands of times, picking the best answer. ARC AGI2 launched 2025. Claude Opus hit 37.6% for $2.20 per task.

Gemini 3 DeepThink hit 84.6%, but those scores came from engineering around benchmark, not genuine intelligence. ARC AGI3 just launched. AI less than 1%, humans 100%. Each version resets AI to near zero.

Each version gets cracked faster than the last. Each crack comes from new ideas, not just more compute. That acceleration matters. There's $2 million in prizes available for AIC Prize 2026, open source only.

And the benchmark is now a live API. Anyone can build an agent, plug it in and test it right now. The environment engine runs at 2000 frames per second, so researchers can actually test at scale. And it's built to be attacked fast.

And the breakthrough, when it comes, flows to everyone. ARC AGI3 isn't just a benchmark. It's become the industry standard. All four major AI labs, Anthropic, Google DeepMind, OpenAI and XAI, now officially report ARC AGI performance on their public model cards.

Which means every time a lab makes a meaningful jump on this test, from 1% to 5%, from 5% to 20%, the underlying capability of the tools you use is genuinely improving. Not like marketing improvement, but actually capability improvement. Here's a practical frame. The tools available today are already genuinely useful.

A freelancer using AI moves faster than a team of three people did two years ago. An agency running AI assisted workflows delivers at costs competitors can't match. And an e-commerce brand using AI for email and ads ships at a fraction of all budgets. All of that with AI that scores less than 1% on the hardest intelligence test ever built.

Gartner's 2026 CIO survey found 42% of enterprises plan to deploy AI agents this year. The investment is real. The competition is real. The trajectory is real.

The people who have spent this period building their AI muscle, learning the tools, building the workflows, understanding the actual limits and strengths. These are the people who scale when the walls come down. The ones who waited will be playing catch up to people with a five-year headstart. ARC AGI 3 didn't tell us AI is broken.

It told us exactly where the gap is. With no gaming, no cherry picking, no leaked test sets. Less than 1% when the cheat sheet is removed. That gap is a whole game and it's closing faster than the pessimists expected.

If you want to stay ahead of it, the AI Profit Boardroom is where 2,600 people are building that lead right now. Four weekly calls, daily tutorials, 30-day roadmaps, and 24-7 support because there's always someone online ready to help you. Link in the comments description. The most honest AGI report card in history just dropped.

Start building.