Full transcript
Today, I'm going to show you a powerful system that you can use to basically loop around all your AI agents so they can do all the quality control for you. So if you're tired of prompting your agents, if you want your agents to run automatically without you, if you want systems where you don't really have to do any prompting anymore, you can just get your agents to loop around and get your work done, quality checked and avoid AI slop, then this system is very, very powerful. I call it the self checking factory. It's basically a factory line where every AI agent inspects the last station's work, one agent builds, the second agent grades it, you remove any quality control issues, you remove the chances of AI slop, and your whole system gets better and better.
And this really comes down to one problem that a lot of people have. I mean, imagine, for example, you're in a bakery where nobody tastes anything. The bread goes straight from the oven to the customer. Some loaves are perfect, some are totally raw in the middle.
And the customer finds out, not the baker. That would be no bueno, right? That would be quite disappointing for the customer. But that's basically how most people run in AI agents.
So they have agents that write the code, the article, the page, hand it straight to you with no checks in between. And so you become the inspector, that becomes a very tiring, frustrating job. And you read every output, you test every build, you catch every broken button, and you wanted a team of agents, but you've got a pile of homework to mark instead. And the moment you stop checking, slop gets shit.
So the self-checking AI factory that I'm showing you today basically breaks that cycle for good. And you might say, well, this sounds complicated. I'm not a coder. Basically, all the system is very simple.
You define what good looks like. So you define the standard. Then you have one agent that implements it and you get another agent to judge it. And then you can set how many rounds it goes round for.
And I'll explain more of that in a setup. And the other good thing about this is once it's looped round and it's completed, the great thing about this agent loop is that everything that you build is saved inside the workspace so you can come back to it whenever you need to. Now, you might be wondering, okay, how does this loop actually work? So basically this is like having two brains working together.
And the whole idea here is that you have an agent that builds a work and you have an agent that grades it, but they're never the same model. So this loops round because you set up the goal in one sentence, the builder makes a thing, the judge scores it out of a hundred, and it just loops round and round until that work is finally done, until it finally gets shipped. And you might wonder, okay, why would you do something like this? Because AI is a terrible judge of its own work.
In the same way that you can't proofread your own writing, it reads what it meant to write, not what it actually write. And so the problem with that is there's no quality control. Normally, if you're just going back and forth inside ChatGPT, and this is why people find like ChatGPT frustrating to use or quality frustrating to use because there's no quality control involved. And so the quality of the outputs is way lower.
And the whole point of using AI is to have like the best intelligence, not the worst. And so the loop only ends one way. Either the work passes or it just loops around until it's finally done. And you can set how many loops actually goes for.
So you can set the maximum number of rounds. So you can say, okay, loop round 20 times until the work is done. And if the work passes the grade on the first time, no problem at all. But if it doesn't hit the quality score, then it keeps looping round and round and round.
You might say, well, what if the judge is wrong? But the judge doesn't need to be perfect. It just needs to be separate, right? So a separate grader with your goal in hand catches the obvious slot.
And that could be, for example, like broken buttons, badly written headlines, missing sections inside your page, ignored instructions, which is usually what, you know, is waste 90% of your time when you're using AI. Like I know a lot of people use Claude and they're just going back and forth with it, babysitting it all day. You don't want to be like that, right? You want to self-check in factory, it can automate and improve itself.
And so in terms of the actual work and how it actually runs, you literally just type in what your definition of done is, right? So you would say, okay, a beautiful website that has 50 blog posts, perfectly SEO optimized for my target niche. And then you would have a judge that actually judges the work. Now we can select which model we want.
So we could go with a pre-model for actually doing the work because it's just building and building usually doesn't require a frontier model, which saves you money in tokens. And then we have over here a system where we can set up the frontier model. So we can have a judge that actually judges the quality of the model's work. And again, you want a separate model for that.
You want like a frontier model for that normally. But if you don't want to use frontier models, you can use something cheap, like for example, GLM pipeline 2 on a coding plan, and that'll just keep going round. Now, if you're wondering, okay, how does that look in practice? So here's an example loop that we ran and you would have round one that loops around and it scores like eight out of a hundred.
Not very good. Well, on round two, it scores 85 and then round three, it passes the bar because it scores a hundred, which is perfect. So your agents get better and better, but you don't have to touch it yourself. And so it's just, it's a way of looping.
And the thing that I would say here is like, you are the customer, right? So you're the one using the AI and the customer should never be the quality control department, but actually with AI, you usually are, unless you have a system like this and that wastes so much of your time. So the way that we want this as well is a system where we have a factory. So it runs on like a factory line and it just improves over and over and over again.
Now there's five stations of the self-checking factory. In every version of this, we've got three different systems inside our agent operating system is, you know, designed as a loop. So we have the loop engineering system over here. We also have a Kanban board, which is a similar sort of system.
But the difference between the loop engineering system and this is that when we're using this one, basically we have teams of agents, so not just one agent building and one agent judging, we have a team of agents. So for example, if we're creating a video like this one, the reason this looks so good is because we had a video director, we had a video builder, we had a content writer, and then we actually have the content judge itself and we can see the verdict over here. So with a Kanban board, you can actually have a team of agents. You just drop in the task.
It gets broken down and triaged into separate sub-agents and sub-tasks, and then you go from there. So that is method number two. And you know, you can build these yourself, but if you want to get my systems from me, you can get them inside the AI Profit Boardroom. Link in the comments and description or just go to the AIProfitBoardroom.com.
And the final system they actually have for this is an agent team. So if we have a look, for example, we can actually go to the agent mastermind of here, this is a way that we can have a group chat between our agents, right? And the great thing about this is when we're building out with this, our agents just chat with each other and they quality check their own work. So I say, come up with the idea.
Like for example, Hermes, come up with an idea for an app. And then we have each of our agents working together, bouncing ideas off each other, and then when something actually gets built, like for example, this, it gets saved and plugged into our ideas pipeline where we can basically build apps in real time. And that's another example of the loop because literally you just set the goal, which is come up with the idea, and then it gets looped around in the group chat and then plugged into our ideas factory. And from there we can just implement and build the idea.
And this way we can create loads of things. And you know, you can go from idea to implementation very quickly. Like you see right here, all these mini apps that we've built. We can open up on full screen.
We can see what we've built here. Looks pretty cool. And it was easy to do. So this is the sort of stuff you can do with this.
So you have a builder and you have an inspector, and then you have the loop return belt inside the factory, right? So if something is not very good or it's not the quality, it's just going to keep going round and round and round. That's what we like. And then once it's finally done, we have the ship gate.
So what we have over here, for example, is the loop section here. And we can see that this is part of the factory that stores all the builds. And that's a build workspace. So we can come back for it later.
And I will say, if you don't have a safe workspace where everything is set up. So for example, inside our agent OS, every single time we create something, it goes inside a workspace. The reason that we do stuff like that is because we want to save this stuff for later. We want to come back for it for later.
Because otherwise what's going to happen is you build stuff and it just gets lost forever and you can never find it. So that's basically the whole system right here. And, you know, some people say as well, when you're building with this, why do you use free models? I use free models because it's literally quite often these tasks, if they've got someone to grade and help them plan, then you can use free models and the quality of outputs will be really good.
But you could use, for example, a cheap workhorse, for example, like GLM 5.2. And then you can use GPT 5.6. You decide what you want. You decide how you want to do it.
And you have full control over which API you want to use inside the system, which is fantastic. And also some people say, well, like AI, it always creates a lot. And it only creates a lot because you're not quality controlling it. If you take charge and you build a system like this, you go from the babysitter of AI to the architect, the engineer.
And you want to be the engineer that has great systems, not worrying about which model you're using and all that sort of thing. It saves you a lot of time. So when you're using the Kanban board, which is the other system we talked about for loop engineering over here, when using Kanban board, you have a planner and that ships and gives tasks to different agents inside the pipeline. And then we have a reviewer, aka the content judge.
And if you have a look inside here, for example, when we're using the content judge system, this is a separate agent profile with Hermes agent. What this can do is grade the homework. So you can see here, it has a QA verdict and it will say, okay, this is a pass. So this is a fail.
Here's why, here's what I like about it, here's why I don't like it. And you see how it's really detailed or broken down and it explains exactly why that stuff is good or not. That's what we want. And then we actually have the HTML for the project and the verdict on it.
And that's why our outputs here look really nice, right? Because we can easily build stuff that's been quality controlled without me having to sit there all day and be like, you know, going back and forth on the keyboard. And also this saves a lot of tokens because it's not you reprompting it and getting frustrated and writing really long feedback modules, you just get an agent to let it do its job and then a judge to check it later. You might also say, well, this worked for my business.
You know, I'm not a coder, blah, blah, blah. Well, the great thing about this whole setup is like, you know, you can, it works for anything, right? The cards and the Kanban board system, it doesn't care what's on them. You could have web tools, you could have SEO articles, you can have landing pages all in the same week on the same system.
So if your work can be described in a sentence and checked against that sentence, it can ride this board, right? That's the whole point of this. And if you have a look at the old way, the old way is that you are the quality control department. So there might be like three to four hours of checking every day, prompting the AI, spotting problems, explaining the fix, waiting again, we've all been through that cycle, repeating the conversation four or five times per task, testing every button and link in every sentence, um, losing finished work in an endless chat scroll.
And if you stop checking the work, then you just get AI slot. Whereas with this system, you, it takes you like two minutes to set the goal. Then you have a judge, you write one clear goal and the agents just do everything for you, right? And the good thing about this is you don't have to reprompt.
You don't have to give feedback. It's all just running autonomously without you. So it's a really powerful system and this is what I recommend. And there's three systems that I've shown you today.
You're the agent group chat where they all bounce ideas off each other and you just give them the idea that they need to work on. You've got the loop engineering system inside the agent OS and that just basically sets a goal and then loops around with a judge that checks if the work's good or not. And then finally we have the Kanban board as well. And the Kanban board is great for like multi-agent workflows as well.
You also might say, okay, I check faster than any AI judge. Why bother? You check better and faster on one single task. But with this system, you can have agents running 24 sevens.
Even whilst you're sleeping, you can make sure the work is quality controlled. So if you want to get the self-checking factory built for you, you know, you could wire this and build just yourself or you can get the whole thing done for you inside the agent operating system, inside the AI performance. You get the loop, the judge, the Kanban board already connected. You get the full agent OS system with the loop engineering and agent Kanban system, a free judge set up, coaching calls where you can set up the factory with other people, a room of 3,900 operators running this exact stack.
And the prompts, CSOPs and member map to meet you inside here. So if you go inside the community, you can ask questions, get help and support. I personally create a video tutorial for everyone inside there every day. Inside the calendar, you can jump on four-weekly coaching calls, get help and support in real time.
And then inside the classroom, you can actually get all of our best training. So if you're a complete beginner, you can go from beginner to expert over here. If you want our agent OS system that I've shown you today, you can get the video tutorial, you can see when it was last updated, you get a guide for setting up and the resources, we also have a full one hour course on how to build an agent OS and how to get the most out of it. We've also got other courses inside here and token minimization playbooks and all sorts of cool stuff, as you can see.
You can also meet people in your local area who are building with AI agents like you. And that's all available inside the AI Profit Wardroom. Link in the comments description or go to the AIProfitWardroom.com. Thanks for watching.
More episodes