How to Build Your Own Agent Operating System

Today I'm going to show you how to build your own agent operating system like you can see right here. So we've plugged in all of our AI agents and the cool thing is about this, like you don't need to be up to code. It's non-technical. Anyone can do this.

I literally created this and it's evolved into this crazy machine with all my workflows for SEO, video agents, or even for example, generating videos and text-to-speech and images using my favorite AI agents. And so I'm going to show you the process I use step-by-step for creating a whole system like this. So you can take like any project or idea that you have and give it an amazing mission control and that way you can control your agents way better. It's way more fun.

You can see everything that you've created. You can build new things into it and you can basically automate anything that you want. So let me show you an example of this practice. So let's say for example, Open Jarvis.

This looks like a really cool open source project and it reminds me of OpenClaw or Hermes. I know it's pretty popular. I think it's got about 1,600 stars on GitHub. Could be really cool to build something like that into an agent operating system.

So how do you do this? The first thing that I would do is I would take the information from the GitHub like this and that's the cool thing about open source projects is that you can take the amazing structure and the architecture or something like this and build it into something even better. So I'll give you an example of this. This is what we did for Hermes agent.

I love the idea of it, of using Hermes agent. What I don't love is going into my terminal and not really being able to control it or manage it in the same way, right? Like for example, if we're looking through the terminal here, it doesn't look that nice. It doesn't look that interesting.

Isn't that fun to use? Difficult CFN we've created, okay? So what I did is I plugged it into mission control like this and then all of a sudden we've got a command board where I can build teams of agents. It's easier to organize, easier to CFN in different places, easier to navigate and also see what we've built previously, which is pretty cool too.

So how do you do this? Let's say for example, we're starting with Open Jarvis. So you can use Hermes agent or Claude or any sort of AI agent that's running locally to help you build this because the mission control you're building is local as well, right? Now you could also do this on GPS.

As an example of that, you can see like Victor built his own agent OS and John as well. These are all AI Graphical Board members and you can see this one was actually built as a mission control on a GPS. So it doesn't have to be built locally, but you do want to build it in the same place where you have your agent stored. So if we go back into Claude now, you can see here.

I said, do you think we could build something like this into our agent OS? It said sure and say yes, and we're using Opus 4.8. I have previously used Opus 4.7 for this and it's still pretty good. And though it said sure and say yes, it fits into your agent operating system almost perfectly and it filled the gap that none of your other agents cover.

Perfect. And this could be like the 8th agent that we had in. And also this works really well with CLIs, right? So for example, Antigravity, they brought out a CLI recently, which you can use for free.

And then we plugged it into the system here as well. Same sort of process. And so number one, you ask Claude whether you can do it. Claude gives you an idea in terms of the recipe and it explains how we can do this.

So based on our existing structure, we could create like a chat. We could have workspace tabs as well with everything that we create with this. And the other cool thing that's unique about this is it could actually run on local models as well. The other cool thing is when you do all of this, every single agent has its own skills.

So for example, if you look at Hermes, that has 150 skills. If you look at OpenCLoud, that has thousands of skills. And so you can share your skills across the same layer. And not just that, but if you have memory like we have of Obsidian, you can plug that into the system too.

But I'll try to simplify it today to not make this too complicated or overwhelming. So you can see here, it talks about the downsides as well. So some of the friction that will be involved. And then we can say, okay, great.

Yeah, that's set up for me. And so you see how literally I'm just talking to Claude almost like I would talk to a developer or someone working on my team. But the difference is Claude works weekends. Claude works 24-7.

Claude doesn't get tired. Whereas if you had a team trying to build this out for you, they might get confused or they might take a day off or everyone needs a break sometimes. So it's a big difference between using Claude to help you build this out and using a developer. It's way more efficient to do this.

And something that I've found developers might take like a few weeks to do, you can do in a few minutes with Claude. That's the other crazy thing about this. So now what it's doing is it's basically looking through the setup for OpenJavis, kind of looking around, understanding what to build into it. And then if we go back into Claude, it's already installed OpenJavis CLI and now it's doing the rest.

So one thing that I'll say here is that what's really useful if you're building this inside Claude particularly is to have bypass permissions on. So I just said it build out and then I can always roll out. I can roll back changes if there's anything I don't like. It's now using OpenJavis and Olama to set this up.

The other cool thing about this is it would be a free local model that runs with OpenJavis. You could probably plug in an API as well into it if you wanted to, but literally there's very little technical work from you. You just need the idea. And again, the whole point of the agent operating system is that you are an operator.

And the other cool thing about this is every time you use it, you get more ideas, right? You get more understanding of what you could do and how you could build into it. What I will say, and this is something that I've learned as we build this out, is that sometimes you can add too much, right? So you don't need all of this.

Even if you just had one agent and you were building with this, it would still be pretty powerful because it's so much better than a terminal and it's customized exactly how you want it to the point where you could change the colors or you could add more workflows here, right? So for example, let's say I've got this SEO tool, I've got the video tool, I've got the studio, I've got notebook.lm inside there and the Kanban. It's okay. Maybe what I want to build next is a workflow for generating ads inside here as well.

And I could easily build that in. And so it makes you very powerful because you learn and understand how to build out your own tool almost like SaaS tool, but you don't have to be technical anymore to it. And I think 99% of people don't even realize you can do this. The other interesting thing about this is that Plored becomes better and better at doing this.

What I actually like to do inside Plored particularly is have a pinned conversation where I can come back to a conversation that has all the context on my previous conversations about the agent Wes. And so that way you've got this ready-to-go conversation. You don't need to explain anything again. It already picks up where you left off and you can just have it run in the background whilst you're just going off and doing other things.

I'll give you another example. So for example, Step 3.7 Flash just released a new free model with Hermes and we've already plugged that into the agent operating system. So what I would say about this as well, so it doesn't matter what new models come out and it doesn't matter what new agents come out because the system stays the same, right? You're never going to fall behind.

And then anything new that comes out within a few minutes, you can just plug it in. And so you have something that constantly evolves and never falls behind and you have a system that consistently improves every single day. And you can see it's adding it into the mission control agent grid automatically too. And now you can see that we have built this in, right?

So it took about six minutes to get this fully installed. Bear in mind, we just set up Jarvis inside our full local operating system. However, it's not magic. So you can see, for example, in Claude here, it says it's tested out, which is perfect.

So it's done the tests in the backend too. And it says that Jarvis is showing in the side panel. But if we go over here, where is it? It's not there.

So you've got to test this stuff out as well. Don't just rely on the AI to tell you. So what should we do next? Paste in a screenshot, explain the error.

And this is how you can troubleshoot back and forth with it. So now if we come back to Open Jarvis, we can see that it's now inside the mission control and it's ready to chat with right there. The other thing I like about this is when you're doing this, right? Claude remembers how you like everything styled, how to fit it in naturally.

And so you really don't need to think much about it. Claude will do all the hard thinking for you. So for example, if you look at Open Jarvis, the way it's branded, the icons it comes up with, the way that the UI looks, it's all very similar to what we've done previously with everything else. And then you can always add workspace if you want to see your previous work inside Open Jarvis as well.

And so that's basically how it works. Didn't take long. The other crazy thing is not only did it set up Open Jarvis, it actually installed the local models as well, which is great. And then if you wanted to remove it, let's say, for example, you make a change, you don't like it, you want to go back.

No problem. So you can say, okay, delete Open Jarvis and restore the previous version. And then you can just roll back the changes that you've made, which is awesome as well. You might say, why would you use Mission Control instead of just using Claude directly?

The difference is if you're using Claude directly, you can't run custom AI agents. You can't have your agents together in one place, right? And you also can't control the UI inside, for example, Claude desktop. And so it's much better to just have your own custom agent OS system.

And so the way that I look at this, I call it the loop method. And basically it's a simple framework, right? So it's a loop that never ends because this agent operating system just gets better every single day. And here are the four simple steps to do this.

So number one, you find a cool agent that you like, right? Maybe it's on GitHub. Maybe it's in your free model that just dropped. You can grab some information about it and then you can plug it into Claude.

After that, you're going to open up Claude and ask and just talk to Claude like it's a member of your team. So you can ask, okay, can we build this into my agent OS? Claude tells you yes or no. Then it gives you the recipe and even tells you the downsides first.

After that, you order it to build. So you can say, great, it's set up for me. Turn on bypass permissions so it can work freely. Let it build.

Go grab a coffee. You can do something else in the background. And like I said, something a developer might take a few weeks to do, you can do in a few minutes. And after that, you just want to prove it actually works, right?

Don't just trust the agent. Claude might say it's done, but then you look, it's not actually there. So go and check it yourself. And also if it's broken, paste screenshot into Claude and troubleshoot it, right?

And then you can look back. And you can do the same thing with Hermes. You can give Hermes a screenshot. You can have Hermes build locally for you.

It's the same sort of process. Claude is a lot faster and smoother with this sort of stuff, but the same process works. And now you can see as well that OpenJava is removed from the AgentOS system. Now, again, some people are going to say I need to know how to code to do this.

You just need an idea, right? You talk to Claude like a person. He does the coding for you. To 9% of people don't even know it's possible.

Other people say this takes weeks and teams of developers. Actually, it takes minutes. As you can see, my AgentOS installed a new tool in about six minutes. So you don't need a theme.

You don't need to wait around. You can just get it done. Other people say new AI comes out all the time. So I'll fall behind.

But the difference is your system never falls behind because the system stays the same. And when new agents come out, you just plug them in. So thanks for watching. Full step-by-step roadmap and a three-day plan to implement it into your business.

As well as a video tutorial on how to use it. We've got the full guide here as well. And this is just an amazing community where you can ask questions, get help and support in real time. You can get access to all my new daily trainings in the AgentOS system.

You can jump on weekly coaching calls about this sort of stuff. Connect with me and ask me any questions. And also connect with people in your local city from the map as well. So feel free to get it.

Link in the comments description or go to the AIProfitBallroom.com. Thanks for watching.