The video answers common questions about building an “agent operating system” where teams of AI agents collaborate using tools like Hermes, Claude, Obsidian, and Kanban boards. It explains how to give an agent long-term ownership of tasks (e.g., ongoing SEO) via scheduled cron jobs managed in one place, and recommends Claude over Hermes for heavier coding work while noting model/API choice affects performance. For quality control and reliable handoffs, it demonstrates adding a dedicated “judge” agent that critiques, scores output, and forces iteration until standards are met. It compares workflows across OpenClaw, Hermes (native Kanban for teams), and Paperclip (multi-tool, company-style orchestration), and argues that adding memory/context through Obsidian improves coordination and creates an auto-updated second brain. It also covers Windows options for an “infinite context engine,” and showcases a custom voice agent example, then directs viewers to the AI Profit Boardroom for the full setup and training.
00:00 Agent OS Overview
01:04 Long Running Tasks
02:56 Coding Model Choices
04:18 Quality Control Judge
07:00 Benchmarking Free vs Paid
07:48 Managing Agent Teams
10:11 Obsidian Memory Vault
11:19 Infinite Context Windows
13:00 Voice Agent Showcase
14:09 Community Setup Wrap Up
Full transcript
So today we're going to be looking at how to build your own agent operating system and some of the best questions I've had about it recently because I know if those people have questions about this it's going to help you too because you probably have similar questions about how to create an amazing agent operating system where your teams of agents can work together building amazing stuff and just creating awesome stuff. I mean for example some amazing things that we've built like for example what we actually found was that when Fable 5 got taken down Fusion just came out right and Fusion is a new way to have like a team of agents working together that can build something better on benchmarks than Fable 5 and I've actually tested this out versus Opus 5 and from what I've seen it actually outperforms Opus 4.8 by a long way and so this is really interesting and what we're going to do today is just answer some of the best questions we have about this stuff how it works etc how to set up Obsidian properly how to get your Hermes agents working with Claude how to build the best agent operating system possible so let's get straight into the questions. So one of the first questions we had from Omar is is it possible for an AI system to take ownership of a single long-term task over an extended period you know so for example like six months like how could you assign it the task of improving a website's SEO performance so you know are there any tools or platforms designed for this type of long-running autonomous workflow so one thing that we have inside our agents is we have the power to build out scheduled tasks so for example if you wanted to give Hermes agent the responsibility of organizing and doing something every week for your SEO website then what you can do is just create a task here or you can actually go direct to the chat and ask Hermes agent or whichever agent you use to do this for you and then you can manage everything in one place if you just go to the cron job section inside manage here and you can do the same with OpenClaw or any other agent so how does this work step by step so you would assign a scheduled task to your agents now I would recommend that you either use Claude or Hermes for this for SEO optimization you would tell it the workflow and tell it every time you do this workflow you improve it and you look at what's working and what's not working and you iterate based on that and you update your workflow and then also run it on a daily schedule so for example it could be daily optimization of MetaTax it could be uploading one new blog per day and that's how you can give your agents responsibility for one task because it's running on a scheduled system and then also you've got the manage section here with scheduled tasks so you can easily see or delete any stuff that you need and to assign a task you just go to the chat with your agent and assign it directly there so Andre has a question here which is Hermes is failing to use Claude code for coding work consistently so when it actually comes to coding work I typically prefer to use Claude code in the first place so for example actually like building out the agent operating system on the back end we'll use Claude desktop because it just tends to do coding workflows better whereas Hermes is great for just doing little tasks or scheduled tasks or recurring tasks now I'd also question which model you're plugging into that too so there's two aspects to this number one I would actually use Claude for coding work instead of Hermes simply because Hermes is great for scheduled tasks or admin tasks or smaller tasks but for coding I tend to find I get better results with Claude so when I'm building out my agent operating system we actually use Claude desktop on the back end and then when it comes to following instructions and skills properly I'd also look at which API you've plugged into Hermes and if it's not doing what you want I would look at using something else instead so for example you could switch to minimax m3 you could switch to grok etc quite often if Hermes is not working that well it can be down to the brain that's plugged into the agent as well so Kyle was posted about his current build for his marketing agency basically he's got a current setup with he's got like Jarvis's dispatcher sub-agents are created from there with a team of advisors and specialists and he's got a knowledge base with Obsidian now too but the hard part he's saying is not can AI do the task you know it's pretty good at that but the problem is getting reliable handoffs clean source rules and enough QA on it as well so how can you build better systems throughout quality control basically so this is actually something that we've been testing out lately to get better results and it's working really nicely so if we go over to the Kanban board here I typically in big tasks like let's say for example you're trying to create a fully edited video with AI and just you know having minimal input but also making sure that everything that you do is set up nicely together what we actually do is we have a judge on the team and so for example we created this separate profile it's called video judge for videos and it looks at the video sees if it's actually good or not and if it's not good enough then it will ask the team to keep iterating until it finally is so you can see it actually scores the team on their output and the quality and it actually outlines the critical issues and everything else so the way that I've been approaching this is building out a team of agents for quality control so you would have for example if you have a video team of agents creating videos then you'd have one agent that's a judge and scores the team on the quality of their work and if it's not good enough they keep iterating until it is good enough and that way you have a separate agent that's only job is to critique review and quality control your team of agents work and with Hermes this can work really well inside the Kanban board too so you can assign a task inside a Kanban board it automatically gets delegated to sub agents and one of those agents should always be a judge who scores the work and tells the team to keep improving until the work is the quality that you need and in the end this looked really nice if we have a look at the final video here which we can preview inside a Hermes workspace so this was the finished product fully edited looks really nice nicely edited and probably better than you know what my team can actually do so the quality of this was just amazing once it finished it looks great it had music in the background as well and again I didn't touch it was just my team working together with a single prompt that I gave them this is quite an interesting one so this is from John and basically what he's done is he's set up like twin agents every time a task is completed inside his agent operating system and basically what he's doing here is he has for every task a free agent and a paid agent and then he compares the difference in quality between them so he can see whether the paid agent is actually good or not and that's a really interesting way of using AI that I've never seen before but it helps you compare everything and just get a benchmark for everything that you're doing versus free models so absolutely love that idea that's a great example of what people sharing inside the AI Profitable Boardroom in terms of like you know how can you check what your agents didn't how can you check the quality of them how can you reduce the cost of everything you're doing as well so this is a good question from James. James is saying he's got an open clause setup with a principal agent and 25 registered agents with their own workspace skills departments etc and he's wondering if there's a better way to manage your agents with proper delegation so for example could you have a Kanban board in a control panel could you move to Hermes should you use a paperclip etc so also should you add more knowledge with Obsidian to avoid duplication and add better context to your agents these are great questions so here's the way that I would look at it if you want teams of agents I don't think open core is particularly good at that simply because it's not really designed for that it's designed for just having one open core instance and working together I do think that Hermes is really optimized for teams of agents because they've created a whole Kanban board and it's kind of native to the Hermes setup so if you want to control your agents and organize them and you're only sticking to one agent Hermes is great for this because you have the Kanban board natively and you don't really get that with open core now I'm sure there are I have seen alternatives but they're just not that smooth when it comes to using open core inside a Kanban board whereas Hermes is is native to this this is what it's designed for it's designed for teams of agents teams of profiles that can all work together on a task as you can see right here when we're creating all sorts of different tasks between our agents and we can have multiple Kanban boards as well for separate projects to keep everything separated too now when it comes to paperclip paperclip is great if you need multiple teams of multiple agents so I would recommend paperclip if for example you need different types of agents working together so if you have open core and then Claude and then Hermes and you want them all inside one team that's where you would use paperclip instead and bear in mind paperclip is designed for really creating like a company a team of agents whereas a Kanban board is not really company focused it doesn't have an org chart it doesn't have a list of issues etc and also paperclip is a lot more visual when you use it bear in mind as well if you're switching from open core to Hermes they actually have a migrate option as well on the onboarding which should probably help you now the final one you know should you add a memory to your agents I would say this absolutely improves everything that you do because when you have a memory and when you have context for your agents not only do these teams of agents for example with Hermes Kanban boards work better but also they can automatically update your second brain so that anything else you do in the future is all linked together as well but you're not updating it you're not manually improving it yourself so for example everything that we've done here like even 18 minutes ago we've got the full daily briefing conversation between myself and Jarvis I didn't create that my agents created that and so when it's automatically updated like this it's super powerful because everything is easy to link together and also you're building something you're building a system that you can come back to over and over again so I definitely recommend using obsidian with your agents too it's great for context you can get your agents to organize and update it for you so you don't have to spend any time organizing it and also it just helps you visualize everything much better and link everything better together and the more you use it every day it improves and gets better and better as a system so definitely recommend that great questions from James there this is a good question from Jason who was asking you know if you want the infinite context engine if you don't know what that is don't worry I'll explain in a second how do you use that in windows so if you're not sure what the infinite context engine is this is a system where I have OMI basically records my screen it records my microphone you don't have to do that but this is just what makes it easier for me and then I export that those notes daily to obsidian so we were talking about our agents updating our context daily so this can take notes on me all day and then what it can do is it can plug that into obsidian memory vault right it syncs automatically so then if we go to obsidian obsidian is updated daily inside our knowledge graph and that plugs into all our AI agents and so it's a really powerful way to get a lot of good context and good memories for your AI agents now if you're on windows the only way that I know the only way that I know to use OMI on windows is if you have a wearable because it doesn't seem to work natively with windows you could have it on your phone instead there's also this that I've set up previously and this kind of does a similar thing to OMI it's an open source project you can see it's got 20 000 stars on github and that's a pretty interesting project as well so that's something else I would look at and it does work on windows so you can see here that it's designed for windows too we actually have a full training on how to use it and again you could take the context from that and then plug it into obsidian instead that might be a better option now this is pretty interesting as well so Benjamin was sharing a voice agent that he's created inside his agent operating system and this kind of a take on our Hermes Jarvis which you can see over here so we can voice activate our AI agents speak to them with a chat and then we can see the whole conversation history here plus everything that's built inside our agent operating system too and that's pretty amazing when you think about it now Benjamin has built out the Logos Oracle and so you can speak to any philosopher that you want right it's trained on philosophy which is pretty amazing itself so you can see here like it's not just for me like anyone who's even non-technical can build with this stuff and you can get really creative that's one of the best things about using for example an agent operating system instead of using like Hermes desktop or these other apps the reason for this is like when you do this it's really powerful because you can customize it exactly how you want it and that doesn't have to be for business right it could be for personal as you can see with Benjamin here so it's a fantastic example of what you can do and how you can customize things exactly how you want it so love the idea love the creativity super impressive so that's basically it for all the questions I've answered everything and all the questions I've had recently about using agent operating systems inside the mission control if you want this whole setup by the way you can get it from the AI Profitable Board link in the comments description or go to the AIprofitableboard.com you get the mission control you get the paperclip setup you get the AI agent mastermind where agents can actually talk together and you can see all the chat history too you got Claude inside there Hermes working you got Hermes Jarvis the studio you have the Kanban board system as you can see we've even plugged in Fusion which is like basically Fable 5 level intelligence but you know you don't have to wait for Fable 5 to come back out right you get something even more powerful so this is something super interesting that we've been building out and testing and it's creating amazing stuff as you can see we also have GLM 512 plugged in there Kimmy Code and the great thing about this is as well we've plugged the memory into the here as well so you can use Obsidian with your AI agents too so if you want to get that that's all inside the AI Profitable Board link in the comments description or go to AIprofitableboard.com this is my AI automation community that helps you scale learn and save time with AI automation we've already helped 3,600 business owners who are active inside there we've got over 182 pages of wins and testimonials from active community members so I know like if we can all get results you can get results too and it's just an amazing community where we learn and we grow together and I really care about the results you know of everyone inside there that's why I answer these questions personally every day now inside the community you can ask questions there's always people online which means you can get help and support I personally answer these questions too inside the classroom you can get access to all of my best trainings and guides including the agent operating system which you can see here and we update this daily you get video tutorial plus you get the installation zip file so you can just quickly install that and then we add new daily trainings based on what's actually useful and I test all this stuff personally inside the calendar you can jump on weekly coaching calls get help and support in real time and inside the map you can actually meet people in your local area who are using AI agents just like you so it's a fantastic way to just meet cool people make lots of friends and also meet people who are doing similar things to you right so feel free to get that link in the comments description or go to the AIprofitborn.com thanks for watching
More episodes