New FREE Hermes Computer Use Agents!

Hermes has a new computer use agent update. So Hermes agent just got superpowers and this free update has made it the most powerful it has ever been because Hermes can now do your computer work for you. Your emails, your notes, your files, your admin, all of it handled in the background whilst you do something else entirely. I tested this out today and it genuinely saved me time within the first five minutes.

One instruction and Hermes had already opened an app, created a note, filled it with 10 personalized ideas and I didn't touch a thing and I'm going to show you all of it in this video. By the end of this you're going to have a free AI agent that handles the repetitive stuff that eats up your day so you can focus on the work that actually helps you grow. And I'll show you the exact setup, the framework I use to get the best results and the one use case that is going to save you the most time starting today. No coding, no technical experience required, anyone can do this and if you want an AI that actually does the work for you instead of just answering your questions, this is it.

Let's get into it. So today we're going to be looking at the new computer use models inside Hermes. Now this is crazy powerful from what I can see. Basically it's a brand new update, it's just been announced a few hours ago and essentially what this means is that Hermes agent can now do stuff on your Mac desktop right.

It can click, it can type, it can scroll, it can drag in the background. That is the most important part, it does it in the background whilst you're not even working on Hermes right. It just goes off and does it right. So your cursor doesn't move, your keyboard focus doesn't change, it doesn't switch spaces on you, it's just you and your agent using the same machine at the same time right as a team which is very crazy stuff.

Now this uses an MCP server to do it and the first thing that we need to do is set up the CLI command which you can see right here. I'm just going to make sure that I have the latest version of Hermes as well so we're going to go to Hermes update inside a terminal. I would recommend you do the same if you haven't already just so you can get the latest updates and skills from Hermes before you set this up and then we'll start working on this. And if you have any questions as you go along feel free to ask.

Now you might be wondering okay what actually is computer games? So imagine you have an agent who can use your computer for you right. They can click, they can type, and they scroll just like a human but they do it in the background without touching your mouse right, without switching windows, without interrupting what you're doing. That is basically Hermes agent computer use.

So it's a feature inside a free open source tool called Hermes agent built by Luce Research and it lets an AI model operate your Mac computer like a person would with any AI model right. It works with any AI model so you could use Claude, you could use Chachi Petit, Gemini or even a free local model running on your machine which is crazy stuff. We've updated now and you might say why does this matter? You know what most people use AI by just copying and pasting back and forth and that's fine but it's slow.

It's the old way to do things right. This is the new way. So it still requires you to do the actual clicking, the searching, the filing etc if you're doing it the old way but with Hermes agent computer use the AI does all of it for you. So it reads your screen, identifies every button, it clicks the right one, types for you, files results, all whilst you're doing something else entirely which is almost scary to be honest with you.

So we're gonna try this now. Bain says is it only on Mac? As far as I can see it is only available on Mac. So yeah and then we're gonna install this CLI command which is Hermes computer use install.

So let's run that now. It's now going to set this up so you can see that was pretty quick so it's already installed which is great. You can also, there's two ways to install this. You can do the dedicated CLI command or you can enable the tool set interactively as well and then after installing no matter which option you took you need to grant OS permissions right which is what you can see right here.

So I'm just going to do that now. So you have to go to accessibility settings and then we have to allow terminal. So let's do this. So we've just got allow these which you can see here.

So I'm just going to click on settings and allow and you can see that basically what I did is I took those instructions here and then just pasted them into Hermes to set that up for me right just to make it easier for me basically. Terrell says hello good to hear. All right so this is configuring computer use as you can see with our Hermes agent. One thing to bear in mind is this is an experimental feature right and obviously you're giving it access to your computer.

So don't do anything that you're not comfortable with. If you're like do you know what I don't really want to give my agents access to my computer you can just watch this video and learn from me right. I'm testing this out. I may uninstall it afterwards because it's a very experimental feature.

I don't really want my agent using my computer all the time but I do want to test this out and see how it goes. So you can see here that Hermes is actually testing this out for us right. So it says use computer use tool to list the apps and then reply with a short confirmation that actually worked which is pretty cool. So Hermes is actually testing this works for us and that's the thing I like about Hermes is it's self-improving it learns itself it improves itself it's so cool in that way.

Now if you want to know some examples of how you could use this here's an example right. So you can give it a prompt like for example find this email about this and summarize what they want me to do blah blah blah and then it'll give you a plan right. Now you can see here Hermes has replied to us and it's like what I checked and changed so verified macOS permissions are granted ensured computer use is enabled verified actually works as well which is really cool. Relevant section includes also verified it with this so you should be good to use this with this one as you can see and you can use it in two ways.

You can use it with this command Hermes t computer use chat or you can actually just use regular Hermes CLI right. So you can use it inside your Hermes you could be on your phone and ask Hermes to just use your computer which is pretty mind-blowing when you think about this. Now they also talk about the best models for this too so you got Anthropic, OpenRouter, OpenAI, Local etc. The only one that it doesn't recommend is text-only models and the reason they don't recommend that is because it doesn't have vision right if it's a text-only model which is a problem because it needs to be able to understand and see screenshots on your computer to be able to do computer use right.

It also has a safety section here so they've added multi-layer guardrails which is good so for example anything where it's I don't know deleting something it requires approval from you right. This is really good because obviously you don't want we also have this open claw which is it could go off and do crazy stuff these guardrails help you avoid that. It's not going to be perfect so just be aware of that but yeah. So let's try this out now.

I'm just going to close notes and I'm going to see if it can open notes up for me and say open up the notes app just to test if this actually works. So let's see what we got here all right so we can see all our apps here it's now going to use computer use so it's using the computer skill and boom it's just opened up notes for us. Wow and this is whilst I'm talking to you whilst I'm operating terminal and checking this out but it can use computer use. Amazing that was so fast and easy.

All right let's test something else out now. So I want to start with the basics and just see how advanced it actually is. So we've tested out creating a new note let's try something else out now. So I'm going to say create open up notes app and create a new note journaling about what you learned from me today or journaling about the best ways you could help me save time day-to-day because obviously Hermes has a lot of context on me.

I'm going to close notes just to make sure it actually opens again and see what it does. I'm going to say open up notes app and create a new note journaling about the best ways you could help me save time day-to-day. Let's see if it can actually do that. So it's brainstorming now coming up with ideas and we I'm using GPT 4.4 for this as an API just so you know it's not even an API I'm just using a codex OAuth login.

So it's opening up notes it's using macOS computer skill you see here it has two different skills right it says Apple notes and has macOS computer use so it's using both of those to operate. Now it's open up we're going to allow that now it's set up whoa ah this worked perfectly. So it's literally just set up a new note and it was so fast I've never seen computer use work that quickly before. So it's created a new note inside our notes app using computer use whilst I was talking to you and it's done it perfectly right.

So it says ways Hermes could save me time day-to-day journal note here are the biggest ways you could save me time inbox content research and this is all personalized to me right it understands for example I talk a lot and do a lot of AI SEO and then it's put that inside the notes here it's set up an idea on personal knowledge capture so it could save useful ideas before they disappear build a lightweight second brain surface or add context when I come back to something later and it's come up with 10 different ways it could help me right and put that into a note which is pretty amazing. I wasn't expecting it to work that smoothly honestly so that's really cool let's try something else out now. So I want to say go into Obsidian app and organize my knowledge base plus add lots of details and context about me plus make it more organized and way more useful as a second brain. So Obsidian if you're not aware it's basically a place where I give all my knowledge on everything I work on but it's still messy right so you can see for example in the graph view it's still a little bit messy it's not that clean to use etc we do have lots of notes and lots of memories on me and what I work on etc as you can see right here it's got a thousand memories on me but it's all inside one note and it's not that nice to use right I think it could be a lot better so I'm going to go inside Hermes and say go into Obsidian app organize my knowledge add lots of details context about me make it more organized way more useful second brain add emojis add titles headlines organize the folders make it super organized and easy to navigate through also improve the knowledge graph and do this with computer use so that's a bit more of an advanced task I would expect it to do it half right at this situation but I've never tested this out it's a brand new feature that just came out so I'm excited to test what it can do and how it works and how powerful it is right and even if it breaks great we learn something but if it does it correctly my mind will be blowing my friend so let's have a look here it's getting its skills so it's pulling in obsidian and computer use here it's reading in the local files as you can see from the obsidian vault and I'm just going to stop that here and say make sure you're using computer use for everything I don't want it to start doing it locally I want it to use computer use just so we can see the power of computer use now I've actually got a custom framework for this stack right here that I've come up with so the way that I would do this is like you give the agent a goal right and you stop giving one-off tasks so instead you give it a full goal for real world outcome so for example it would not just be like write me an email but it'd be like finalize emails summarize what they need draft a reply put it the drafts folder something like that one prompt but multiple actions now you can see here that it's beginning to use computer use and it's actually asking for permissions so I'm going to allow this as you can see and then we'll see what it does in the background here so this is obsidian we've got the terminal over here and I'm going to allow this access so it's good they ask for permissions as well before it does something like this because it's a bit more but it could delete everything for example so now it's prompting and you can see it's struggling with that so I'm going to say fix a computer use back end here and then you want to oversee right you don't want to operate so we've talked about giving it a goal you want to give it a goal hand it to your computer right then you want to oversee which is what we're doing here so you don't want to operate you become the ceo not the operator and you review the work you approve the big decisions but you're not clicking anything else yourself right now I don't think this is going to be like fully amazing as you can see it's still in beta but we're going to allow this to work and then we can stack in any model that we want right so we could use claude we could use free local models we could swap models based on what we prefer and then you want to transform that output into something useful which is like more content more client work more research right more productivity systems and that's basically how this whole stack works this is how the framework works so you can see here it's beginning to use computer use now so you're saying it's free yeah it's like computer use is free to use because you can use a local model right or you can use a free api for example our alpha on open router you could use something like this actually they've just got inside news research on the portal they've just announced that quen3 is free as well for now so you could use a free api with this as well and computer use is free plus hermes agent is free it's a free open source project so now it's going into this is actually hermes using obsidian right so i can see the notes over here and it's actually typing over here right now it's typing in the background so even when i click on for example back to here or back to terminal it's operating obsidian in the background which is pretty crazy stuff and it's actually prompted the agent to use computer use only for this example what would be the best model to run locally it depends on your setup the best way to figure that out is go into lm studio go into this section here if you want to run local models and just see which model works so you can see from me for example it will tell me okay nemetron 3 nano omni is a little bit too it's not going to work that well right so it'll actually tell you what's working and what's not so if we go on the drop down it says likely too large blah blah blah and you can just scroll through the models to see which one you think is the best for you that's the way that we do it christopher says quen 3.6 yeah that's a free model that's available right now with news portal christopher says thanks for the great work happy dub so you can see it's actually using the mouse here this is hermes using my computer right so it's clicked on home and it's got the cursor now the one thing i will say about this is when it's doing something complex like this it does seem a bit slow so there are limitations and i just want to be 100% honest with you right it was amazing opening up notes it was amazing that that with this it seems to be a little bit slower than i would like so just be aware of that too let's stop that task for now and we're going to go back into hermes here and then what i'm going to say here is go into terminal and have a conversation with hermes let's see if it can talk to itself all right you can see the api limit has been hit with hermes i'm just going to switch the model trad says hermes so complex it's simple for day-to-day stuff it's pretty easy to learn but the way that you want to look at it is like we're doing something like computer use right which is a bit more complicated it's a brand new feature it's obviously difficult for people to get their heads around at first so it's just using hermes day-to-day super chill but then if you want to set up something like computer use obviously it requires a bit more technical expertise right it's not something we're all going to get right first time around and that's okay that's part of the process right so i'm going to say open up hermes in terminal using computer use to say hello jonah says thanks for what you do maybe one of these days an interview with a user might be a good idea yeah happy to do that at some point we actually did a lot of coaching sessions inside the aiprofit boardroom so you can see what people are using it for and how it works and that sort of thing so you can see it's now using a computer use i've switched over to kimi k 2.6 because it seemed to use a lot of tokens on codex right codex actually ran out of usage so that's something to be aware of as well it seems to use a lot of usage so it's now navigating as you can see it's clicking the mouse and it's navigating over to terminal which you can see here so the cursor is moving over it's clicked in there so you can see this is hermes agent using the computer it does say if you look at the token efficiency guidelines inside hermes it says the screenshots do use a lot of tokens now there's some layers of optimization as you can see right here but i think that's one of the problems that we found there's also some limitations of course as well so it's mac os only you need good performance etc but it is opening up hermes here so it has managed to open up hermes which is pretty cool so it's done that in the background this is meta watching hermes talk to hermes inside the terminal here and then it's written hello here and so you can see like it actually wrote hello inside the terminal with hermes it managed to open up a new terminal window set up hermes talk to itself and you can see that hermes responded hey julian what are we working on today so hermes talk to hermes here which is crazy this is pretty interesting i'm actually going to stop it there there's some things that i would say about this it's really cool it seems good for basic tasks i wouldn't use it for like massive tasks or super complicated stuff like organizing obsidian it seemed to fall down on that but for like just basic stuff the most useful stuff was like just writing notes and journaling right like i can see right here that was pretty cool so that's the way that i would use it but pretty cool fun to use definitely worth testing out it's a major level up from where hermes computer use was before as well so just to recap on this is basically a new feature inside hermes that allows you to get hermes to operate your computer for you in the background so you don't need to do anything i've talked through the framework and how you can use it how you can set up we actually have 100 prompts inside the guide as you can see right here along with a 30-day roadmap and a step-by-step operating procedure for implementing this stuff right um as you can see 30-day roadmap right here the crazy thing about this is like we didn't really need to be technical to set this up we just paste it in the instructions from hermes guide it can do real tasks like we saw inside notes you can't do super complicated stuff that's just something to be aware of but yeah really cool stuff there so super powerful um let's talk about some limiting beliefs that might be holding people back on this so some people say i can't do real computer tasks but actually hermes computer you can click scroll type and operate any app on your mac in the background and you've seen it live today it's not perfect but it works other people say you need to be a tech expert to use this i'm not a coder i'm not developer or programmer i think if you can type a sentence you can give an agent a task and you just need to install it and once it's installed you're good to go right other people say it will take over my computer and mess everything up right it actually asks you for permissions as you saw today before it does anything right so if it does anything like for example moving files or deleting them it will actually ask you before you go ahead of it and some people say i feel i'm too late everyone else is already ahead of me with ai background computer use is brand new and this release literally just dropped today right most people have never heard of it so you're early and that's your advantage so if you want to get the full guide on hermes computer use you can get it inside the ai profit boardroom link in the comments description or go to the aiprofitboardroom.com this is my automation community that helps you save time grow and scale with automation and agents you can post questions inside the community and actually create videos daily for people who ask questions and cover them in videos like this inside the calendar you can jump on four weekly coaching calls share your screen ask questions about your setup meet other people doing stuff with ai agents inside the map you can actually connect with people using ai in your local city and then inside the classroom you get access to all of my new best trainings right so if you want to learn for example more about hermes computer use we've got the guide with 100 prompts a 30-day roadmap and a step-by-step framework and how to use it right there we also have tons of training on hermes use cases agent workspace how to run it for free with our alpha the desktop app etc so loads of cool stuff on it so feel free to get that link in the comments description or go to the aiprofitboardroom.com julio says you have agents in production yeah yeah i use agents every day for stuff right particularly aiso that's my favorite use case right now is using it for aiso is really powerful for that how to switch models in hermes all right so what you do is you go inside your terminal here make sure you have hermes running like this and then you just type in model and select the model that you want to use so i can switch between models right here let me show you that i'm just going to drag that so you can see i can switch between like all these different models i've set up now if you want to set up a new model inside hermes you just type in hermes setup like this and then you can just switch between the different models you want to set up inside here so that's how you do it