AI News Today
← All episodes
Episode 31 · June 12, 2026 · 08:52

Automate Anything with Hermes Jarvis, here’s how!



I Built a Voice-Activated AI Assistant That Remembers Everything (Jarvis + Hermes Agent + ElevenLabs)

The video demonstrates a voice-activated AI assistant called Jarvis (built from Hermes Agent) that can open websites, pull memories from an Obsidian vault, create notes, give spoken day summaries, and build projects on command, including generating a working Snake game with live previews and a gallery of everything created. The creator shows features like wake-word listening, conversation history, wall mode, diagnostics such as “show me my numbers,” and viewing a team of agents. The system is powered by Claude Desktop in the background and uses ElevenLabs for voice, with ongoing improvements and multilingual/joke capabilities. The episode explains why housing everything in one dashboard is more convenient than juggling separate tools, mentions optional phone-number access with security concerns, and invites viewers to join the AI Profit Boardroom for the full Agent OS system, trainings, updates, coaching calls, and community.

00:00 Jarvis Roasts Me
00:13 Grand Intro Request
00:48 What I Built This Week
01:12 Agent OS Demo Tour
01:57 Voice Built Snake Game
02:29 Conversation History Previews
03:28 How It Works Under Hood
04:33 Show Me Gallery Numbers
06:03 Why One Dashboard Wins
07:03 Phone Number Access Risks
07:37 Join AI Profit Boardroom
08:48 Final Wrap Up

Full transcript

Jarvis, roast me before we get started. Very well, sir. Shall I open Google to research whether the miracle of life truly began with your arrival, or are you content to take that on faith? Jarvis, create a grand introduction for me to my audience.

Of course, sir. Here is a suitably grand introduction you may tailor to your taste. Ladies and gentlemen, esteemed guests, and distinguished colleagues, it is my distinct honour to present to you a true visionary, a relentless innovator, and a master of many crafts. A person whose intellect is matched only by their unwavering dedication, and whose presence commands the attention of any room they enter.

Hermes, be honest, what have I actually been working on this week? I believe you meant Jarvis, sir, but I'll forgive the lapse. Based on your vault, you've been working on your thumbnails log, which suggests your focus of late has been on designing, reviewing, or cataloguing thumbnails, likely for video or content production. Shall I pull up the latest entries for you?

Jarvis, open up juliangoldie.com. Opening it now, sir. Alright, so what we're seeing here is basically Hermes agent transformed into an AI, voice-activated AI agent that can open things up, that can build things for you. So, for example, if I ask it to check my obsidian log, which we did earlier, it can basically pull memories, which you can see right here from my obsidian.

So it has access to all the memories in my vault. If I ask it to build something, it will actually build something on the spot. For example, check this out, and we can have a wake word here, which means I don't even need to click it. I can just have it listening in the background.

So let's try this out. Jarvis, build out a snake game. Built and running, sir. Have a look.

And you can see we actually have the game here ready to play, and it's working perfectly, right? We can also see a preview when we build stuff out, which is pretty cool. For example, we go back to that tab, and I just want to show you a snake game as an example, because obviously if it's building something, it takes a little while to reply. It's not just like instantly, boom, there you have a snake game.

It's not like that, but it just gives you an idea of what you can do with this. You can also talk to it here, and then you've got this whole conversation and all of the conversation history between myself and Jarvis. Now, does it always reply perfectly first time round? No, absolutely not.

But is it powerful? Is it useful? Can it actually build stuff? Absolutely.

So, for example, here, you can actually see if we use the phrase, build me, then it was building the snake game. Indeed, rough edges and all, it's still remarkably useful. And then it's getting carried away. So if we come back here, you can see all of the stuff that we've actually built with it as a live preview, which is pretty amazing.

And so when you're doing this as well, you can see all the stuff you've built like that. And then you can also go full screen warm mode as well and have a conversation with it. Now, additionally, as you saw before, it can open up websites for us. It can create notes inside our notes app or whatever we want it to do.

And it's just kind of like a trained, witty assistant that replies to us in kind of like a voice. Now, if you're wondering, how did we build this? Essentially, what I actually have is I have Claude desktop, and that builds for me in the background. Now, this uses a combination of 11 labs and Hermes agent to build this out.

Now, the good thing about that is number one, I can always improve it because it's an operating system, it's ready to go. And there's so much more built into this, but this is an example. And then number two is that it can give me ideas on how I can use it and how I can improve it as well. So even, for example, when I was coming up with ideas for this particular video and what I could showcase, it actually came up with a bunch of stuff that we could include and mention inside the video itself.

And you can see here, like, it will show stuff up. So like I mentioned before, we can say, open things and open them. We can say, catch me up, and it will actually do a spoken summary of the day based on our obsidian vault. We can link it to our second brain.

So it can essentially look at our obsidian vault and then give us ideas. It can build by voice. And we can also say, show me, and then it pulls in the gallery. Let me show you an example of this.

Show me everything I've built. Here's everything we've built, sir. And you see how it's got all of this stuff plugged in directly. And we can just open up and we can play it.

We can see everything that we've built. We can see the preview, et cetera. That's pretty crazy, right? So we clicked on this to preview it.

And look, we've got that ready to go, which is pretty amazing in itself. And so anything that we've built with this, we can plug it into the system and say, hey, can we build this? Additionally, it knows you. So it remembers everything about you.

And then we can also say, show me, and not just show me what I've built, but show me my numbers or show me my team or run a full diagnostic. So let me show you an example of this. Show me my numbers. Pulling up the numbers, sir.

And then the other one is show me my team. So what that means essentially is we can see all our agents. Show me my team. Here's the team, sir.

And you can see your full team as well and what you set up inside the agent operating system too. Now, this is getting better and better. Like, you know, we're improving it all the time. It's really becoming something super powerful.

I mean, there's a lot of functionality to this. So for example, you've got the war mode. You've got the previews. You can open stuff up.

It can build stuff for you. You can pull in stuff that you've built. It can look at your obsidian vault. Like, there's a lot that we're plugging into this and just trying to make it the most useful voice agent we possibly can.

It can also crack jokes. It can speak multiple languages. It's pretty amazing in itself. So if you're thinking about building something like this, I definitely recommend it.

One of the things that I found most useful is that I just have one place where I can come back to it anytime I want. Now, if you've ever built a voice agent, you know that feeling of like, oh, I have to open up 11 labs and then I have to log in again and then I have to do blah, blah, blah. And that's super annoying. Whereas if you have it inside one saved place, you can come back to it.

You can bookmark it. You can have it inside the dashboard and you can switch between things. So if I just need to chat, I've got that here. If I need a conversation, we've got it here.

And if we've got Hermes Jarvis, we have that ready to go here. And so what I like about this is it's just very easy to navigate to and we can come back to it at any time. And then also it connects to a full system, right? This is the difference.

So again, you've got the obsidian. We've got like different things that we've built with apps and ideas over here. We have Claude. We have Hermes.

Everything just interlinks. And that, again, makes it much more powerful when you're building out something like this. So I would say this is the best way to use a voice agent. I've not really seen anyone else doing it like this, but I think it's a great way to do it.

And then also the other thing I would say here is if you're not using 11 Labs, one of the other things to note is that you can give it a phone number. So you can attach your phone number and then call your agent whenever you want to, when you're out and about. I don't recommend that because if someone else gets access to that number, well, you know, like they could control your agent. So that's why I actually did it.

I had it set up and then I actually removed it because I just don't think it's the right way to use a voice agent, but it's a pretty cool idea. Maybe you could set up some features where you say, okay, you know, you can only retrieve stuff and you can only do certain things that you limit to it. So that's basically it. Now, if you want the full system with the idea factory, with the SEO content, with the video agent, with the game studio, with the, you know, the image generator, notebook LM plugged in here, a Kanban board where you can orchestrate teams, paperclip built in so you can have a whole team of AI agents working like this and agent group chat with all of your agents ready to go inside one place.

You can get that inside the AI Profitable Boardroom. Link in the comments description or go to the AIprofitableboard.com. This is my AI community where you'll learn how to grow skill with AI automation. And we've got an awesome community where it's very active and lots of cool people posting awesome stuff.

Inside the classroom, you get access to all my new trainings, including the agent OS system. We've also got a full section on AI voice agents, how to set this stuff up, Claude 5, the Hermes idea factory, paperclip and agent OS, just loads of cool stuff right here. But we actually update this system daily with new updates, the agent OS, and then you can get new updates and new cool stuff to it. Additionally, inside the calendar, you can jump on coaching calls.

We can build this stuff together. You can ask questions, et cetera, in real time. Inside the map, you can meet people in your local area. And this is all available in the AI Profitable Boardroom.

Link in the comments description or go to the AIprofitableboard.com to get access. That's how to use it. Thanks for watching. Cheers, bye-bye.

More episodes

Browse all episodes →