01What Is Hermes Jarvis and How It Works
Hermes Jarvis is a real-time voice-activated version of the Hermes AI agent. It listens to your voice, responds out loud, and can open apps and control your computer on command.
In this episode, Julian Goldie showed Hermes Jarvis opening Google and Obsidian, telling jokes, switching to Swahili, and even reading a website link he sent in the chat. The agent works in the middle, connecting your voice to the Hermes open source project.
It uses memories stored inside an Obsidian vault. That means it knows the date, what you have open, and what you worked on recently. It also keeps a list of your daily briefings and your conversation history.
The key point is that Hermes Jarvis is a genuine tool, not a gimmick. It responds literally instantly and opens stuff up instantly. Julian said it is the first time he has been impressed by this sort of voice agent technology.
02Main Features of the Voice Agent System
Hermes Jarvis comes with several features that make it useful for daily work. Real-time voice responses are the biggest one. It talks back to you like a person, and you can interrupt it mid-sentence.
- Wake word: You can flip it on and speak without touching anything.
- Arc reactor: A button you tap to talk whenever you want.
- Fast answers and daily briefings: It reads your day or week to you.
- Weekly briefings: You can switch to a weekly summary of your tasks and notes.
- Voice switching: You can change between all the available voices.
- Command box: You can type instead of talking when you prefer.
- Wall mode: A separate monitor that is voice-activated and always ready.
- Link reading: You can send a link in the chat, and Jarvis reads it and replies.
- Live preview: It can build things for you while you watch.
It also stores briefing history. You can click on previous briefings from the last few weeks and see everything you built with Hermes in voice mode.
03How to Set Up Hermes Jarvis Like Julian
Julian showed how he built his own Hermes Jarvis system step by step. Here is the process he followed.
- Start with the Hermes agent open source project as your base.
- Add a voice layer so you can talk to it and it can talk back.
- Use ChatGPT real-time as the API for fast voice responses.
- Connect your Obsidian vault so the agent can read your memories and notes.
- Set up a wake word so you can activate it hands-free.
- Iterate for weeks with Claude and Claude Desktop, going back and forth to improve small details.
- Test daily briefings and weekly briefings until they work smoothly.
- Add features like link reading, wall mode, and conversation history.
Julian said he spent about four hours a day improving the AgentOS system. This morning was the Hermes Jarvis version. He also mentioned that you do not have to use Hermes.
You can set this up with Claude, OpenClaude, or Gemini instead.
04How to Use Hermes Jarvis in Daily Work
Using Hermes Jarvis is simple. You just talk to it. In the episode, Julian gave it several commands to show how it works.
He said "Open up... Google" and it opened Google right away. Then he asked for a joke, and it told one.
He asked for a funnier joke, and it gave another. He switched the language to Chinese and then to Swahili, and the agent followed along.
He also said "Open up Obsidian" and it opened the app. It gave him a good afternoon briefing with nine open items and nine notes in motion. It listed his top task, which was a Vault LLM wiki restructure.
For numbers, he said "Show me the numbers" and the agent displayed them on the screen. You can also say "Show me" and it pulls up whatever you ask for. You can send a link in the chat and Jarvis will read it and give you feedback.
The agent can even build things. Julian mentioned it built a meditation app inside voice mode. You can interrupt it mid-flow to stop or change what it is doing.
05Results and Comparisons From the Episode
Julian compared his current setup with the one he had before. The results were clear. The old setup used 11 Labs for voice, and it sounded better but was too slow to actually use.
He switched to ChatGPT real-time as the API. That made the responses fast enough for real conversations. Hermes Jarvis responds literally instantly and opens apps literally instantly.
He also compared Hermes Jarvis to the old chat-only version of Hermes. He said there used to be a talk section, but now Hermes Jarvis totally replaces it. When he has a choice between going to Hermes Jarvis or going inside the chat, he goes directly to Hermes Jarvis.
The reason is simple: it is easier to use, easier to manage, and easier to organize. He said the agent is getting closer and closer to something he genuinely loves. He called it fantastic and said it is the first time he has been impressed by this type of voice agent.
06Tips, Limits, and Mistakes to Avoid
Julian shared a few honest tips and warnings from his experience building Hermes Jarvis. The biggest mistake he mentioned was using a slow voice API. He warned that 11 Labs sounded better but was too slow to use. Speed matters more than sound quality for real conversations.
- Use ChatGPT real-time as the API, not 11 Labs, for fast responses.
- Build in small iterations. Julian improved the system a little bit each day for weeks.
- Do not expect to build it all in one sitting. It takes many rounds of back and forth with Claude.
- Be transparent about limits. Julian said Hermes Jarvis does not have real-time access to Obsidian yet.
- Memory improvement is still needed as you go along.
One limit is that the agent cannot read your Obsidian vault in real time. Julian is working on adding that. Another limit is that remembering things is something that needs improvement over time.
He also noted that the chat section might be removed since Hermes Jarvis replaces it.
07Who Should Build a Voice Agent Like This
This episode is for anyone who has been thinking about building voice agents. Julian said now is the time. It is better than ever, and it is the first time he has been genuinely impressed.
If you want to build something useful with Hermes, this shows you that it is possible. It is not just a gimmick. It is a genuinely useful voice agent that can do real daily work.
You can set it up with any model. It does not have to be Hermes. You can speak to Claude, OpenClaude, or Gemini.
That makes this approach flexible for different builders.
Julian also showed where to get his full setup. It is inside the iProfit Boardroom at theiprofitboardroom.com. That includes the Hermes agent setup, Hermes Jarvis, the agentic OS, and access to his trainings, community, and tutorials.
08Why Voice Agents Matter Now
Voice agents are becoming practical for real work. Hermes Jarvis shows that you can talk to your computer like you talk to a person. You can open apps, get briefings, read links, and build things with your voice.
The key is speed. When the voice responds instantly, it feels natural. You can interrupt it mid-flow, and it handles it seamlessly.
That makes it easier to manage your day without typing.
Julian said the agent knows the date, knows what is open, and knows what you worked on recently. All of that comes from memories inside the Obsidian vault. That kind of personal context is what makes it useful.
He also said you can watch his live streams to see how he improves the system. He spends about four hours a day on it. That dedication is what turned a basic voice agent into something he loves.
If you want to build one yourself, the main takeaway is simple: start iterating. Use the right API, connect your notes, and improve a little bit every day.
