01What Hermes Computer Use Is and How It Works

Hermes computer use is a new free open-source feature from Nous Research that lets your Hermes AI agent control your Mac for you. The agent can click, type, scroll, and operate any app on your computer — all in the background while you keep using your machine for other tasks.

Instead of taking over your mouse and keyboard, the agent sees your screen through screenshots, decides what action to take, and then executes those actions in an invisible layer. You don't have to switch windows or wait for it to finish. You and the agent work in parallel on the same computer.

The update just dropped, and it's completely free. If you already use Hermes agent locally, you can add computer use with just a few commands.

02Main Features of Hermes Computer Use

This is not just another AI demo. Hermes computer use brings several powerful capabilities:

  • Background operation — The agent works without taking over your cursor or keyboard. You can keep working while it does tasks in the background.
  • App control — It can open apps, create notes, clean up folders, and more. In the episode, Julian showed it opening the Notes app, creating formatted notes about SEO workflows, and even opening an app it had recently created.
  • Phone control — You can set up Hermes locally and then control your computer from your phone. The AI agent keeps working even when you are not in the room.
  • Guardrails — The tool has built-in approval prompts for destructive actions like deleting files or sending emails. You see what the agent intends to do before it does it, and you can deny it.
  • Free and open source — No subscription fees. You run it on your own machine.

03How to Set Up Hermes Computer Use in 5 Minutes

Setting up Hermes computer use is simple. You only need to be able to copy and paste commands. Here are the steps shown in the episode:

  1. Update your existing Hermes agent installation to the latest version.
  2. Open a terminal on your Mac.
  3. Run the command: Hermes computer use install (or paste the install URL from the CUA driver page).
  4. Grant the accessibility permissions that the system asks for. Hermes can guide you through this.
  5. Test the setup by giving the agent a simple task, like creating a note in the Notes app.

That is it. The whole process takes about one minute, not hours. There is no need for a dedicated CLI — you can enable it interactively.

The feature currently works on Mac, and it may work on Windows too, though Julian has not tested it on Windows.

04How to Use Hermes Computer Use: Walkthrough Examples

Once you have Hermes computer use installed, you can give the agent goals and watch it work. In the episode, Julian showed several real examples:

  • Create a formatted note — He asked the agent to create a beautifully formatted note in the Notes app about SEO workflows. The agent wrote a full journal entry and saved it. You could see the finished note immediately after.
  • Clean up your desktop — He said clean up my desktop using computer use so it is neatly organized. The agent started organizing files in the background.
  • Open an app and type — He asked the agent to open up an app you have recently created. It opened the Hermes agent app. Then he added type hello, and the agent typed it.
  • Journal a note — He said create a note explaining the best ways to use computer use with Hermes agent. The agent wrote two detailed notes and opened them in text.

You can give the agent a broad goal, not just a single task. It will work on that goal in the background, and you can oversee the results.

05Tests and Comparisons: Hermes vs NSN

In the episode, Julian compared Hermes computer use with NSN (a more technical automation tool). He pointed out several differences:

  • Setup speed — Hermes computer use takes about one minute to install. NSN can take three to four hours to set up similar functionality manually.
  • Technical skill required — Hermes is no-code. You just copy and paste commands. NSN requires technical knowledge and is fiddly.
  • Reliability — With Hermes, you can get results in seconds. Julian showed a note created in eight seconds. NSN can break easily.
  • Cost — Hermes is free and open source. NSN in the cloud charges a subscription.

The episode also mentioned that older computer use agents were less practical because they took over your mouse and keyboard. You had to wait for them. Hermes computer use solves that by working in the background.

Julian noted that the agent is not yet perfect for hour-long projects, but it is already useful for background tasks.

06Tips, Limits, and Mistakes to Avoid

When using Hermes computer use, keep these tips and warnings in mind:

  • Guardrails exist — The agent shows you what it wants to do before doing destructive actions like deleting files or sending emails. Always review these prompts and deny anything you are not comfortable with.
  • It is not perfect — The limits are real. Do not expect the agent to handle multi-hour projects without supervision. Start with small background tasks.
  • You do not need to be a tech expert — If you can copy and paste a terminal command, you can install this. Many people hesitate because they think it is too technical, but it is not.
  • You are not late — This release just dropped. Most people have not heard of it yet. You have a head start.
  • You can stop the agent mid-task — If something goes wrong, you can interrupt the agent at any time.
  • Run locally for privacy — Julian recommends running Hermes locally rather than on a VPS to avoid privacy problems.

Julian also mentioned that both Hermes and Claude are the smoothest agents he has used for this type of work.

07Who Hermes Computer Use Is For and Why It Matters

Hermes computer use is for anyone who wants their AI agent to do real work on their computer without waiting for it. It is especially useful for:

  • People who are not technical — You do not need coding skills. The no-code setup lets anyone get started.
  • Busy professionals — You can give the agent a goal, walk away, and come back to results. Julian compared this to being the CEO while the agent becomes an app.
  • Anyone using Hermes agent — If you already run Hermes locally, adding computer use is a natural next step.
  • Users who want to control their computer from their phone — The agent works even when you are not in the room.

This matters because it changes the model of how we work with AI. Instead of babysitting the agent, you can let it run in the background. It creates a parallel workflow where you and the AI work on the same machine simultaneously.

That is a big shift from earlier computer use agents that locked up your machine.