What Hermes is, and why it is different

Most AI agents you have seen are task tools. You give them a job, they do it, they forget you. Hermes Agent, from Nous Research, is built on a different idea: an agent that is yours, that remembers you, and that gets better the longer you use it.

Three things make it stand out:

• It has a persona. A single file, SOUL.md, defines how it talks. Edit it and the agent changes character on the next message.
• It remembers. Conversations, facts, and context persist across sessions in a local memory, so it is a companion, not a one-shot.
• It learns. Hermes has a library of skills (research, notes, GitHub, documents, smart home, and many more) and a background curator that creates and refines skills from experience. The agent that finishes your week is not the one that started it.

All of that personality, memory, and learned skill is data about you. Which raises the obvious question: where does the model behind it run?

Why local is the right default for a personal agent

When a model only answers questions, where it runs is mostly about privacy and costs. When a model is your personal agent, holding your persona, your memory, and skills it learned from your own habits, it becomes about ownership. If the brain lives in someone else’s datacenter, then your assistant’s memory of you, and everything it learns from you, flows through a system you do not control: its uptime, its terms, its access policy, all of which can change without you.

Running the model locally changes the property, not just the policy:

• Your context stays on the machine. Your persona, your memory, the skills the agent builds: none of it has to leave your disk to reach the model.
• It works offline. Once the model is downloaded, you can disconnect entirely and keep talking to your agent. Turn on airplane mode and it still answers.
• No per-token bill. A personal agent you talk to all day, that runs scheduled jobs in the background, would meter up fast against a cloud API. Locally it costs nothing per token.
• No silent model swaps. The weights are a file on your disk. Your agent’s brain does not change under you between sessions.

The tradeoff is latency and raw capability: a model that fits on a laptop is smaller than a frontier cloud model. For a personal agent that lives with your data, that tradeoff is easy, and it keeps getting better as small models improve.

QVAC is the piece that makes the local half practical. It is an open-source SDK and CLI from Tether that runs models on your own device across every major GPU backend (NVIDIA, AMD, and Intel through Vulkan, Apple Silicon through Metal), and it ships an OpenAI-compatible server. That server is the bridge Hermes plugs into.

How it fits together

Two long-running pieces and you:

1. The QVAC server holds the model in memory and answers OpenAI-compatible requests on a local port. Start it once and leave it running.
2. Hermes runs the agent: persona, memory, skills, tools. You point it at the QVAC server once, then just talk to it.
3. Your messages go to Hermes, which thinks using the local model and acts with its tools.

The model inference never leaves the machine. (Some skills, like web research, still use the internet for that specific tool, the same way a browser does. The brain stays on-device.)

Which model to run

Hermes is tool-heavy: it sends a large set of tools on every turn, so it pays to run a capable model. The setup below uses Qwen3.6-35B-A3B, a mixture-of-experts model: 35 billion parameters of knowledge, only about 3 billion active per token, so it stays fast for its size. To switch models, change the constant in the config file (step 2 of the setup).

Be realistic: these are small local models, and tool-heavy agent work is where they struggle most. The 4B is for trying the setup, the 9B mainly adds reliability over the 4B rather than smarts, and the 35B-A3B MoE is the only one that drives the full toolset dependably, so it is what we recommend. It is a capable on-device assistant, not a frontier cloud model.

A GPU helps a lot but is not required. QVAC uses Apple Metal on Apple Silicon and Vulkan on NVIDIA, AMD, and Intel. Run qvac doctor to see what your hardware supports. On the 35B-A3B, expect the first turn to take around 20 seconds (Hermes sends a large system prompt plus its tools), and faster turns after. For a snappier agent, narrow the active skills.

Set it up

npm install -g @qvac/cli

mkdir -p ~/qvac-hermes && cd ~/qvac-hermes
cat > qvac.config.json <<'EOF'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3.6-moe": {
        "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M",
        "type": "llamacpp-completion",
        "default": true,
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 32768, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
EOF

mkdir -p ~/qvac-hermes && cd ~/qvac-hermes
cat > qvac.config.json <<'EOF'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3.6-moe": {
        "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M",
        "type": "llamacpp-completion",
        "default": true,
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 32768, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
EOF

mkdir $HOME\qvac-hermes; cd $HOME\qvac-hermes
@'
{
  "plugins": ["@qvac/sdk/llamacpp-completion/plugin"],
  "serve": {
    "models": {
      "qwen3.6-moe": {
        "model": "QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M",
        "type": "llamacpp-completion",
        "default": true,
        "preload": true,
        "config": { "tools": true, "toolsMode": "static", "ctx_size": 32768, "gpu_layers": -1, "reasoning_budget": 0 }
      }
    }
  }
}
'@ | Set-Content -Encoding utf8 qvac.config.json

qvac serve openai --model qwen3.6-moe

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

hermes config set model.provider custom
hermes config set model.base_url "http://localhost:11434/v1"
hermes config set model.default qwen3.6-moe
hermes config set model.api_key qvac

hermes

What you just built, and what to try next

You now have a personal AI agent running entirely on your own machine. A good first test is the obvious one: ask it whether it is in the cloud or local, then enable airplane mode and ask again. It keeps answering, because the inference was never leaving your device.

From there, the things that make Hermes Hermes all run on the same local model:

• Give it a personality. Edit ~/.hermes/SOUL.md, then send a message. The persona is loaded fresh each time, so the change is immediate.
• Let it remember. Tell it something about how you work, start a new session later, and it recalls it. The memory lives in ~/.hermes/.
• Use a skill. Run hermes skills to see the library, then ask it for something real: take structured notes, organize a project on its task board, summarize a folder of documents. Pick local skills (notes, files, board) to stay fully offline; web skills will use the internet for that tool.
• Let it improve. Over time the curator refines skills from what you do. That learning happens on your hardware and stays there.

For speed, narrow the active skills to what you actually use (hermes skills, hermes tools); a smaller toolset means a smaller prompt and faster turns on a local model.

Notes and limits

• A model that fits on a laptop is not a frontier cloud model. Expect it to need clearer instructions and to be slower than a cloud API today. That gap is closing as local hardware speeds up and small models improve.
• "Local" here means the model runs on your device. A given skill may still call the internet for its own job (web research, for example), the same as any app would. The intelligence, your persona, and your memory stay on the machine.
• QVAC is Apache 2.0 and free. Hermes Agent is from Nous Research (github.com/NousResearch/hermes-agent). The model is downloaded once and cached locally. QVAC source and docs: github.com/tetherto/qvac and docs.qvac.tether.io.