How to Get Hermes Agent to Use AgentMail (And Why It's Harder Than It Should Be)

Someone on Reddit asked me this today, and honestly, we went through the exact same struggle. So here's the full breakdown of what's going wrong and how to fix it.

The Problem

You've set up AgentMail. You've got your API key. You've added it to ~/.hermes/config.yaml. Hermes says it can see the MCP AgentMail service running. But when you ask it to check your inbox... nothing. Or worse, it gives you a 403.

This is one of the most frustrating setup experiences with Hermes Agent. Here's why it happens and how to actually fix it.

The Exact Issue from Today

The person on Reddit described it perfectly:

"How do you get Hermes to use AgentMail? I have set it up as per the instructions but for the love of God I can't get Hermes (local LLM) to use it, even when I give it the commands and it tells me it can see the MCP Agent mail service running. I have never struggled with any AI things this hard!!"

We hit the same wall today. Here's what we found.

Why It Doesn't Work: 3 Common Failure Modes

1. The API Key Has No Permissions

When you generate an API key from the AgentMail Console, it might not have the right permissions. A 403 (not 401) means your key is valid but has no permissions. The key needs at minimum:

inbox_read
message_read
message_send

Fix: Go to console.agentmail.to → API Keys → verify permissions are granted.

2. The Account Email Isn't Verified

Even with a valid API key, AgentMail requires the account email to be verified. Check that inbox for a verification email from AgentMail and click the link.

3. Hermes Isn't Reading From Config

The API key must be in ~/.hermes/config.yaml under the agentmail: section:

agentmail:
  api_key: "am_us_..."

Not in environment variables. Not in a separate file. And the key name is api_key, not am_key or token.

The Fix (Step by Step)

Step 1: Sign Up at AgentMail

Go to https://agentmail.to and sign up. You can use any email provider — Gmail, Outlook, Hotmail, whatever. AgentMail will send a verification email to whatever address you use.

Step 2: Create an Inbox and API Key

From the AgentMail Console, create an inbox (e.g. yourname@agentmail.to)
Generate an API key
Make sure the key has inbox_read, message_read, and message_send permissions

Step 3: Store the Key in Config

Edit ~/.hermes/config.yaml:

agentmail:
  api_key: "am_us_...here"

Step 4: Test It

Ask Hermes: "Check my agent inbox for new emails."

If it works, you'll see your inbox listed with any unread messages. If you get a 403, go back to Step 2 — it's a permissions issue.

Step 5: Create a CLI Helper (Optional but Recommended)

The raw SDK works, but a CLI helper makes life easier:

python3 ~/.hermes/scripts/agentmail_cli.py triage

This lists all inboxes with unread message counts and previews. Much easier than asking Hermes to debug the SDK every time.

Alternative: Himalaya (Works With Most Email Providers)

If you want a CLI email solution that works with Hermes and your existing email, Himalaya is a solid alternative. It uses IMAP/SMTP directly.

Works with: Gmail, Yahoo, Fastmail, ProtonMail, and most IMAP providers.

Does NOT work well with: Outlook/Hotmail. Microsoft no longer allows app-specific passwords for IMAP access, and the OAuth2 route requires Azure AD app registration which is tedious and time-consuming.

# Install
brew install himalaya

# Configure
himalaya account configure

# Use from Hermes
himalaya envelope list
himalaya message read 42

TL;DR

AgentMail signup works with any email (Gmail, Outlook, Hotmail, etc.)
403 error = API key lacks permissions. Check console.agentmail.to
Account not verified = Check your email for AgentMail verification
Key not recognized = Must be in ~/.hermes/config.yaml as agentmail.api_key
Alternative = Himalaya works with most providers except Outlook/Hotmail
Outlook/Hotmail users = Use AgentMail (works with any signup email) or create a Gmail for your agent

The whole process should be easier. Hopefully this saves someone the headache we went through today.