Building a HIPAA-Ready AI Agent on GCP with Hermes

Everyone Wants an Agent on Their Data. Few Can Put PHI Near One.

In October, Nous Research raised $90M at a $1.2B valuation to take its open-source Hermes agent into enterprises, after 22M+ downloads since February (WSJ, 2026-10-08). Every one of those companies has a database it would like an agent to read. In health care and insurance, some of that data is protected health information (PHI).

Most advice on a HIPAA compliant AI agent is a vendor checklist. This is a build. We made Clive, a Hermes agent that does a sales manager's admin work for an inbound Medicare sales floor of about 20 reps: the morning numbers, who to huddle with, attendance checks, the midday update, drafts. The first commit was 2026-11-04. It runs on one private Google Cloud VM next to a Postgres database that a pipeline fills from the company's dialer, CRM and enrollment tools.

The argument: for an agent, most of HIPAA is enforced below the model, in places a prompt cannot reach. Which vendors can receive a byte. Where a call is processed. What a database role can read. What the network allows. What a human must approve. How long anything lives.

”

"We should treat all failures in the same way: find the fundamental causes and redesign the system so that these can no longer lead to problems." — Don Norman, The Design of Everyday Things (§ "CHAPTER FIVE: HUMAN ERROR? NO, BAD DESIGN")

HIPAA Asks Organizations Questions; the Agent Gets Layers

HIPAA is not a property of software. A covered entity needs a risk analysis, written policies, staff training and signed contracts. Software carries out the technical parts. Three pieces of the rule matter most for an agent.

• Business associates. A covered entity may disclose protected health information to a business associate only with satisfactory assurances, documented in a written contract (45 CFR 164.502(e); HHS publishes sample BAA provisions). Every vendor that touches the data is in that chain.
• Minimum necessary. Covered entities and business associates must make reasonable efforts to limit the information they use, disclose or request to the least needed for the purpose (164.502(b)).
• Technical safeguards. The Security Rule's 164.312 names five: access control, audit controls, integrity, person or entity authentication and transmission security.

Each maps to a layer you can build and test:

Why Hermes, Not OpenClaw

We have written a lot about OpenClaw: its memory, its identity files, and the head-to-head with Hermes. OpenClaw is the stronger choice for a fleet of personas across many channels. For an agent that sits next to PHI, four Hermes features decided it.

The identity file is the other reason. OpenClaw's SOUL.md is a file the agent is meant to edit, and we covered how that became an attack surface; its gateway also shipped a one-click RCE. In Clive, each SOUL.md lives in git and a script pushes it to the VM. The agents have no file toolset, so none of them can rewrite its own rules. Fairness cuts the other way too: Hermes has a shorter security record, which is not the same as a cleaner one.

Hermes's plugin hooks did the rest of the work. Two small plugins fixed what config could not: one rewrites the Vertex host for the us region, the other retries a capacity error once before falling back.

Only Vendors With a BAA Ever Receive a Byte

Rule one in Clive's repo: no vendor, channel or tool sees PHI unless it has a BAA. The model path has two vendors, and no other model key exists on the machine.

• Vertex AI, under Google's BAA. Google has renamed the product Gemini Enterprise Agent Platform. Its HIPAA page lists products, not models, and says "Complying with HIPAA is a shared responsibility between you and Google." It says not to use pre-GA offerings with PHI, so we use GA models only and keep preview features off.
• OpenAI, under a BAA, through one endpoint. Clive calls only /v1/responses, with store: false, on OpenAI's US-residency host, through a restricted key for a dedicated project. It never calls Assistants, threads, conversations, vector stores, files, batches, background mode or hosted tools. OpenAI's data-controls guide says plainly that "Web Search with live internet access is not HIPAA eligible and is not covered by a BAA."

Why there is no Claude in the stack

Not for lack of wanting it. The BAAs with Google Cloud and OpenAI were self-serve: a few clicks in each console. Anthropic's is a sales conversation. Claude on Vertex AI does sit under Google's BAA, but getting quota for it went through an account rep, and a sales floor that wants its morning numbers this week can't wait on a capacity ticket. So Clive runs Gemini and OpenAI models, and no Anthropic key exists on the machine. If Claude is your first choice for a regulated build, start the paperwork before you write any code.

Channels follow the same test. Telegram has no BAA and is out. Slack counts only inside a workspace its BAA covers, and that BAA covers Slack, not your bot. Slack says it "does not have a business associate agreement with any third-party application providers," so the app's side of the conversation (here, Hermes on our VM) sits under the agent operator's own BAA with the covered entity.

What "eyes off" means

"We have a BAA with OpenAI" hides a detail. OpenAI's BAA guide defines Modified Retention as any of four states: Modified Abuse Monitoring, Zero Data Retention, Safety Retention or Eyes Off. Two of them get confused:

The guide says Eyes Off means "customer content will be retained in abuse monitoring logs and may be retained for application state". Our org is Eyes Off, and the admin API refuses changes to it. That is not zero retention: a response sent with store: true could be fetched back later. We send store: false on every call.

OpenAI support confirmed to us on 2026-11-05 that with an executed BAA and an org provisioned for Modified Retention, Eyes Off lets eligible endpoints, /v1/responses among them, process PHI even if data is retained. Sending store: false anyway costs nothing and keeps application state out of the picture.

Pin the Region: Vertex "Global" Can Process Anywhere

A BAA says which products may hold PHI. It does not say where your prompt runs. Google's documentation does: "When you use the global endpoint, requests and their cached data can be served from any region." Implicit caching keeps prompt prefixes for up to 24 hours, so this covers stored prompts as well as the live call.

Clive calls Gemini at Vertex's us multi-region, never global, on the regional host aiplatform.us.rep.googleapis.com. Hermes builds a host name from a region string and Google rejects the result for us, so a small plugin swaps in the regional host. That host is not served on Google's private-access addresses, so the call goes through the egress proxy described below. OpenAI calls go to its US-residency host.

It costs something. In a 30-call test, us was slower and stalled more than global. We took the slower path because the other one cannot tell you where the data went.

Read-Only Is a Database Role, Not a Prompt

Clive's analyst profile answers data questions by writing SQL. Telling a model to "only read" is a hope. The database enforces it, in layers:

1. Identity. The agent signs in as a Cloud SQL IAM database user with a short-lived token from the VM's service account. No password exists to leak.
2. Role. The user belongs to a read-only role with a 30-second statement timeout. Grants go to roles, never to the service account, and by named relation, never by schema, so a new table stays invisible until someone grants it.
3. Tool. The agent reaches the database only through one MCP server on the host. It parses each statement and refuses anything but a single SELECT over an allowlisted table.
4. Session. The connection is read-only, with an empty search path, and returns at most 50 rows.

The shape of the tool's check, simplified:

tree = sqlglot.parse_one(sql, read="postgres")           # one statement, or reject
if not isinstance(tree, exp.Query):
    reject("only SELECT queries are allowed")
for node in tree.walk():                                  # no INSERT, COPY, SET, LOCK...
    if isinstance(node, FORBIDDEN_NODES):
        reject(f"{type(node).__name__} is not allowed")
for table in tree.find_all(exp.Table):                    # schema-qualified, on the list
    if f"{table.db}.{table.name}" not in ALLOWED:
        reject("not an approved table")
# function denylist: pg_*, dblink, file readers, query_to_xml...
 
conn.execute("SET SESSION CHARACTERISTICS AS TRANSACTION READ ONLY")
conn.execute("SET statement_timeout = 30000")
conn.execute("SET search_path = pg_catalog")
rows = cur.fetchmany(MAX_ROWS + 1)                        # 50, then flag truncated

The tool's docstring ranks them: the role is "the real boundary," and the checks are "a second one that fails earlier and explains why."

The checks have to be precise or they leak. A query can define a CTE named like a system catalog view, such as pg_stats, which holds sample values of each column. A naive check does not recognize the name and passes it. This one tracks which WITH clauses are in scope at each table reference. The tool also logs every statement, not its results, which is the audit-controls answer.

Minimum necessary, as a design rule

The harder question is what the agent may read at all. For call transcripts, a stateless labelling job in the data pipeline reads the text, on a model under a BAA. Clive reads the labels as counts by rep and week, never the transcript. "Need-to-know" is no policy until something can check it:

”

"Who determines “need-to-know” and how?" — Ross Anderson, Security Engineering (§ "9.2. WHAT IS A SECURITY POLICY MODEL?")

In Clive, a grant answers it: a table is in the role or it is not. Transcript labels are allowed under that rule and not yet built.

The Network Says No Before the Agent Can Ask

An agent that can run code can try to reach anything. Clive's host is built so that an attempt fails and leaves a record.

• No public address. The VM has no external IP, and organization policy forbids one. The only way in is a private mesh network (Tailscale). Google's identity-aware proxy is closed and kept as break-glass. Tailscale signs no BAAs, and it doesn't need to here: traffic is end-to-end encrypted WireGuard, and its security page says it stores only metadata and cannot inspect traffic.
• Egress allowlist. The agent's user can reach loopback, the metadata server, the database and Google's private-access range. Everything else is rejected and logged.
• Named hosts only. Public internet goes through a CONNECT proxy on port 443 with a short list:

us.api.openai.com
aiplatform.us.rep.googleapis.com
slack.com
files.slack.com
(Slack's websocket hosts)

• Shell commands cannot mint tokens. The host mints Vertex tokens from the cloud metadata server. Agent shell commands run in Docker with outbound traffic dropped and the metadata address blocked. A command the model writes cannot ask for a credential.
• Toolsets off. Web, browser, terminal, file, code execution, delegation and cron are off for every bot. The analyst gets the SQL tool, the routines profile gets none.

That last item has a story. Our lockdown script disabled toolsets, and a review four days in found it had changed only the command-line platform. On Slack, the bots kept every tool. The fix sets toolsets per platform. Hermes's docs say "the container itself is the security boundary" for the Docker backend, and that profiles do not sandbox the agent. The sandbox is the operating-system user, the container and the firewall.

”

"Software should also be designed so that the default configuration, and in general, the easiest way of doing something, should be safe." — Ross Anderson, Security Engineering (p. 215)

A lockdown in a default holds. A lockdown in a checklist held on the one platform somebody tested. For the wider threat model, see the agent attack surface and the safety stack. For what happens when containment lives in the prompt, see rogue agents and containment.

The Agent Drafts; a Human Sends

Clive writes coaching notes, reminders and updates. A manager reads and sends them. Clive's toolsets include nothing that messages a rep or a member; replies go to the manager who asked. The config behind it:

approvals:
  mode: manual          # dangerous actions always ask a person
  cron_mode: deny       # scheduled runs cannot approve anything
  unattended_mode: deny
memory:
  write_approval: true  # no memory write lands without a human yes
telemetry:
  shared_metrics:
    enabled: false
terminal:
  backend: docker

In Hermes, manual mode prompts on dangerous commands, and a prompt that times out is denied. Scheduled and unattended runs cannot answer, so they deny. Our rule is that manual approval covers send, pay, delete and publish.

Memory gets the same rule. Hermes keeps two small memory files per profile (see how Hermes implements memory). With write approval on, a write on a messaging platform is staged until someone approves it. Clive's memory holds manager preferences only, with no numbers, member data or judgments of a rep. The analyst has memory off. This is the human-in-the-loop firewall at small scale: a person approves at the point where something irreversible happens.

Everything Expires, and Restore Is Tested

Retention is a security control, because data that is gone cannot leak.

The restore drill matters more than the schedule. A backup you have not restored is a belief. Ours rebuilds Clive from a snapshot into a VM that cannot send anything, then checks that Hermes, both bots, the SQL tool, the task board and the session stores came back, and that no outbound path exists. It passed 15 of 15 on 2026-11-05, then deleted everything it made.

Two limits show up. Free text in the session store can hold PHI, and Hermes has no switch to stop storing it, so 90 days is a ceiling we want to cut. And a snapshot holds what was later deleted.

The Model Is the Slowest Part You Don't Control

The median passes review. The tail does not. Over Nov 4–10, Google's metrics show 133 of 2,904 calls to Gemini 3.8 Flash (4.6%) got a 429. The cause was not our quota, route, endpoint or thinking setting. An average hides this; see why averages lie for agent fleets. What we did:

• Wait longer, then fall back. Hermes drops a silent stream after a stall limit. We moved it from 30 s to 180 s, because a cancelled call is probably still billed. Of 83 successful calls whose first token took over 30 s, 41 took over 120 s. Hermes then retries once and falls to the next provider.
• Retry a 429 once on the same vendor. By default Hermes falls back at once and keeps the session off Vertex for 60 s. A small plugin gives one retry first. Of 51 Vertex calls sent within 5 s of a 429, 47 went through.
• Change the order. Since 2026-11-10 the manager-facing bot runs an OpenAI model first and Gemini Flash second. The analyst runs gpt-6.1-sol at medium effort. Gemini now carries only the scheduled routines.

Accuracy and cost on the analyst's question set (29 questions with known answers, run 2026-11-11):

Sol gets three more questions right at about 15 times the cost per right answer, so we run it on the analyst and watch the bill. These costs are Hermes's estimates from a price table, not reconciled with an invoice; the "per correct" framing follows cost per completed task. The misses are about meaning, not SQL: which teams count as "the floor," which of two sale counts to use. Graders need an independent source (building agent evals): in an earlier run the analyst checked a wrong answer against a table that shared its mistake, and the two agreed.

The budget is $150 a month on Vertex, with alerts at 50%, 90%, 100% and on forecast. GCP cannot see OpenAI's bill, so a five-minute heartbeat reads Hermes's usage records and warns above $5 in 24 hours. Our estimate, before measuring, is $0.50 to $1.00 per team per day, against about 95 minutes of manager time. It is an estimate, not a result. For forecasting an agent bill in general, see AI spend forecasting.

What's Still on the List

A vendor checklist ends at launch. A build keeps a list.

• The VM disk uses Google's default encryption. Moving it to a customer-managed key is scripted; old snapshots keep the old key until deleted by name.
• Transcript labels and an audit agent are designed, not built.
• Approvals. In Hermes, any authorized user can click an approval, not only the requester. That needs a plugin once there are 20 managers.
• The incident runbook is written and has not been rehearsed.

Each item sits in a risk register with an owner, a proposed fix and a rough cost.

A Build Order You Can Copy

What to Do Monday

Test your weakest layer with an attack, not a review. Ask your agent to read a table it should not, to open a connection to a host it should not, and to run a shell command that asks for a cloud token. Each should fail somewhere the model does not control and leave a log line. If one succeeds, you have found where the prompt was doing the system's job.

Then write the open-items list, with an owner on every line. Your lawyer will read it first.