Hermes iOS Companion

Overview

Hermes iOS Companion is an independently built mobile control surface for the open-source Hermes Agent. It lets me continue conversations, inspect tool execution, handle security approvals, use voice input, and follow long-running tasks when I am away from the computer that hosts the agent.

The project is a deployed personal system and private beta, not a general consumer chat app. I designed and built the iOS client, its interaction model, gateway protocol integration, security boundaries, and Apple platform surfaces. It is not an official Nous Research product.

A Mobile Control Surface, Not a Chat Wrapper

The primary workflows are organized around agent continuity rather than message volume. Sessions stay grouped by project and instance, tool execution remains inspectable without taking over the conversation, and approval requests surface the command and authorization scope before a decision.

These screens were captured from a current-main Simulator build using fixture sessions, example commands, and isolated instance names. No production endpoint, credential, or private conversation is present.

The Problem: Desktop Agents Lose Their Interface When You Walk Away

A desktop AI agent can read files, run tools, and complete long tasks, but its interface usually disappears the moment the user leaves the desk. A mobile chat box alone does not solve that problem. The phone also needs to communicate execution state, preserve session continuity, surface system events, and make high-risk actions explicit.

The product goal was therefore not to shrink the desktop experience. It was to identify the few responsibilities that matter on a phone: lightweight input, status awareness, interruption handling, and trusted confirmation.

The Product Decision: Keep the Runtime Off the Phone

The iPhone does not run the agent runtime. Hermes remains on a Mac or VPS, where it has the files, tools, compute, and long-lived process context it needs. The phone acts as a secure control surface that sends input and renders state from the Gateway.

This boundary reduced product and security complexity. The Gateway remains the source of truth. The mobile client can reconnect, reconcile, or lose a transient event without becoming a second authority for the task itself.

Architecture: One Source of Truth, Several Feedback Paths

The iPhone reaches each Hermes Gateway through Tailscale HTTPS. Mac and VPS profiles keep endpoint identity, credentials, sessions, and notification routing separate. Requests move to the selected Gateway, while SSE streams provide foreground progress and APNs, Widget, Live Activity, and Dynamic Island provide system-level status.

SSE and push are feedback layers, not competing databases. After interruption or reconnect, the client reconciles against Gateway state instead of assuming every transient event was received.

The Work Continues Outside the App

Share Extension turns a URL, text, or image into agent input and preserves it in an outbox when immediate delivery is unavailable. WidgetKit exposes instance-aware status, while ActivityKit and Dynamic Island carry live task state into the system UI. Native text selection keeps long agent output useful as working material rather than a read-only transcript.

These Simulator captures prove the rendered Apple-platform surfaces. Physical-device acceptance separately covers APNs delivery, Face ID or passcode app lock, Share outbox delivery, and lifecycle behavior that Simulator screenshots cannot establish.

Core Challenges

The difficult work was not rendering chat bubbles. It was preserving trust and continuity across networks, processes, devices, and iOS lifecycle boundaries.

Remote Control Must Stay Legible

Remote execution creates a human factors problem: the user needs enough context to make a decision without reading a desktop-sized trace on a phone. The approval flow exposes intent and authorization scope, then requires an additional confirmation for dangerous session grants. Separately, Face ID or device passcode protects app re-entry and destructive Gateway restart.

During execution, reasoning and tool calls collapse into a compact progress surface. Details remain available for inspection, but the final answer stays outside the trace so status, diagnostic detail, and outcome do not compete for attention. The public screens in this case study use safe simulator fixtures from the current main build.

Validation Snapshot

The strongest evidence is the complete system running across a physical iPhone and two self-hosted environments. Automated tests cover deterministic behavior, while device acceptance verifies the Apple services and network conditions that mocks cannot prove.

Known Limits and Release Boundary

Hermes iOS Companion is a private beta and deployed personal system. It is not on the App Store, is not publicly available, and is not presented as an official Hermes app. Setup still assumes a self-hosted Hermes Gateway and Tailscale access, which is appropriate for the current audience but not consumer onboarding.

Public screenshots use example data and sanitized fixtures. Live Gateway URLs, API keys, pairing payloads, Tailscale domains, device tokens, APNs credentials, and private conversation content are intentionally excluded.