How GridWeave is built

A single Go server binary that runs as many roles, a Rust agent on each device, a React web app, and three places that hold state: PostgreSQL, a NATS bus and object storage. This page says what is built and what is not.

Under construction. Not released.

The shape of it

One binary, many roles, state in three places.

The server gridweaved takes the role to run as its first argument. A service process holds connections and caches only. State lives in PostgreSQL, on the bus and in object storage, so a role can run as many copies as needed.

GridWeave components Clients on the left: the web app and the gridweave command line reach the server roles api over HTTPS and relay over WebSocket; the gridweave-agent on each device reaches the agent-gateway over mutual TLS and the relay over WebSocket. In the middle, one gridweaved binary with the roles api, relay, agent-gateway, worker, web, amt (optional, simulator only) and bridge (not implemented). To the right, the NATS JetStream bus, and beyond it three kinds of state: PostgreSQL, object storage and the key store. Clients and devicesgridweaved rolesBusState HTTPS WebSocket mutual TLS NATS JetStream Web appand gridweave (command line) gridweave-agentRust, on each device apiHTTP API, event stream relayterminal and files sessions agent-gatewayagent listener, commands workeroutbox, jobs, schedules webserves the web app assets amtoptional, simulator only bridgenot implemented PostgreSQLstate, audit, outbox Object storagelocal disk or S3 API Key storefiles or OpenBao
Components of GridWeave. Solid lines are request and event paths, dashed lines are WebSocket sessions and optional links. The roles use PostgreSQL directly as well as the bus; the figure keeps the lines short. In words: the web app and gridweave call the api role over HTTPS and join terminal and files sessions on the relay role over WebSocket. The agent connects to the agent-gateway over mutual TLS and to the relay over WebSocket. The amt role also accepts connections from Intel AMT firmware. The bridge role is not implemented.

The binaries are gridweaved (the server), gridweave-admin (the operator tool), gridweave (the user command line) and gridweave-agent (the endpoint agent). They were renamed in feature 043 from nsd, nsadmin, nsctl and nsagent. The NS_* configuration variables, the ns.* protocol packages and the ns_ database roles kept their names.

Every change a user makes is written in one PostgreSQL transaction together with an outbox row. A worker then appends the audit entry and publishes the event to the bus, at least once and with a stable event id, and the event reaches browsers over a WebSocket if the tenant, topic and permission allow it. The audit entry is therefore recorded with the change, not as an afterthought.

Server

The roles of gridweaved.

The Go server is one module and one binary. The first argument selects the role, in a separate process or all in one. gridweave-admin is the operator tool and is not a service.

Roles of the server binary and what is built, from the product's server roles page. Status as of 13 November 2026.
RoleWhat it doesStatus
apiThe HTTP API under /api/v1, the event stream, authentication and authorisation. Needs the database, the bus and a key store.Built
agent-gatewayThe agent listener: enrolment, mutual TLS, presence, reports and forwarding of commands. Holds no authoritative state, so several gateway nodes behave as one.Built
relayThe data plane of terminal and files sessions: WebSocket legs, pairing, flow control, revocation. Terminal and files only; the other protocols the contract names are defined and refused. Taps terminal sessions into a recording when policy says so, and admits guest links.Built
workerThe outbox relay, scheduled jobs and the janitor for sessions a dead relay left open.Built
webServes the web application assets. They are embedded in the binary only by the release build.Built
amtOptional. Intel AMT: a listener for connections from the firmware, management over WS-Management, and device operations over the bus. Off unless enabled in the configuration.Simulator only
all-in-oneEvery implemented role in one process, with an embedded bus and a local object store. Used for development and small installs.Built
bridgeProtocol bridges. The role is registered and exits with a message that it is not implemented.Not built

Each role opens only the keys it needs. For example the gateway never opens the Command CA key that signs commands, so a compromised gateway cannot sign one. Configuration is one YAML file plus environment overrides, checked against a schema; unknown keys are refused.

Device

The agent, gridweave-agent.

A Rust program that runs as a service on each device. It enrols once with a token, then keeps one long-lived connection to the gateway.

  • Service on three systems

    A systemd unit on Linux, a LaunchDaemon on macOS and a Windows service. Proven on Ubuntu 24.04 ARM64. On macOS it ran as a foreground process and the service install was only a dry run. The Windows parts written most recently have never been run.

  • Connection and identity

    TLS 1.3 with the platform root as the only trust anchor, HTTP/2, one stream, a heartbeat every 15 seconds by default and reconnects with backoff. The certificate is renewed at two thirds of its life. The key is a protected file, not in a TPM or the Keychain.

  • Collectors

    Per-system collectors report device state, network and inventory to the server. There are separate collectors for Linux, macOS and Windows.

  • Executors

    Commands run only after the agent has verified them. The executors for Unix and for Windows are separate code, and only a verified command can reach them: the compiler refuses any other path.

  • Relay session client

    A session offer is one more signed command. The agent checks it and the grant inside it, opens its leg to the relay, asks the person at the device when policy says so, and serves a terminal or a files session.

  • Six messages, no code

    The agent acts on exactly six server messages on its stream and none of them carries code to run. Updates arrive by a separate path (next card) and are signed manifests checked against keys built into the agent; today those are development keys.

  • Self-update and rollback

    An installed agent asks the gateway for an update every 15 minutes by default, but only learns a version and a reason, never an address. It downloads from the server it enrolled with, verifies the signed manifest and the artifact, swaps its binary, and exits so that the service manager starts the new one. If the new build is not connected and stable within its window, the previous build is restored, either by the new build or by a watchdog timer that the previous binary armed first. Proven on Linux under systemd; macOS launchd and Windows are unverified.

  • Staged rollouts, server side

    The gateway’s update endpoints answer from a rollout engine: rings over a stable hash of the device id, a weekly window, a concurrency limit, per-group and per-device policies and a health gate that halts the rollout when too many devices fail or roll back. Every refusal is audited. Managed in the console under Updates, with the agent.rollout.manage permission.

  • Names and paths

    The service, unit, directories and local socket are all named gridweave-agent: for example the systemd unit gridweave-agent.service, /etc/gridweave-agent/agent.toml and /run/gridweave-agent/gridweave-agent.sock. A local socket lets helper applications talk to the agent, with peer-credential access control. The Windows and macOS names were not run after the rename.

Not built: proxy support (the agent connects directly), a WebSocket fallback, and a helper application for consent prompts on Linux and macOS desktops.

Web, data and keys

The web app and the three places that hold state.

  • Web application

    React 18 with TypeScript and Vite. Its API client is generated from the OpenAPI contract, so the browser and the server cannot drift apart unseen. It was redesigned with Tailwind CSS and shadcn/ui and is checked by a layout check at three widths in both themes. It includes Terminal and Files tabs that use the relay, a recordings player, a sharing tab for guest links and an Updates area for releases and rollouts. Folder browsing in Files is not built.

  • PostgreSQL 16

    The system of record. Every tenant table carries a tenant id and a row-level security policy that is forced, so the database enforces isolation and not only the application. A test discovers every such table and checks that one tenant cannot read, move, insert or delete another's rows.

  • NATS JetStream

    Presence (a key per device whose lease any frame from the agent renews, three heartbeat intervals and at least 45 seconds), rate limits and single-use grant records live in JetStream buckets. Commands travel as request and reply, and events as publish and subscribe, on subjects prefixed by tenant. all-in-one embeds the bus; split roles need an external server.

  • Object storage

    An S3-compatible API or local disk, behind one gatekeeper that scopes every key to a tenant. The first users are the agent release store and terminal session recordings. The S3 driver was checked only against SeaweedFS and local TLS test servers.

  • Key store

    One interface for every private key and wrapped secret, with three drivers: plain files (the default, for development only), AES-256-GCM encrypted files, and OpenBao Transit for production. The production driver is not the default, and its chart path was rendered but not run against a real OpenBao.

  • The database never holds a key

    PostgreSQL stores references to keys and wrapped values, never a private key. Password hashes, TOTP secrets and identity-provider secrets are wrapped by the key store.

Contracts first

Interfaces are defined before the code.

Three machine-readable contracts are the only place an interface is defined. A contract change lands before or with the code that uses it, and clients are generated, never written by hand in parallel.

The contracts, what each describes and what is generated from it.
ContractDescribesGenerated into
OpenAPI 3.1The HTTP API under /api/v1: 150 operations at the time of writing.The Go server interface and the TypeScript client types of the web app.
AsyncAPI 3.0The event stream a client opens over WebSocket, and the list of event types.Nothing; a check compares it with the events the server emits.
ProtobufThe agent protocol, relay grants, and the relay framing, control messages and credit.Go types for the server; the agent compiles the same files at build time.
  • Every API operation declares the permission it needs, or the build fails.
  • Protobuf field numbers are never reused, and receivers ignore unknown fields.
  • An API change that removes or retypes something already in the main branch fails a check.
  • No operation takes free text as a command, and no response schema carries a credential except the few listed once-only secrets.
  • Generated code is committed and a check fails if regenerating changes it.

Path of a command

The server signs, the agent decides.

An operator runs a script on up to 500 devices. Each device receives its own signed envelope and checks it before acting.

A command travels from an operator to the agent and backSequence of eleven steps. The operator posts to the api. The api records the job and audit entry in one transaction and signs each envelope with the Command CA signer. It sends a request on the bus; the bus hands the command bytes to the agent-gateway holding the device; the gateway forwards the signed command to gridweave-agent. The agent verifies chain, signature, device, tenant, time, replay and permissions, then runs the command or answers refused. The result goes back through the gateway and the bus to the api, which stores it and emits a job.updated event to the operator. Operator api bus (NATS) agent-gateway gridweave-agent 1 POST /devices:run 2 record the job and its audit entry in one transaction; then sign each envelope with the Command CA signer 3 request cmd.{tenant}.{device} 4 command bytes 5 SignedCommand frame 6 the agent verifies: chain, signature, device, tenant, time, replay, permissions 7 run it, bounded, or answer REFUSED 8 CommandResult 9 cmdres.{tenant}.{device} 10 result 11 stored; job.updated event
The path of a command, in eleven steps. The gateway forwards the signed bytes without reading them and cannot alter them. A result returns on a subject whose device part is the one the connection authenticated as.
  • The audit entry is in the outbox in the same transaction as the job, so the order is recorded, then sent.
  • The Command CA is a fourth issuer under the installation root, which the agent pinned at enrolment. Its signer certificate lives 24 hours. Device certificates cannot sign commands, and neither can the gateway.
  • The agent checks the chain and the signature before it parses the envelope. It then checks that the device and tenant are its own, that the command is not expired (a lifetime of at most two minutes) and has not been seen before, and that the permissions named in the envelope cover the command.
  • With no connection to the device the answer is that the agent is not connected, within two seconds.

Windows executors were verified on one virtual machine that later became unreachable; the effect of shutdown, reboot and sleep on Windows was left out of testing on purpose.

Path of a remote session

The relay carries sessions, not trust.

A terminal or files session joins a client and an agent through a relay. Each side presents a signed, single-use grant.

A remote session is created, paired and endedSequence of thirteen steps. The client posts to the api, which authorizes, merges consent, inserts the session and picks a relay. The api sends a signed offer carrying the agent grant through the bus and the agent-gateway to gridweave-agent, which verifies the command, grant, relay address and pin. The api answers the client with the relay address and the client grant. The client and the agent each connect to the relay with their grant as the first frame. The relay consumes each grant once, checks the session and pairs the legs, then answers Accepted and Ready. Consent is asked at the device according to policy. Data flows within the credit each side has granted. A revoked permission, device or tenant ends the session with a named reason. Client api relay agent-gateway gridweave-agent 1 POST /sessions 2 authorize, merge consent, insert the session, pick the least loaded relay 3 signed offer with the agent grant, via the bus 4 SignedCommand 5 the agent verifies the command, then the grant, the relay address and its key pin 6 201: relay_url and client grant 7 connect: client grant is the first frame 8 connect: agent grant is the first frame 9 consume each grant id once, check the session, then pair the two legs 10 Accepted, then Ready 10 Accepted, then Ready 11 consent at the device: none, notify or ask 12 data frames, only within the credit granted 13 revoked permission, device or tenant: the relay closes the session with a named reason and the agent enforces again
The path of a remote session, in thirteen steps. Step 3 goes to the device as an ordinary signed command, so the agent verifies it as it verifies any command. The product's tests show a session ending within two seconds of a revoked permission.
  • A grant is an Ed25519 signature over its claims: session, tenant, device, role, protocol, permissions, consent policy and a window of at most 60 seconds to connect.
  • The relay consumes each grant id once with an atomic create in a bus bucket, so a replay at the same or another relay is refused. A grant never appears in a URL, a log or an audit entry.
  • Flow control is credit based, per direction and in bytes. A sender may not exceed the credit it was given, and the relay returns credit only after the other side has taken the frame.
  • Consent is asked at the device when the policy says so: none, notify, or ask. With nobody to ask, a setting decides, and the default is to refuse.
  • When policy or the operator asks for a recording, the relay writes both directions of a terminal session into the tenant’s object storage in chunks with a hash chain, and the database holds the reference values, so a reader detects a changed, dropped or reordered chunk. By default a recording that cannot be stored ends the session. A guest link is a second kind of client grant: the token is 256 bits, only its hash is stored, a redemption is checked against its creator’s current rights, and revoking the link closes both legs.

Remote desktop on the relay is not built. Terminal and files are the only protocols opened, and the real agent has been exercised through the relay by one end-to-end test on macOS. Recording and guest sharing exist for terminal sessions only, and were tried with fake agent legs and the real relay, not a real agent or browser; the agent shows no notice of recording or of a guest yet.

Deployment shapes

Runs in one process or in many.

None of these is a production deployment. The product says so for each one.

  • All-in-one, to try it

    gridweaved all-in-one runs every implemented role in one process with an embedded bus and a local object store. It needs PostgreSQL.

  • One Linux host

    All-in-one under systemd with PostgreSQL and TLS. Reference files are in the product. It was performed on one Ubuntu virtual machine and is not production ready.

  • Split roles with Compose

    The roles run as separate containers behind one origin, with an external NATS server that has JetStream. A split deployment needs a shared key directory with the file key drivers, or one token per role with OpenBao.

  • Kubernetes with Helm

    A Helm chart with a Deployment per role. It was installed on one single-node k3s cluster only. The ingress, autoscaling and network policies were not run on a cluster. It is for evaluation.

  • Container images

    A server image with every role, built as static binaries and run as an unprivileged user, and a Linux agent image. Both build for arm64 and amd64, but nothing is signed or published, and the amd64 builds were not run.

  • The dockerised development environment

    make dev starts PostgreSQL, Keycloak, a certificate authority, an administrator and the server. make dev-docker runs the same inside containers, and builds and tests can also run in a builder container.

Scale

What was measured, and what was not.

One load test on one machine. These numbers describe that machine. They are not a result for the reference hardware and they do not show that any scale target is met.

One host, shared with other work.

The test ran on an Apple M4 Max with 128 GB, in use by other work during every run (load average 14 to 19 for the main run). The load generator, the server and a PostgreSQL container with two virtual CPUs shared that machine. Nothing ran on Linux, on x86-64 or on more than one gateway process, and no figure was repeated.

Results with 10,000 simulated agents held for three minutes on that machine, one server process, a 15 second heartbeat. From the product's load report of 13 November 2026.
MeasureResult
Agents held10,000 of 10,000 connected, with no error class in the steady state
Command from the API accepting the job to the agent holding it95th percentile 198 ms, with 50 commands a second
Device seen online after it connects95th percentile 357 ms during the ramp
Server memoryAbout 1.45 GB resident, roughly 137 KB and 7 goroutines per connected agent
Server CPU while holdingAbout a quarter of one core, median
All 10,000 cut and reconnecting99% back in 56.8 s with the default database pool of 16, and in 19.2 s with a pool of 64, under very high host load (one sample each)

Known limits of these results

  • The 50,000-agent target was not run. Extrapolating the memory linearly gives about 6.8 GiB, which overshoots the 6 GiB bound of that target by about 13%. Nothing tests that the scaling is linear, and the memory per connection was not profiled.
  • Heartbeat and presence lease: fixed in feature 042. With a 60 second heartbeat the old 45 second lease lapsed every cycle and devices flapped between online and offline. Now any frame from the agent renews presence, the lease follows the announced heartbeat (three intervals, at least 45 seconds) and the server refuses to start with a lease shorter than twice the heartbeat. In one 150-second run with 200 simulated agents at a 60 second heartbeat, the disconnect events fell from 407 to 0. Offline detection can take longer than 45 seconds when the heartbeat is lengthened: up to 180 seconds at 60. The 10,000-agent results above predate the fix and used the 15 second heartbeat; the fix was not re-measured at 10,000 or on Linux.
  • The database pool bounds a mass reconnect. Each agent connection costs five transactions, and the pool defaults to the number of CPUs.
  • Enrolment burst. 10,000 enrolments at 1,000 a minute from one address, all accepted. This ran once and is not asserted by a test.