Runtime

Runtime

The runtime service is the per-tenant data plane pod. Each tenant gets exactly one runtime pod — runtime-t-{tenant_short} — running inside the paper-board Kubernetes namespace. The pod is stateless: it carries no Postgres schema and stores nothing beyond its own in-flight request state.

What runtime does

Runtime is a thin gRPC proxy between inbound agent invocations and the agents service. When a client posts to POST /v1/sessions/{session_id}/invoke, the runtime-router authenticates the JWT, resolves the correct tenant pod, and calls runtime.v1.RuntimeService.Invoke on that pod. The tenant pod then calls the agents service over internal gRPC, injecting tenant context (x-tenant-id, x-org-id, x-user-id) as metadata headers.

Runtime itself never calls compute or modifies agent state. Those concerns live in the agents and compute services.

System context

client
  │
  │  POST /v1/sessions/{session_id}/invoke  (HTTP/SSE)
  ▼
runtime-router  (single shared ClusterIP, handles auth + tenant dispatch)
  │
  │  runtime.v1.RuntimeService.Invoke  (gRPC, server-streaming)
  ▼
runtime-t-{tenant_short}  (per-tenant pod)
  │
  │  agents gRPC (x-tenant-id / x-org-id in metadata)
  ▼
agents  (Phase 1.1 — LLM + tool dispatch)
  │
  ▼
compute  (Phase 3 — gVisor sandbox, ExecCommand)

Tenant pod lifecycle

A tenant pod is created on the tenant’s first request and runs continuously until idle. The tenant-lifecycle-controller manages the full lifecycle:

1. First request — EnsureReady creates a Kubernetes Deployment + Service for runtime-t-{tenant_short} and returns immediately once the pod is scheduled.
2. Active — each routed request updates the runtime.paper-board.io/last-seen-at annotation on the Deployment.
3. Idle timeout — after 60 minutes with no requests, IdleSweep (runs every 5 minutes) scales the Deployment to 0. The Deployment and Service are kept; DNS entries remain valid.
4. Cold start — the next request after scale-to-zero triggers a scale-up. The router waits up to 10 seconds for the pod to become ready, then retries the Invoke RPC once.
5. Weekly cleanup — Deployments that have been at 0 replicas for 7 or more days are hard-deleted by WeeklySweep.
6. Suspension — Suspend hard-deletes the Deployment and Service immediately (used for tenant churn or admin action).

Consumers

Two services depend on runtime:

• agents — receives Invoke RPC calls from per-tenant runtime pods; agents’ identity is propagated via gRPC metadata set by the router.
• compute — called by agents during tool dispatch; runtime is not in the compute call path, but the tenant pod’s TENANT_ID env var is what agents uses when creating sandboxes.

Phase shipped

Phase 3 (shipped 2026-05-23). Runtime has no own Postgres schema (ADR-0002: schema-per-service; runtime is a stateless control plane). The agents.usage_events metering table — written by agents — is the only persistent artifact that runtime-routed requests produce.

Phase 7 introduces a gateway service that absorbs the router’s auth and routing responsibilities. Until then, runtime-router is the sole public-facing HTTP entry point.