API Reference
API Reference
Section titled “API Reference”Overview
Section titled “Overview”Runtime exposes two surfaces:
| Surface | Protocol | Entry point |
|---|---|---|
| Public invocation | HTTP/SSE | POST /v1/sessions/{session_id}/invoke via runtime-router |
| Internal gRPC | gRPC + Protobuf | runtime.v1.RuntimeService on port 50051 |
The HTTP surface is the only surface clients call directly. The gRPC surface is internal — called by runtime-router after routing to the correct tenant pod.
Proto source: paper-board/proto/paperboard/runtime/v1/runtime.proto
HTTP: invoke session
Section titled “HTTP: invoke session”POST /v1/sessions/{session_id}/invoke
Section titled “POST /v1/sessions/{session_id}/invoke”Initiates or continues an agent session. Responses are Server-Sent Events (SSE).
Path parameters
| Parameter | Type | Description |
|---|---|---|
session_id | string (UUID) | The agent session identifier, created by the agents service. |
Request headers
| Header | Required | Description |
|---|---|---|
Authorization | Yes | Bearer <JWT> — identity-issued JWT; router calls identity.AuthService.VerifyJWT to extract tenant_id, org_id, user_id. |
X-Request-Id | Recommended | Caller-supplied request identifier; forwarded as x-request-id gRPC metadata to the tenant pod. |
X-Trace-Id | Recommended | Distributed trace identifier; forwarded as x-trace-id gRPC metadata. |
Response
200 OK — Content-Type: text/event-stream
The response body is a stream of SSE events. Three event types are emitted:
| SSE event | data shape | Description |
|---|---|---|
| (default) | <text chunk string> | Partial assistant text. Embedded newlines are escaped as \n. |
turn_done | {} | The LLM turn completed. No more text chunks for this invocation. |
error | {"message": "<description>"} | A non-recoverable error occurred mid-stream. |
401 Unauthorized — missing or invalid JWT.
503 Service Unavailable — tenant pod unavailable after cold-start retry. Retry-After: 5 seconds.
Cold-start behavior
If the tenant Deployment is scaled to 0 (idle), the router calls EnsureReady, waits up to 10 seconds for the pod to be reachable, then retries the Invoke RPC once. If the retry fails, the router returns 503.
gRPC: runtime.v1.RuntimeService
Section titled “gRPC: runtime.v1.RuntimeService”Invoke
Section titled “Invoke”rpc Invoke(InvokeRequest) returns (stream InvokeEvent);Server-streaming RPC. Called by runtime-router on the per-tenant pod. The tenant pod forwards the request to the agents service, injecting tenant context as gRPC metadata.
InvokeRequest
Section titled “InvokeRequest”| Field | Type | Description |
|---|---|---|
session_id | string | Agent session identifier. |
request_id | string | Request identifier (echoed from X-Request-Id header). |
trace_id | string | Distributed trace identifier (echoed from X-Trace-Id header). |
Tenant context (tenant_id, org_id, user_id) is NOT part of this request message. It is propagated via gRPC incoming metadata headers:
| Metadata key | Set by | Value |
|---|---|---|
x-tenant-id | runtime-router | Tenant UUID from JWT claims |
x-org-id | runtime-router | Org UUID from JWT claims |
x-user-id | runtime-router | User UUID from JWT claims |
x-request-id | runtime-router | Request identifier |
x-trace-id | runtime-router | Trace identifier |
The tenant pod reads TENANT_ID and ORG_ID from its pod environment (injected at creation time by the controller); the metadata values carry the per-request user identity.
InvokeEvent
Section titled “InvokeEvent”| Field | Oneof type | Description |
|---|---|---|
text_chunk | TextChunkEvent | Partial assistant text from the LLM. |
turn_done | TurnDoneEvent | The current LLM turn has completed. |
error | ErrorEvent | Non-recoverable error; stream ends after this event. |
TextChunkEvent.content is a UTF-8 string. ErrorEvent.message is a human-readable description; do not parse it programmatically.
Status codes
| gRPC code | Meaning |
|---|---|
OK | Stream completed normally. |
UNIMPLEMENTED | Invoke is not yet wired to agents (pending S12). |
UNAUTHENTICATED | Metadata missing required tenant headers. |
INTERNAL | Agents call failed. |
Health
Section titled “Health”rpc Check(HealthCheckRequest) returns (HealthCheckResponse);Standard gRPC health protocol (grpc.health.v1). Each tenant pod registers runtime.v1.RuntimeService with SERVING status on startup.
grpc_health_probe -addr=<pod-ip>:50051 -service=runtime.v1.RuntimeServiceTenant pod lifecycle — controller API (internal)
Section titled “Tenant pod lifecycle — controller API (internal)”The controller is not a gRPC service. It is an in-process library called by cmd/controller/main.go. The methods below are the public Go API; they are summarized here for reference.
EnsureReady(ctx, tenantID, orgID) → (deployName, scaleState, error)
Section titled “EnsureReady(ctx, tenantID, orgID) → (deployName, scaleState, error)”Idempotent. Ensures the tenant Deployment exists at replicas ≥ 1.
scaleState | Meaning |
|---|---|
running | Pod was already running; no action taken. |
scaled-up | Deployment existed at replicas=0; scaled to 1. |
created | New Deployment + Service created. |
On conflict between concurrent callers (K8s AlreadyExists), returns running. Retries on K8s transient errors (Conflict, ServerTimeout, TooManyRequests) with exponential backoff: 100ms / 200ms / 400ms / 800ms / 1.6s. Returns DEADLINE_EXCEEDED after 5 failed attempts.
IdleSweep(ctx)
Section titled “IdleSweep(ctx)”Scales Deployments to 0 when runtime.paper-board.io/last-seen-at is older than 60 minutes. Called every 5 minutes by the sweep loop.
WeeklySweep(ctx)
Section titled “WeeklySweep(ctx)”Deletes Deployments (and their Services) that have been at replicas=0 for ≥ 7 days. Called daily.
Suspend(ctx, tenantID) → error
Section titled “Suspend(ctx, tenantID) → error”Hard-deletes the tenant Deployment and Service. Not recoverable; the next request will re-create them via EnsureReady.
UpdateLastSeen(ctx, deployName) → error
Section titled “UpdateLastSeen(ctx, deployName) → error”Patches runtime.paper-board.io/last-seen-at on the Deployment using a strategic merge patch. Called by the router after each successful Invoke forward.
Naming and discovery
Section titled “Naming and discovery”| Resource | Name pattern | Example |
|---|---|---|
| Deployment | runtime-t-{tenant_short} | runtime-t-a1b7558f |
| Service | runtime-t-{tenant_short} (same as Deployment) | runtime-t-a1b7558f |
| DNS | {deploy-name}.paper-board.svc.cluster.local:50051 | runtime-t-a1b7558f.paper-board.svc.cluster.local:50051 |
Tenant short-id derivation
tenant_short = lower(hex(tenant_uuid)[0:8])
Example: a1b7558f-d0ed-9de1-0aaa-... → a1b7558f
Collision policy
On hash prefix collision between two different tenants, the controller appends -2, -3, … until a free slot is found. The resolved short-id is stored in the Deployment’s runtime.paper-board.io/tenant-short annotation. All subsequent lookups for that tenant read the stored annotation rather than recomputing.
Annotations and labels
Section titled “Annotations and labels”| Key | Set on | Set by | Value |
|---|---|---|---|
runtime.paper-board.io/tenant-short | Deployment | controller | Resolved short-id (e.g. a1b7558f) |
runtime.paper-board.io/tenant-id | Deployment, Service | controller | Full tenant UUID |
runtime.paper-board.io/org-id | Deployment | controller | Org UUID |
runtime.paper-board.io/last-seen-at | Deployment | router (via UpdateLastSeen) | RFC3339 timestamp of last routed request |
runtime.paper-board.io/tenant | Deployment (label) | controller | true — label selector for sweep operations |
app.kubernetes.io/managed-by | Deployment (label) | controller | runtime-controller |