Skip to content

Deployment

Relayna Gateway ships as one binary and one Docker image. The image serves both the core proxy and the admin portal because the admin UI is embedded in the gateway-api binary.

Version 0.1.26 keeps that deployment shape and consolidates Entra-authenticated portal sessions, request-plane authorization, and scoped owner monitoring on one confidential Web/API application. Separate managed identities receive gateway.invoke or gateway.monitor.read. The release has no database migration, but it replaces the duplicated Entra audience/client-ID variables with ENTRA_APPLICATION_ID. Existing operator-token access and registered-service traffic remain unchanged. It retains endpoint-level failure monitoring, buffered-body admission, OpenAPI discovery and endpoint billing, persisted service timeout controls, structured terminal timeout responses, the Aurora Teal Admin UI, bearer-prefixed custom LiteLLM credential header values, LiteLLM wildcard passthrough, per-route canonical OpenAI mode selection, direct LiteLLM bearer delegation, trusted-ingress dashboard/admin passthrough, opt-in Microsoft Entra ID authorization, and Apigee gateway patterns. Admin UI 2.0 remains compiled into the same static asset contract on the gateway binary. See Current Feature Highlights, Entra Portal and Service-owner Monitoring, Entra Integration Requirements, OpenAPI Service Pricing, Entra ID Auth, and Apigee Gateway Path for the public feature delta.

Docker Image

Build the image:

docker build -t relayna-gateway:0.1.26 .

Run it with required dependencies:

docker run --rm \
  -p 8080:8080 \
  -p 8081:8081 \
  -e DATABASE_URL="postgres://user:password@host.docker.internal:5432/relayna_gateway" \
  -e REDIS_URL="redis://host.docker.internal:6379" \
  -e LITELLM_BASE_URL="http://host.docker.internal:4000" \
  -e LITELLM_SERVICE_KEY="sk-litellm-service-key" \
  -e GATEWAY_ADMIN_TOKEN="op_live_replace_with_secret_value" \
  -e RELAYNA_STUDIO_BASE_URL="http://host.docker.internal:8000" \
  -e GATEWAY_BIND_ADDR="0.0.0.0:8080" \
  -e GATEWAY_CONTROL_BIND_ADDR="0.0.0.0:8081" \
  -e GATEWAY_MAX_BUFFERED_REQUESTS="8" \
  -e GATEWAY_MAX_INFLIGHT_BUFFER_BYTES="536870912" \
  -e LOG_LEVEL="gateway_api=info,gateway_proxy=info" \
  relayna-gateway:0.1.26

The proxy listens on port 8080. The control API, admin portal, readiness, and metrics listen on port 8081.

Version 0.1.26 reuses the existing portal members, exact service memberships, managed-identity bindings, OIDC login transactions, and opaque portal sessions. Existing operator tokens remain the break-glass bootstrap path and existing service registrations are not granted to members automatically.

The body-admission defaults reserve no memory at startup. They cap concurrent managed body buffering at eight requests and 512 MiB of aggregate serialized body data. Keep this byte budget below the container memory limit to leave headroom for JSON parsing, guardrails, connection pools, and normal runtime state.

GATEWAY_ADMIN_TOKEN is optional. Set it only for the first startup against a fresh database when you want to seed a known op_live_... operator token. Omit it to let Gateway generate and print a one-time operator token. After an active operator token exists in PostgreSQL, env changes are ignored; rotate the token from the Admin portal to change it.

PostgreSQL Container

For local container testing:

docker run --rm --name relayna-postgres \
  -p 5432:5432 \
  -e POSTGRES_USER=relayna_gateway \
  -e POSTGRES_PASSWORD=relayna_gateway \
  -e POSTGRES_DB=relayna_gateway \
  postgres:16

Use:

postgres://relayna_gateway:relayna_gateway@host.docker.internal:5432/relayna_gateway

Redis Container

docker run --rm --name relayna-redis -p 6379:6379 redis:7

Use:

redis://host.docker.internal:6379

Kubernetes

The repository includes a production-hardened example manifest at deploy/kubernetes/relayna-gateway.yaml. It keeps public proxy traffic and the private control plane on separate Services.

  1. Use the image published by the tag-based release workflow:
ghcr.io/sarattha/relayna-gateway:0.1.26

To build and publish manually to another registry:

export RELAYNA_GATEWAY_IMAGE="<your-registry>/<your-org>/relayna-gateway:0.1.26"
docker build -t "$RELAYNA_GATEWAY_IMAGE" .
docker push "$RELAYNA_GATEWAY_IMAGE"
  1. Update the Deployment image when you use a different registry or tag:
image: <your-registry>/<your-org>/relayna-gateway:0.1.26
  1. Store secrets through your cluster secret manager:
kubectl create secret generic relayna-gateway-secrets \
  --from-literal=DATABASE_URL='postgres://user:password@postgres:5432/relayna_gateway' \
  --from-literal=REDIS_URL='redis://redis:6379' \
  --from-literal=LITELLM_BASE_URL='http://litellm:4000' \
  --from-literal=LITELLM_SERVICE_KEY='sk-litellm-service-key' \
  --from-literal=GATEWAY_ADMIN_TOKEN='op_live_replace_with_secret_value' \
  --from-literal=RELAYNA_STUDIO_BASE_URL='http://relayna-studio-backend:8000' \
  --from-literal=RELAYNA_STUDIO_TOKEN='studio-gateway-token' \
  --from-literal=ENTRA_AUTH_ENABLED='false' \
  --from-literal=ENTRA_RELAYNA_KEY_HEADER='X-Relayna-Key' \
  --from-literal=APIGEE_TRUSTED_HEADER_ENABLED='false' \
  --from-literal=APIGEE_TRUSTED_HEADER_SECRET=''

For Entra portal sign-in, also create the dedicated relayna-gateway-portal-oidc Secret with the matching portal-private-key.pem and portal-certificate.pem values. Put portal and owner Entra identifiers, issuer URLs, redirect URLs, and the temporary PORTAL_ADMIN_EMAILS plus PORTAL_ADMIN_OBJECT_IDS allowlists in relayna-gateway-config. See Entra Integration Requirements.

  1. Apply the manifest:
kubectl apply -f deploy/kubernetes/relayna-gateway.yaml
  1. Verify readiness:
kubectl rollout status deployment/relayna-gateway
kubectl port-forward svc/relayna-gateway-control 8081:8081
curl http://127.0.0.1:8081/admin-ui/readyz
  1. Validate the manifest before applying cluster-specific edits:
kubectl apply --dry-run=client -f deploy/kubernetes/relayna-gateway.yaml
  1. For an Entra-enabled deployment, run the read-only raw-manifest verifier:
scripts/entra/verify-deployment.sh \
  --namespace <gateway-namespace> \
  --control-ingress-namespace <internal-ingress-namespace> \
  --certificate-file <approved-public-certificate.pem>

Network Exposure

Expose only relayna-gateway-proxy to clients that need LLM traffic. Keep relayna-gateway-control private or protected by internal ingress, VPN, identity-aware proxy, or strict network policy.

Browser and administrator control-plane paths are rooted under /admin-ui, while workload monitoring is rooted under /owner/v1. An AKS ingress must route both prefixes to the private control Service even when another gateway owns /, /healthz, /readyz, and /metrics. Use /admin-ui/healthz, /admin-ui/readyz, and /admin-ui/metrics for probes and scrapers.

The Kubernetes example uses restricted pod security defaults:

  • non-root UID/GID 10001
  • readOnlyRootFilesystem: true
  • seccompProfile: RuntimeDefault
  • allowPrivilegeEscalation: false
  • all Linux capabilities dropped

The supply-chain work also adds strict CI security checks, release metadata validation, SBOM generation, image signing, provenance, and documented temporary security exceptions. Treat failures in those checks as blocking unless an exception is explicitly documented in Security Exceptions.

The included NetworkPolicy is intentionally restrictive. Replace the example namespace labels, service selectors, and provider egress CIDR with values that match your cluster. The placeholder provider egress block uses documentation CIDR 203.0.113.0/24 and will not reach real providers until you replace it or route provider traffic through an approved egress gateway.

For local development, port-forward the control Service or run the container directly. Do not expose /admin-ui/admin/*, /admin-ui/metrics, or operator token workflows on a public ingress.

Guardrail Configuration

Database migrations create the guardrail catalog, key policy, execution event, and per-key override tables/columns on startup. The built-in pii-redact catalog entry is enabled but not default-on, so existing keys keep unguarded behavior until an operator selects guardrails for the key.

Use Admin portal Guardrails to manage global catalog defaults:

  • runtime_config is the actual default config used when a guardrail runs.
  • config_schema documents the shape operators should use for runtime config.
  • HTTP guardrail endpoint URL, timeout, and bearer token are separate catalog fields. Bearer tokens are write-only and are never returned by the API.

Use Admin portal Keys to manage each key:

  • mandatory_guardrails always run for that key.
  • optional_guardrails may run when the client requests them.
  • forbidden_guardrails are hidden from discovery and rejected if requested.
  • guardrail_config_overrides tunes selected guardrails per key.

Per-key overrides are shallow-merged over the catalog runtime config. They must be JSON objects, and they only take effect when the guardrail is applied by mandatory, optional, default-on, or client-requested policy. For example, one key can restore pii-redact placeholders in responses while another leaves redacted placeholders in the final output:

{
  "guardrail_policy": {
    "mandatory_guardrails": ["pii-redact"],
    "optional_guardrails": [],
    "forbidden_guardrails": [],
    "guardrail_config_overrides": {
      "pii-redact": {
        "restore_output": false
      }
    }
  }
}

When guarded traffic may stream, ensure every selected response guardrail supports during_call; otherwise Gateway fails closed with guardrail_unavailable instead of buffering an unsupported stream.

Studio Import Connectivity

Gateway imports Studio services by calling the Studio backend endpoint GET /studio/gateway/services. The configured Studio base URL should therefore be the backend base URL. RELAYNA_STUDIO_BASE_URL and RELAYNA_STUDIO_TOKEN remain startup fallbacks; operators can override them in Admin portal Settings without restarting Gateway. Clearing the persisted base URL returns Gateway to the environment fallback.

Deployment shape Example value
Gateway and Studio on the same host http://127.0.0.1:8000
Gateway in Docker, Studio on host http://host.docker.internal:8000
Gateway and Studio in Kubernetes http://relayna-studio-backend:8000
Gateway to protected Studio over TLS https://studio.internal.example.com

Test Studio directly:

curl -sS "$RELAYNA_STUDIO_BASE_URL/studio/gateway/services"

Test through Gateway after startup:

curl -sS \
  -H "Authorization: Bearer $GATEWAY_OPERATOR_TOKEN" \
  -X POST \
  http://127.0.0.1:8081/admin-ui/admin/studio/connection/test

curl -sS \
  -H "Authorization: Bearer $GATEWAY_OPERATOR_TOKEN" \
  http://127.0.0.1:8081/admin-ui/admin/studio/services

If Gateway returns studio_unavailable, check that the backend URL is reachable from the Gateway process, that the path /studio/gateway/services exists, that the effective token matches Studio's expected token when authentication is enabled, and that Studio returns valid service names and route patterns.