Kubernetes-Native Secure Execution Platform for Autonomous AI Agents

grab · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Secure execution platform for autonomous AI agents

Grab needed a way for autonomous AI agents to act in real production environments without turning credentials, internal services, network access, or long-running workloads into uncontrolled risk. Prompt-level guardrails were insufficient because agents call tools, tools call services, and services use credentials.

Constraints:

• each agent environment must be isolated by user and agent
• agents must not receive raw upstream LLM credentials or proxy-only secrets
• all network access needs explicit containment and application-aware audit
• browser access, LLM calls, general HTTP/HTTPS egress, Vault access, and Kubernetes API access all cross trust boundaries
• long-running agents can persist for days or weeks and need lifecycle management
• the safe path must be easy enough that teams do not work around it
• agent actions must be attributable, inspectable, revocable, and recoverable

Architecture

Grab built Palana, a Kubernetes-native secure execution platform for autonomous agents. Each agent runs in a per-user, per-agent namespace with its own service account, RBAC, EFS-backed /data volume, ingress, network policies, and lifecycle state. Admission webhooks inject proxy configuration and enforce pod restrictions. LLM calls go through a LiteLLM wrapper that derives identity from Kubernetes context and retrieves upstream credentials from Vault. General egress is forced through a proxy namespace where OPA evaluates host, method, and agent identity. Vault separates agent-readable secrets from proxy-only secrets, so agents can cause approved credentialed requests without reading raw credentials. Observability logs proxy decisions, Git activity, LLM requests, lifecycle events, and idle-shutdown signals so operators and ops agents can audit or intervene.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  pcli_and_portal["pcli and portal"] --> UserAgent_custom_resource["UserAgent custom resource"]
  UserAgent_custom_resource --> Palana_operator["Palana operator"]
  Palana_operator --> Agent_namespace["Agent namespace"]
  Palana_operator --> Admission_webhooks["Admission webhooks"]
  Admission_webhooks --> Agent_pod["Agent pod"]
  Agent_namespace --> Agent_pod
  Agent_pod --> Persistent_data_volume["Persistent data volume"]
  Traefik_browser_ingress["Traefik browser ingress"] --> Agent_pod
  Concedo_OIDC["Concedo OIDC"] --> pcli_and_portal
  Concedo_OIDC --> Traefik_browser_ingress
  Agent_pod --> LiteLLM_gateway_wrapper["LiteLLM gateway wrapper"]
  LiteLLM_gateway_wrapper --> Vault["Vault"]
  LiteLLM_gateway_wrapper --> GrabGPT_upstream["GrabGPT upstream"]
  Agent_pod --> Proxy_namespace["Proxy namespace"]
  NetworkPolicy_and_Cilium["NetworkPolicy and Cilium"] --> Agent_namespace
  NetworkPolicy_and_Cilium --> Proxy_namespace
  Proxy_namespace --> OPA_policy_engine["OPA policy engine"]
  Proxy_namespace --> Vault
  Proxy_namespace --> Observability_systems["Observability systems"]
  LiteLLM_gateway_wrapper --> Observability_systems
  Palana_operator --> Observability_systems
  Observability_systems --> Idle_shutdown_reaper["Idle shutdown reaper"]
  Idle_shutdown_reaper --> Agent_pod
  Idle_shutdown_reaper --> Persistent_data_volume
  Observability_systems --> Ops_assistant_agent["Ops assistant agent"]

Components

Data flow

A user logs in with pcli or the portal, creates an agent, optionally registers secrets, and runs a template or image. Palana writes a UserAgent or Agent custom resource, and the operator creates namespaces, service accounts, role bindings, storage, network policies, ingress, and policy state. Admission webhooks inject proxy variables. Browser traffic enters through Traefik; LLM traffic goes to the LiteLLM wrapper in the gateway namespace; HTTP/HTTPS egress goes through the proxy namespace; Vault credentials are read only by components authorized to use them. Structured logs and activity signals feed observability, idle shutdown, and operational support agents.

Why this architecture

Palana puts enforcement at the platform trust boundaries rather than relying on prompt rules. Per-agent namespaces give a simple boundary for RBAC, storage, network, logging, quota, and lifecycle controls; proxy-only secrets separate credential use from credential access; Layer 3/4 policies contain network paths while Layer 7 proxy policy gives expressive host/method/identity controls and audit; self-service CLI/templates make the secure path easy enough for teams to use.

Models used: LiteLLM · GrabGPT · Claude Code UI template · future SPIFFE/SPIRE agentic identity integration

Scale

• Grab operates across more than 900 cities.
  “Grab is Southeast Asia’s leading superapp, serving over 900 cities across eight countries”article, stated
• Grab operates across eight countries.
  “serving over 900 cities across eight countries (Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Vietnam).”article, stated
• Palana supports agents that persist for days or weeks.
  “Once agents persist for days or weeks, “run a container” becomes an incomplete product.”article, stated
• Palana agent environments include an EFS-backed persistent volume mounted at /data.
  “Storage: an Amazon Elastic File System (EFS)-backed persistent volume claim (PVC) mounted at /data”article, stated

Alternatives considered (roads not taken)

Prompt-level guardrails and model policies only

Rejected because: They do not control tool, service, credential, ingress, egress, Git, or Kubernetes API boundaries.

Give agents raw upstream LLM credentials

Rejected because: Palana routes LLM traffic through a wrapper that derives identity and retrieves credentials centrally.

Let agents read all secrets they can use

Rejected because: Proxy-only secrets let agents make approved credentialed requests without exposing real tokens in filesystem, logs, environment, or prompts.

Use only Kubernetes NetworkPolicy for network control

Rejected because: NetworkPolicy contains traffic but Layer 7 proxy policy is needed for application-aware host, method, identity, and audit decisions.

Stop idle agents and discard runtime state

Rejected because: Palana stops compute but preserves /data, RBAC, namespace, and Vault state for easy resumption.

Patterns

safety-policy-enforcement (governance)

Palana enforces identity, egress, secret, ingress, Git, and Kubernetes API controls at platform trust boundaries rather than relying on model behavior.

tool-use-orchestration (architecture)

Agents can use browsers, LLM gateways, HTTP services, Git, Kubernetes API paths, and credentialed services only through mediated platform routes.

observability-tracing (operations)

Palana treats logs and activity signals as safety infrastructure for audits, idle shutdown, and operational support agents.

agent-memory (architecture)

Persistent /data volumes and preserved namespace/Vault/RBAC state give long-running agents durable local state across shutdown and resumption.

multi-agent-orchestration (architecture)

Palana is evolving toward supervisor systems, agent swarms, Slack-native agents, and operational agents with scoped namespaces and credentials.

Tradeoffs

Use namespace isolation as the unit of trust.

Chose one namespace per user and agent over shared agent runtimes with weaker boundaries.

Rationale: Namespaces provide a consistent place for RBAC, storage, network policies, labels, quotas, lifecycle controls, incident inspection, and isolation.

Separate credential use from credential access.

Chose proxy-only Vault secrets over raw credentials inside agent runtimes.

Rationale: Agents can perform authenticated work through approved proxies without exposing real tokens to process environment, filesystem, logs, or prompts.

Split network control into Layer 3/4 and Layer 7.

Chose NetworkPolicy/Cilium containment plus proxy/OPA application policy over using only one network-control layer.

Rationale: NetworkPolicy contains traffic paths, while the proxy enables host, method, agent identity, and audit decisions.

Stop idle compute but preserve agent state.

Chose idle reaper with preserved /data, RBAC, namespace, and Vault state over letting all long-running agents stay active indefinitely.

Rationale: Long-running agents consume resources and expand monitoring surface, but users still need resumable environments.

Gotchas (what bit them)

Prompt guardrails do not secure real-world agent actions.

Root cause: Agents call tools and services that use credentials outside the model prompt.

Resolution: Put controls at platform trust boundaries such as identity, egress, secrets, ingress, Git, and Kubernetes API access.

Credential exposure through agent runtimes is a high-leverage risk.

Root cause: If agents can read real tokens, filesystem, logs, environment variables, or prompt context can become credential stores.

Resolution: Use proxy-only secrets and placeholder values so agents can cause approved credentialed requests without reading raw credentials.

Sanitized user identifiers can accidentally become authorization subjects.

Root cause: The display-safe or path-safe user ID is not necessarily the raw identity Kubernetes RBAC should match.

Resolution: Keep raw identity as authoritative on the custom resource and use sanitized versions only for names, labels, namespaces, and Vault paths.

Long-running agents turn container launch into an incomplete product.

Root cause: Users need resume semantics while operators need cleanup, audit history, credential rotation, image updates, and external stop controls.

Resolution: Add idle shutdown and lifecycle management that stops compute while preserving state.

A secure path that is too hard invites workarounds.

Root cause: Teams will avoid the platform if they must learn Terraform, Vault policy syntax, Kubernetes RBAC, and proxy configuration before trying an agent.

Resolution: Expose pcli, templates, and a portal so the safe path is the easy path.

Evaluation

The article evaluates Palana through production lessons rather than benchmark metrics: whether platform-layer controls prevent credential/network/identity risks, whether observability supports audit and operations, whether self-service makes secure agent execution practical, and whether lifecycle controls handle long-running agents.

Metrics: proxy allow/deny decisions · Git activity · LLM request activity · agent lifecycle events · idle-shutdown signals · namespace/user/host/component queryability · OOM detection by operational agents

Outcomes

• Palana gives Grab a self-service way to run autonomous agents with isolation, identity, secrets, network access, and auditability as defaults.
• The platform supports Slack-native agents, remote development environments, operational agents, supervisor systems, and agent swarms.
• Proxy-only secrets let agents perform authenticated work without receiving raw credentials.
• Idle shutdown stops compute while preserving state for resumption.
• Operational agents can inspect Palana logs and advise users about workload issues.

The article does not publish Palana workload counts, but it places the platform inside Grab, which serves over 900 cities across eight countries. The architecture is explicitly production-oriented and describes lessons learned from putting the system into production.