Deploying AI Coding Agents in Highly Regulated Environments with Secure Infrastructure

ona · Tech · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Secure AI coding agents for regulated enterprises

ONA needed to let finance, government, and similarly regulated organizations use AI coding agents without leaking source code, sensitive data, or policy control to external vendors.

Constraints:

• Regulated customers face legal obligations and strict data protection requirements.
• Banning AI tools creates shadow AI usage and uncontrolled leakage risk.
• Agents need enough freedom to edit, run, and test code while remaining isolated and disposable.
• Customers need audit logs, admin controls, and network restrictions.
• Source code, model traffic, and credentials must stay inside the customer's cloud/network.

Architecture

ONA's platform runs software-engineering agents inside isolated disposable development environments: virtual machines with dev containers, project dependencies, and developer access. For regulated customers, the environments run in the customer's own VPC; the runner pulls tasks, spins up or stops environments, connects directly to Git providers and LLM providers inside the customer's network, and records admin/audit state. ONA keeps only metadata and client-side encrypted private data, avoiding access to source code or sensitive project contents.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  developer["Developer"] --> disposable-development-environment["Disposable development environment"]
  ona-control-plane["ONA control plane"] --> customer-vpc-runner["Customer VPC runner"]
  customer-vpc-runner --> disposable-development-environment["Disposable development environment"]
  disposable-development-environment --> git-provider["Git provider"]
  disposable-development-environment --> llm-provider["LLM provider"]
  disposable-development-environment --> admin-controls-and-audit-logs["Admin controls and audit logs"]
  customer-vpc-runner --> client-side-encrypted-metadata["Client-side encrypted metadata"]

Components

Data flow

A developer or control plane schedules coding-agent work; the customer-side runner pulls environment and agent instructions, creates disposable VM/dev-container environments, and agents work against the customer's Git and model providers directly inside the VPC. Developers can SSH or connect via IDEs, while admin controls, audit logs, and client-side encryption constrain and record activity without routing source code through ONA.

Why this architecture

The key design is to put both agent execution and external integrations inside the customer's network. Disposable VM environments give agents room to edit and run code, while VPC deployment, pull-based runners, admin controls, audit logs, and client-side encryption address the regulated-enterprise trust boundary.

Models used: Anthropic · AWS Bedrock · Vertex AI · Software engineering agents (provider-flexible)

Scale

• The source cites a survey where 45% of employees admitted using banned AI tools.
  “45% of the people confessed that they use already banned tools banned AI tools in their company.”transcript, stated
• The source cites 58% of surveyed employees sending sensitive data to public AI services.
  “58% said that they have um sent company uh well sensitive company or client data into chat GPD or Gemini.”transcript, stated
• The source cites 40% of employees willing to violate policy if AI makes work faster.
  “40% just claim that they will use the AI tool even knowing that is against company policy if that is going to make their task quicker.”transcript, stated

Alternatives considered (roads not taken)

Ban AI tools entirely

Rejected because: The talk argues that employees will use AI anyway, creating worse shadow-AI risk.

Manual approval for each AI tool or department

Rejected because: The speaker calls this cumbersome and does not recommend it.

Use ChatGPT as the company-wide AI solution

Rejected because: Copy-pasting code into a generic chat tool does not fit developer workflows or regulated data boundaries.

Route source code or model traffic through ONA

Rejected because: Regulated customers need direct network connections to Git and model providers with no vendor man-in-the-middle.

Patterns

privacy-preserving-processing (security)

Coding agents run in the customer's VPC and connect to customer Git/model providers without ONA seeing code.

tool-use-orchestration (agent)

Agents operate in configured development environments with access to Git providers, LLM providers, IDE/SSH workflows, and network controls.

guardrails-io-filtering (security)

Admin controls constrain what agents may execute and what networks they may access.

observability-tracing (operations)

Audit logs track agent and environment activity for regulated enterprise review.

Tradeoffs

Place execution in customer VPCs

Chose Customer-owned VPC runners and environments over Vendor-hosted default execution.

Rationale: The VPC design gives regulated customers data residency and network control while still allowing the agent platform to coordinate work.

Use disposable isolated environments

Chose Independent VM/dev-container environments per agent/work item over Shared persistent developer machines.

Rationale: Disposable isolation allows parallel work and safe recovery when an agent breaks the environment.

Let customers choose model providers

Chose Direct customer-network connections to Anthropic, Bedrock, Vertex, or other providers over A single ONA-managed LLM backend.

Rationale: Provider flexibility lets regulated customers align model traffic with their own security, procurement, and compliance constraints.

Gotchas (what bit them)

Banning AI can increase rather than reduce risk.

Root cause: Employees may use prohibited public tools anyway when they improve productivity.

Resolution: Provide a secure regulated workflow instead of relying on policy-only bans.

AI coding agents need destructive freedom inside a safe blast radius.

Root cause: Agents must edit and run code, which can break an environment.

Resolution: Run agents in isolated disposable VMs that can be discarded and recreated.

Inbound vendor access is a poor fit for sensitive customer networks.

Root cause: Opening ports or accepting pushed instructions can conflict with regulated security postures.

Resolution: Use a customer-side runner that pulls information and creates environments from within the customer's cloud.

Evaluation

The talk emphasizes security architecture, auditability, and customer-network isolation rather than a formal benchmark. The operational checks are whether data stays in the customer's network, environments remain disposable/isolated, and activity is auditable.

Metrics: Audit-log coverage · No vendor access to customer code · Client-side encryption for private stored data · Disposable environment recovery

Outcomes

• Created a regulated-environment architecture for AI software engineering agents.
• Enabled agents to run in parallel inside disposable VM/dev-container environments.
• Kept source code and model communications inside customer-controlled networks.

The case is framed around regulated sectors such as finance and government, with survey evidence showing substantial shadow-AI use when organizations rely on bans.