Building an Agentic DevOps Copilot for Infrastructure Automation

qovery · Tech · 2025 · source Agentic: yes · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: DevOps infrastructure automation

Qovery wanted a DevOps copilot that could understand infrastructure intent, automate deployments and configuration work, and move beyond brittle one-intent-one-tool mappings.

Constraints:

• Users ask open-ended infrastructure questions and multi-step requests.
• Tools must have clear versioned stateless interfaces.
• Tool chaining is fragile unless intermediate state and validation are tracked.
• Planning latency and compliance-friendly model hosting remain production concerns.

Architecture

Qovery evolved its copilot through four phases: hardcoded intent-to-tool mapping, dynamic agentic planning over tool invocations, resilience/retry/replanning layers, and conversation memory for session continuity. The alpha system uses Claude Sonnet 3.7, QDrant, stateless tools, validation between steps, and memory for context.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  developer[Developer] --> devops-copilot[DevOps Copilot]
  devops-copilot --> planner[Planner]
  planner --> tool-registry[Tool Registry]
  tool-registry --> infrastructure-apis[Infrastructure APIs]
  tool-registry --> resilience-layer[Resilience Layer]
  resilience-layer --> devops-copilot
  devops-copilot --> conversation-memory[Conversation Memory]
  conversation-memory --> vector-store[Vector Store]

Components

Data flow

A user asks an infrastructure question; the copilot analyzes intent, plans a sequence of versioned stateless tool calls, validates outputs between steps, replans or retries on failure, and uses conversation memory to maintain continuity across the session.

Why this architecture

Qovery moved from predictable hardcoded mappings to dynamic planning because real user questions required flexible tool chains, then added recovery and memory because dynamic agents fail and need context.

Models used: Claude Sonnet 3.7 · QDrant

Scale

• Complex task planning can currently take up to 10 seconds.
  “* Improving the speed of planning: It can take up to 10 seconds to plan complex tasks, which is ok in a testing phase, but not so great for production.”article, stated
• The copilot is currently in Alpha.
  “Our Copilot is in Alpha. You can ask it: * “Generate usage stats over the last 30 days for team X.” * “Optimize this Dockerfile.” * “Stop all environments inactive for 6h and notify the team.””article, stated

Alternatives considered (roads not taken)

Hardcoded intent-to-tool mapping.

Rejected because: It was predictable but inflexible and required every new intent to be manually mapped.

“We started with a simple agent architecture: detect a user’s intent, and map that intent to a tool or action. ”article, stated

“* Every new intent had to be hardcoded * Complex workflows required chaining tools manually * No flexibility for unexpected or unplanned user requests”article, stated

Agentic planning without recovery.

Rejected because: If one tool failed or produced unexpected output, the whole plan broke.

Sessionless copilot.

Rejected because: Conversation memory improved references, continuity, and deeper multi-step work.

“”article, stated

“This drastically improved user experience - and opened the door to deeper, multi-step optimization and monitoring tasks.”article, stated

Patterns

tool-use-orchestration (devops-agent)

The copilot dynamically plans a sequence of tool invocations over stateless tools with clear interfaces.

"Each tool:

• Has a clear interface (input/output)
• Is versioned and stateless"

— article, stated

fallback-degradation (reliability)

The resilience layer analyzes failures, fixes or updates the plan, and retries with a corrected approach.

"Analyzes the failure

• Updates its plan or fixes the step
• Retries with a corrected approach"

— article, stated

agent-memory (context)

Conversation memory lets the copilot reuse prior answers and maintain continuity in the session.

"It allows the Agentic DevOps Copilot to:

• Reuse previous answers
• Understand references and context
• Maintain continuity across a session"

— article, stated

latency-optimization (runtime)

Planning speed is a named next improvement because 10-second planning is acceptable for testing but not production.

Tradeoffs

Planner design

Chose Dynamic agentic tool planning over Hardcoded intent mapping.

Rationale: Dynamic planning supports unanticipated user needs and multi-step workflows.

"Far more scalable and flexible

• Can solve unanticipated user needs
• Encourages clean tool abstraction"

— article, stated

Reliability layer

Chose Retry and replanning around tools over Failing the whole workflow on tool mismatch.

Rationale: Multi-step infrastructure workflows need graceful recovery.

Gotchas (what bit them)

Intent-to-tool mapping does not survive real user variety.

Root cause: Users ask unexpected optimization and debugging questions outside pre-mapped intents.

Resolution: Switch to an agentic planner that chooses tool sequences dynamically.

Tool chaining is fragile.

Root cause: Outputs must match expected inputs across steps, and a single failure can break the plan.

Resolution: Track intermediate state, validate between steps, and allow replanning.

Model hosting choices affect enterprise compliance.

Root cause: Some business users want model options that fit their compliance standards.

Resolution: Plan self-hosted model options in addition to Claude Sonnet 3.7.

Evaluation

Qovery evaluates by iterating through alpha usage, tool interface behavior, validation between steps, planning latency, and successful completion of previously unanticipated workflows.

Metrics: Planning latency · Tool validation success · Retry/replan recovery · Multi-step workflow completion · User experience continuity

Outcomes

• The copilot moved from basic intent mapping to dynamic planning, resilience, and memory.
• Resilience layers enabled successful multi-step workflows that were not anticipated during development.
• Conversation memory improved user experience and opened deeper monitoring and optimization tasks.

The current alpha uses Claude Sonnet 3.7, cites planning up to 10 seconds for complex tasks, and references QDrant for vector database support.