Agent Registry and Dynamic Prompt Management for AI Feature Development

gitlab · Tech · year n/a · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI feature delivery

GitLab needed to change prompts, parameters, and providers without waiting for application release cycles.

Constraints:

Self-managed customers update slowly
Prompts were hardcoded in Ruby classes
Provider switching should not require app code changes

Architecture

GitLab's agent registry moves AI feature behavior out of hardcoded Ruby classes and into the AI Gateway. The main application calls an AI Gateway endpoint for a use case such as generate description; the Agent Registry resolves a base agent and its available collections; registry configuration stores the agent name, provider, model, prompt templates, and parameters; the gateway then calls the selected provider such as Claude or Vertex/Gemini. Prompt and provider changes can be deployed in the gateway without waiting for the monolith release cycle.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  gitlab-monolith[GitLab monolith] --> ai-gateway-endpoint[AI Gateway endpoint]
  ai-gateway-endpoint --> agent-registry[Agent Registry]
  agent-registry --> base-agent[Base agent]
  agent-registry --> prompt-templates[Prompt templates]
  agent-registry --> provider-configuration[Provider configuration]
  prompt-templates --> llm-provider[LLM provider]
  provider-configuration --> llm-provider
  llm-provider --> generated-description[Generated description]
  generated-description --> ai-gateway-endpoint

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`GitLab monolith`	Release-bound application code that previously held provider and prompt logic.	`AI Gateway endpoint`
`AI Gateway endpoint`	Use-case-specific API endpoint for operations such as generate description.	`GitLab monolith`, `Agent Registry`
`Agent Registry`	Central registry resolving use cases to base agents, providers, prompts, parameters, and collections.	`AI Gateway endpoint`, `Prompt templates`, `Provider configuration`
`Base agent`	Reusable agent definition for a use case such as issue description generation.	`Agent Registry`, `Prompt templates`
`Prompt templates`	Gateway-managed prompts and parameters that can change without a GitLab release.	`Agent Registry`, `LLM provider`
`Provider configuration`	Model/provider selection such as Claude or Vertex/Gemini.	`Agent Registry`, `LLM provider`
`LLM provider`	External or platform model endpoint producing the issue description.	`Prompt templates`, `Provider configuration`, `Generated description`
`Generated description`	AI-created issue description returned to the GitLab user.	`AI Gateway endpoint`

Data flow

A GitLab feature request enters the monolith and calls a use-case-specific AI Gateway endpoint. The gateway asks the Agent Registry for the relevant agent, loads its provider/model/prompt/parameter configuration, sends the request to the chosen LLM provider, and returns the generated issue description. Changing a prompt or provider updates gateway-side registry configuration instead of Ruby application code.

3 supporting quotes

“if we want to make a change to the prompt we need to do it here which means that we need to wait for a new gitlab release to come out with these changes”transcript, implied

“the work that we're doing in uh the agent registry work is moving all of this logic to the AI Gateway uh and implementing a prompt registry which will abstract all of these details the prompts the parameters and even uh which provider model we're using”transcript, stated

“you have now a centralized way to refer to every important detail about the agent behavior that you want to modify so we have the the name the provider which in this case we're using uh cloud and we have all the prompt templates”transcript, stated

Why this architecture

GitLab chose a gateway registry because prompt and provider iteration was blocked by application releases, especially for self-managed customers that update less frequently. Centralizing agent behavior in the gateway lets GitLab update prompts, parameters, and providers transparently while keeping the monolith API stable.

3 supporting quotes

“change to the prompt we need to do it here which means that we need to wait for a new gitlab release to come out with these changes and um in gab.com we deploy uh very often but our self-managed customers don't update to every new version they wait for stable releases they uh release every sorry”transcript, stated

“self-managed customers don't update to every new version they wait for stable releases they uh release every sorry they update every once once in a while so this is a problem because we cannot deliver um from improvements for them at the speed that we would like”transcript, stated

“update all of these things in a way that's completely transparent to all self-managed customers so this is what that looks like at the moment we will have a a an endpoint in the AI Gateway for each specific operation here I've already made one for Generate description”transcript, stated

Models used: Claude · Vertex/Gemini

Scale

The demo switches providers from Claude to Vertex/Gemini through registry configuration.
“we have the the name the provider which in this case we're using uh cloud and we have all the prompt templates and so the advantage with this is that uh we can make changes here and all these changes will be containing the AI Gateway and we can deliver that very rapidly to to users”transcript, stated

Alternatives considered (roads not taken)

Keep prompts and providers hardcoded in Ruby classes

Rejected because: Every change would require waiting for a GitLab release.

“if we want to make a change to the prompt we need to do it here which means that we need to wait for a new gitlab release to come out with these changes and um in gab.com we deploy uh very often but our self-managed customers don't update to every new version they”transcript, stated

Bind behavior directly to one provider

Rejected because: The registry makes provider/model selection configurable for the same agent use case.

“the prompts the parameters and even uh which provider model we're using so that we can uh update all of these things in a way that's completely transparent to all self-managed customers so this is what that looks like at the moment we will have a a an endpoint”transcript, stated

Patterns

`prompt-versioning` (configuration)

Prompts and parameters are centralized outside release-bound application classes.

“the agent registry work is moving all of this logic to the AI Gateway uh and implementing a prompt registry which will abstract all of these details the prompts the parameters and even uh which provider model we're using so that we can uh update all of these things in a way that's completely transp”transcript, stated

`model-routing` (configuration)

Provider/model choice is part of the registry abstraction.

“these details the prompts the parameters and even uh which provider model we're using so that we can uh update all of”transcript, stated

Tradeoffs

Where to manage AI feature prompts.

Chose AI Gateway agent registry over hardcoded monolith Ruby classes.

Rationale: Gateway-side configuration lets prompt/provider changes reach customers faster than release-bound application code.

“we cannot deliver um from improvements for them at the speed that we would like”transcript, stated

Gotchas (what bit them)

Hardcoded prompts slowed AI iteration.

Root cause: Every prompt change required a full GitLab release cycle.

Resolution: The Agent Registry moved prompt configuration into the gateway.

“GitLab faced a challenge with their AI-powered issue description generation feature where prompts were hardcoded in Ruby classes, meaning any prompt changes required a full GitLab release cycle.”summary, stated

Evaluation

The demo validates dynamic prompt/provider behavior rather than reporting numeric metrics.

Outcomes

GitLab can update prompts and switch providers without touching the main application.

The summary emphasizes faster delivery to self-managed and SaaS customers.

“This architecture allows GitLab to update prompts, switch between LLM providers (demonstrated with Claude and Vertex/Gemini), and modify agent behavior without touching the main application codebase, enabling rapid delivery of AI improvements to all customers transparently.”summary, stated