Enterprise-Scale LLM Platform with Multi-Model Support and Copilot Customization

telus · Telecommunications · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Telecommunications enterprise AI platform

Telus needed a centralized platform for building, governing, and operating custom AI copilots across many teams, channels, models, clouds, and data-residency requirements without locking builders into one model or one application surface.

Constraints:

The platform had to support many internal copilots and tens of thousands of users.
Different use cases need different models, cloud providers, costs, latencies, and data-residency choices.
Canadian telecom use cases can require all data to stay in Canada.
Copilots need configurable system prompts, knowledge bases, guardrails, and access controls.
Tool use and function calling need a planner/executor flow with moderation and validation.

Architecture

FuelEx sits as a centralized management layer above hardware, cloud, and foundation-model providers. FuelEx Core wraps integrations, observation, model orchestration, moderation, validation, vector databases, SSO, APIs, and cloud/model services. FuelEx Apps expose web, Slack, Google Chat, and future Teams/VS Code/GitHub/Chrome surfaces. Custom copilots have system prompts, documents, knowledge bases, model choice, and guardrails. A proxy layer manages multi-cloud model hosting, load balancing, fallback, and retries; a Lab lets builders compare model behavior, cost, and latency before deployment.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  fuelex-core["FuelEx Core"] --> fuelex-apps["FuelEx Apps"]
  fuelex-core --> model-proxy-layer["Model proxy layer"]
  fuelex-core --> vector-database["Vector database"]
  fuelex-core --> responsible-ai-guardrails["Responsible AI guardrails"]
  fuelex-apps --> channel-integrations["Channel integrations"]
  copilot-configuration["Copilot configuration"] --> fuelex-core
  copilot-configuration --> vector-database
  vector-database --> model-proxy-layer
  model-proxy-layer --> planner-executor-tool-runtime["Planner executor tool runtime"]
  model-proxy-layer --> lab-experimentation["Lab experimentation"]
  planner-executor-tool-runtime --> model-proxy-layer
  lab-experimentation --> model-proxy-layer
  responsible-ai-guardrails --> fuelex-core

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`FuelEx Core`	Centralized management layer for integrations, observation, model orchestration, moderation, validation, APIs, and cloud/model services.	`FuelEx Apps`, `Model proxy layer`, `Vector database`, `Responsible AI guardrails`
`FuelEx Apps`	User-facing web and chat applications for copilots.	`FuelEx Core`, `Channel integrations`
`Channel integrations`	Slack, Google Chat, planned Microsoft Teams, VS Code, GitHub Actions, and Chrome extension surfaces.	`FuelEx Apps`
`Copilot configuration`	System prompt, model settings, connected knowledge bases, access controls, and guardrail settings for each copilot.	`FuelEx Core`, `Vector database`
`Vector database`	Stores embedded document content for RAG, including Canada-hosted TurboPuffer options.	`Copilot configuration`, `Model proxy layer`
`Model proxy layer`	Routes requests across Azure OpenAI, AWS Bedrock Claude, GCP, Cohere, and Canada-hosted models with load balancing, fallback, and retries.	`Planner executor tool runtime`, `Lab experimentation`
`Planner executor tool runtime`	Function-calling flow where the model plans tool calls and execution runs internet search, image generation, or other tools.	`Model proxy layer`
`Lab experimentation`	Builder environment for comparing models, prompts, cost, latency, and exporting experiments as code.	`Model proxy layer`
`Responsible AI guardrails`	Education, policy, moderation, validation, and scope/fairness checks for responsible AI deployment.	`FuelEx Core`

Data flow

A builder creates a copilot with a system prompt, optional documents/knowledge bases, model choice, and guardrails. Users access it through a web app or messaging channel. For RAG, uploaded files are emitted into a vector database and retrieved by similarity search. For tool use, a planner sends available tools and the question to the LLM, an executor runs the selected tools, and the results are sent back for final response generation.

“you can create your own system prompt customize that you can add documents and knowledge bases to your co- and then we use the retrieval augmented generation architecture”transcript, stated

“in the planner step we basically send all the available tools to the llm along with the original question that the user asked”transcript, stated

Why this architecture

Telus built FuelEx as a centralized layer because enterprise AI needed one place for model access, governance, guardrails, integrations, observability, and builder experimentation while still letting teams create tailored copilots in their existing tools. The multi-cloud proxy reduces model lock-in and supports data residency, cost, latency, fallback, and retry needs.

“fuel ex um we operate on top of all of that we're the centralized management layer that provides um monitoring and provides that layer of management”transcript, stated

“we have this proxy layer on top that allows you to do load balancing or uh basically any kind of like fall back and retries because we have so many models that we need to manage”transcript, stated

Models used: OpenAI models on Azure · Claude on AWS Bedrock · Google Gemini · Gemini 1.5 Flash · Cohere Command R · DALL-E · TurboPuffer

Scale

FuelEx had more than 30,000 custom copilots built.
“so far we've had more than 30,000 custom co-pilots built on uh the fuel platform”transcript, stated
FuelEx had more than 35,000 active users.
“then we have more than 35,000 active users on the platform so far”transcript, stated
Spock resolved over 46% of bot interactions without switching to human agents.
“we've seen over 40 6% of the interactions with the bot being resolved by the co-pilot itself so we don't switch over to agents”transcript, stated
The Spock rollout coincided with a 21% month-over-month decrease in agent interactions.
“in June uh we saw a 21% decrease in agent interractions month over month which is a huge saving for Talis”transcript, stated
Telus trained more than 500 people on responsible AI education.
“we've put an emphasis on education with over 500 trained data stewards and thousands trained impr prompt engineering”transcript, stated

Alternatives considered (roads not taken)

Lock the platform to one model or cloud provider.

Rejected because: FuelEx uses multiple clouds/models and a proxy layer for switching, fallback, retries, cost, latency, and residency needs.

“we're all in the cloud but we're not in just one Cloud um so we actually have our different models hosted across different uh Cloud providers”transcript, stated

Force users into a new standalone web app.

Rejected because: Telus wanted copilots where employees already work, including Slack and Google Chat.

“we wanted to meet people where they work we don't want you to go to yet another web application to be able to use gen”transcript, stated

Build copilots on top of OpenAI Assistants or another hosted copilot service.

Rejected because: Telus wrote its copilot endpoint from scratch as its own solution.

“we actually write our copilot from our from scratch so um we have a copilot endpoint that is similar to assistance uh with open AI but it's fully um our solution”transcript, stated

Use output guardrails on streaming responses.

Rejected because: The team found streaming output hard to guardrail directly, so guardrail architecture needed special handling.

“one challenge with guard rails has been that when you're streaming the response from the lolm um you can't really have guard rails on the output on the stream response”transcript, stated

Patterns

`model-routing` (platform)

FuelEx routes across models hosted on Azure, Bedrock, GCP, and Canada-hosted providers through a proxy layer with load balancing, fallback, and retries.

“we have open a models hosted on aour we have CLA models um hosted on Bedrock so we have a lot of different um basically Cloud providers”transcript, stated

`rag-retrieval` (knowledge grounding)

Custom copilots attach documents/knowledge bases and retrieve embedded file content through vector similarity search.

“when you upload a file we're emitting the content of that file putting it in a vector database and then when we do a similarity search”transcript, stated

`tool-use-orchestration` (function calling)

Function calling uses a planner/executor architecture where the LLM plans tool calls and the execution step runs them.

“we're using a planner executor architecture here the way it works is that in the planner step we basically send all the available tools to the llm”transcript, stated

`guardrails-io-filtering` (responsible AI)

FuelEx Core includes moderation and validation, with guardrails detecting out-of-scope or unfair responses.

“the core component is that centralized management layer that we talked about all the Integrations all the observation layer um or like orchestration across different models um moderation validation”transcript, stated

`privacy-preserving-processing` (data residency)

For Canadian data residency requirements, FuelEx can use Canada-hosted vector databases and models.

“as a Canadian Telo all the data for certain use cases have to stay in Canada all the time”transcript, stated

Tradeoffs

Build a centralized GenAI management layer.

Chose FuelEx Core plus FuelEx Apps over Separate one-off copilots per team or channel.

Rationale: Centralization gives shared model orchestration, moderation, validation, monitoring, and APIs while allowing many custom apps.

“so fuel is basically constructed from two main parts we have the fuelx apps and Fu core”transcript, stated

Meet users where they work.

Chose Slack, Google Chat, web, and planned Teams/IDE/browser integrations over Only a dedicated FuelEx web application.

Rationale: Using existing messaging and workflow surfaces improves adoption and reuse of the same copilot configuration.

“same system prompt same model same settings because we're building this at a platform”transcript, stated

Support Canada-hosted infrastructure for some use cases.

Chose Canada-hosted vector DB and model options over Defaulting all data to global cloud regions.

Rationale: Canadian telecom data-residency requirements require local hosting for some use cases.

“our vect database is Turbo puffer which is a Canadian company and it's hosted in Canada as well”transcript, stated

Gotchas (what bit them)

Model lock-in conflicts with enterprise experimentation and residency needs.

Root cause: Different copilots need different models, clouds, costs, latencies, and compliance boundaries.

Resolution: Use multi-cloud model hosting and a proxy layer for routing, fallback, retries, and experimentation.

“be able to switch between different models and services and experiment”transcript, stated

Streaming responses complicate output guardrails.

Root cause: Output content is emitted incrementally before full-response checks can run.

Resolution: Treat guardrail design as layered and avoid assuming normal output filtering applies to streaming.

“when you're streaming the response from the lolm um you can't really have guard rails on the output on the stream response”transcript, stated

A web-only AI product creates adoption friction.

Root cause: Employees already work in chat and workflow tools and may resist another destination app.

Resolution: Expose the same copilots in Slack, Google Chat, and planned Teams/VS Code/GitHub/Chrome integrations.

“we don't want you to go to yet another web application to be able to use gen so even though we have our web application still”transcript, stated

Evaluation

FuelEx evaluation uses model-comparison labs, cost and latency measurements, production outcome metrics such as Spock resolution/agent-interaction rates, responsible-AI review, and guardrail validation.

Metrics: Model response quality across panels · Cost · Experiment duration/latency · Spock resolution rate · Agent-interaction reduction · Responsible AI training and review

“we can change our settings we can look at the cost and how long our experiment took place and we can actually build out experiments in our lab”transcript, stated

“the same system prompt might not act the same across models how you might use one instead of the other and um play around with it”transcript, stated

Outcomes

FuelEx had more than 30,000 custom copilots and more than 35,000 active users.
The internal Spock bot resolved over 46% of interactions without switching to agents.
Telus reported a 21% month-over-month decrease in agent interactions in June.

The platform is an enterprise-wide Telus system with tens of thousands of users, many copilots, and model/provider choices across multiple clouds.

“so far we've had more than 30,000 custom co-pilots built on uh the fuel platform those are all the co-pilots internal app tell us that we use”transcript, stated

“in June uh we saw a 21% decrease in agent interractions month over month which is a huge saving for Talis”transcript, stated