Plus One: Internal LLM Platform for Cross-Company AI Adoption

prosus · Tech · 2023 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Internal generative AI adoption platform for a global technology group

Prosus needed a way for teams across many portfolio companies, languages, and domains to discover useful generative AI use cases, test them safely in existing workflows, manage hallucinations and costs, and decide when to use commercial models, RAG, fine-tuning, or agentic execution.

Constraints:

• Prosus serves about two billion users across education, food delivery, groceries, finance, and other sectors.
• Teams needed a shared way to experiment across many LLM and generative model capabilities.
• The platform had to support both engineering and non-engineering users.
• Hallucinations were unacceptable for sensitive contexts such as education and internal knowledge queries.
• Deploying at Prosus scale required cost, quality, and latency routing across model endpoints.
• Non-English languages and code create tokenization cost penalties.
• Agentic workflows were promising but brittle, prompt-sensitive, model-sensitive, and non-deterministic.

Architecture

Prosus built Plus One as a Slack-based internal AI assistant that aggregates large language models and other generative models behind one social interface. Users can request image generation, audio transcription, code review, documentation, summarization of internal Google Docs, data enrichment, and analysis. The system collects feedback through reactions, including a dedicated hallucination signal, routes requests across models based on cost, quality, latency, and capacity, uses RAG for internal knowledge such as Confluence, supports structured JSON outputs for downstream use cases, and keeps answers visible to teams for social quality control. Prosus also experimented with agents that execute Python analysis, but treats production agents as early and brittle.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Slack_Plus_One_interface["Slack Plus One interface"] --> Model_routing_layer["Model routing layer"]
  Slack_Plus_One_interface --> Feedback_reactions["Feedback reactions"]
  Model_routing_layer --> LLM_backends["LLM backends"]
  Model_routing_layer --> Generative_media_models["Generative media models"]
  Model_routing_layer --> RAG_pipeline["RAG pipeline"]
  Model_routing_layer --> Cost_monitoring["Cost monitoring"]
  RAG_pipeline --> Internal_knowledge_bases["Internal knowledge bases"]
  Feedback_reactions --> Hallucination_monitoring["Hallucination monitoring"]
  Feedback_reactions --> Model_routing_layer
  Hallucination_monitoring --> Prompt_and_RAG_tuning["Prompt and RAG tuning"]
  Hallucination_monitoring --> Backend_evaluations["Backend evaluations"]
  Prompt_and_RAG_tuning --> RAG_pipeline
  Prompt_and_RAG_tuning --> Model_routing_layer
  Python_analysis_agent["Python analysis agent"] --> Model_routing_layer
  Social_quality_control["Social quality control"] --> Slack_Plus_One_interface
  Social_quality_control --> Feedback_reactions

Components

Data flow

A user asks Plus One in Slack; the platform chooses an appropriate model or generative backend, optionally retrieves internal knowledge through a RAG pipeline, builds a prompt with relevant context and attribution, returns the answer in the team channel, collects emoji feedback including hallucination reports, and uses that feedback plus backend evaluations to improve prompts, routing, RAG settings, and model selection. For agentic data analysis, uploaded files can flow into a Python execution environment before charts or analysis are returned.

Why this architecture

Prosus chose a Slack-based shared platform to make experimentation visible, low-friction, and socially reviewable across portfolio companies. Aggregating models behind Plus One let teams discover real use cases before committing to fine-tuning or productization, while feedback, RAG, routing, and cost monitoring made the same interface useful for production lessons about hallucination, affordability, and scale.

Models used: large language models · generative models · open-source image generation models · audio transcription models · commercial models · fine-tuned models · smaller language models · Python execution environment

Scale

• Prosus portfolio companies serve about two billion users.
  “across the group we serve about two billion users through the companies”transcript, stated
• Prosus had been exploring generative AI use cases for about 12 to 18 months.
  “about you know maybe 12 to 18 months ago when we were seeing that these models were becoming”transcript, stated
• Thousands of users had experimented with Plus One.
  “thousands of users have been experimenting with plus one”transcript, stated
• Users asked more than half a million questions to Plus One in roughly a year.
  “we've had as I mentioned thousands of users over half a million questions asked to plus one in the last year or so”transcript, stated
• Prosus saw about a 50/50 split between engineering and non-engineering Plus One usage.
  “about a 50/50 split between engineering and non-engineering”transcript, stated
• Initial hallucination feedback was around one in ten answers.
  “we started with one out of 10 or so answers uh you know fell into that uh category”transcript, stated
• The reported hallucination rate later fell comfortably below 2%.
  “actually now this is a stat from June we're uh you know comfortably below 2%”transcript, stated
• Prosus operates in almost 100 countries.
  “we are working in many different languages almost a 100 countries across our uh uh group”transcript, stated
• The presenter says Spanish required about double the tokens compared with English in the shown example.
  “for Spanish in this case you're paying double the tokens um uh and therefore double the money”transcript, stated
• The presenter says Python required more than double the tokens compared with English prose for the same character count.
  “the same number of characters in Python are more than double”transcript, stated

Alternatives considered (roads not taken)

Start every use case by fine-tuning proprietary models

Rejected because: Prosus found teams could discover value faster with prompt engineering, model selection, and commercial models before taking on fine-tuning and hosting complexity.

Use one default model endpoint for all requests

Rejected because: Capacity, latency, cost, and quality vary by task, so Plus One needs routing across model endpoints.

Treat RAG as a general hallucination cure-all

Rejected because: Prosus found RAG is hard, context-specific, and sensitive to retrieval strategy, embeddings, hybrid search, chunking, prompt construction, and attribution.

Trust individual LLM answers in isolation

Rejected because: The team put Plus One in Slack so answers are visible to colleagues and can be corrected through social quality control.

Rush agentic workflows into production

Rejected because: Prosus found agents exciting when they work but brittle, non-deterministic, prompt-sensitive, and model-update-sensitive.

Patterns

model-routing (model-ops)

Plus One routes requests across multiple model endpoints and capabilities by task, capacity, quality, latency, and cost.

cost-aware-routing (cost)

Prosus monitors task costs and routes to smaller, cheaper, or more efficient models and prompts when they meet quality and latency needs.

rag-retrieval (retrieval)

Plus One connects models to internal knowledge bases such as Confluence and uses retrieval, indexing, prompt injection, and attribution to reduce hallucinations.

observability-tracing (operations)

Prosus collects user reactions, hallucination labels, backend evaluations, model/version behavior, and cost histories to monitor platform quality.

human-in-the-loop-gate (control)

Plus One deliberately operates in Slack and existing quality processes so team members, subject matter experts, teachers, or content teams can review and correct outputs.

structured-output-contract (output-control)

Users discovered business use cases by asking Plus One to transform answers, such as dish ingredients, into JSON suitable for downstream enrichment workflows.

prompt-versioning (prompt-ops)

Prosus uses prompt engineering and dynamic prompt construction to reduce hallucinations and adapt RAG context to specific use cases.

tool-use-orchestration (agent-architecture)

The experimental data-analysis agent orchestrates file upload, Python execution, chart generation, and analysis through Plus One.

Tradeoffs

Put Plus One in Slack.

Chose shared Slack workspace interaction over private one-user AI tools.

Rationale: Slack lowers adoption friction and creates social quality control because teammates can see, critique, and correct outputs.

Use model routing instead of a single best model.

Chose routing by cost, quality, latency, and capacity over one model endpoint for all tasks.

Rationale: Different tasks and capacity states require different tradeoffs, and latency can become infinite when capacity is unavailable.

Use commercial/general models to discover use cases before fine-tuning.

Chose prompting, model choice, and general models first over collecting custom datasets and fine-tuning immediately.

Rationale: General models can reveal valuable tasks quickly and avoid the hosting and operational complications of fine-tuned models until value is proven.

Balance flashy innovation with low-hanging operational use cases.

Chose both candy and vegetables over only novel demo-heavy use cases.

Rationale: Existing problems such as data enrichment may deliver immediate value even if they are less flashy than new conversational experiences.

Use humans and existing review processes where answers affect users.

Chose team review, subject matter experts, teachers, and content curators over fully automated user-facing answer publication.

Rationale: Prosus works in education and other sensitive domains where quality expectations already depend on human review.

Gotchas (what bit them)

Hallucinations were common enough to need a dedicated feedback path.

Root cause: Models answered questions about internal information, recent facts, or general factual details they could not reliably know.

Resolution: Prosus added a Pinocchio hallucination reaction, categorized hallucination causes, tuned prompts, and connected models to internal knowledge through RAG.

RAG pipelines are difficult to generalize across content types.

Root cause: Retrieval strategy, embeddings, hybrid search, chunk size, prompt construction, and attribution vary by documentation, code, books, papers, Jira cards, and other formats.

Resolution: Tune the RAG pipeline per context and treat retrieval/prompt hyperparameters as use-case-sensitive rather than universal.

Global language and code workloads create hidden token cost penalties.

Root cause: Tokenizers trained primarily on English can use far more tokens for non-English text or code with the same character count.

Resolution: Account for tokenization economics when choosing models and deploying across languages or code-heavy tasks.

Agentic execution compounded errors and produced inconsistent user experiences.

Root cause: Agents combine non-deterministic model behavior with multi-step execution, prompt sensitivity, and model-update sensitivity.

Resolution: Keep agents exploratory, use them where the value is compelling, and avoid overclaiming production reliability until consistency improves.

Successful prototypes can hide production cost and impact problems.

Root cause: At large-company scale, a working POC still has to be cost-effective and move business metrics.

Resolution: Evaluate affordability, routing, and metric impact before productizing flashy generative AI use cases.

Evaluation

Prosus evaluates Plus One through Slack reaction feedback, a dedicated hallucination emoji, model/version comparisons by task type, backend evaluations for known failure buckets, productivity studies, cost tracking, and production-quality review by humans or subject matter experts.

Metrics: Thumbs-up feedback · Thumbs-down feedback · Heart feedback · Pinocchio hallucination feedback · Hallucination rate from about 10% to below 2% · Model/version performance by task type · Productivity and speed studies · Cost per task · Latency and capacity behavior · Token counts by language and code · Backend evaluations for hallucination buckets

Outcomes

• Thousands of employees used Plus One and asked more than half a million questions in roughly a year.
• Prosus observed a roughly even split between engineering and non-engineering usage.
• The platform helped non-engineers perform technical tasks and engineers perform writing/documentation tasks.
• Hallucination feedback decreased from roughly one in ten answers to below 2%.
• The team identified internal knowledge RAG, model routing, tokenization economics, and agent brittleness as reusable lessons for portfolio companies.
• Plus One helped discover production use cases in education, food delivery, conversational ordering, Stack Overflow-style Q&A, and data enrichment.

Prosus is a global group serving about two billion users, and Plus One was used by thousands of internal users with more than half a million questions in about one year.