Enterprise-Wide AI Assistant Deployment for Collective Discovery

prosus · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise-wide generative AI adoption

Prosus needed a way for employees across a large portfolio of companies to discover useful generative AI applications safely, learn by doing, and graduate bottom-up experiments into production products without prescribing all use cases centrally.

Constraints:

• The organization spans many companies, functions, countries, and existing knowledge bases.
• Employees needed a low-friction assistant inside existing workflow tools such as Slack.
• The assistant had to protect sensitive data with no-learning and no-retention policies.
• Hallucinations needed to be visible and reduced through feedback, model improvements, prompting, and user education.
• Use cases needed to emerge from stress testing before being promoted into normal engineering practice.

Architecture

Prosus built Toqan.ai as a general-purpose internal AI assistant first available through Slack and then across the group. It integrates commercial, open-source, and in-house fine-tuned LLMs; image, voice, document, data-analysis, and code tools; and internal knowledge bases for grounded answers. Privacy guardrails prevent model learning or retention of sensitive data. Feedback mechanisms collect positive and negative signals, including Pinocchio hallucination reports, while education and usage analytics help teams discover, stress-test, and graduate use cases into production applications.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  slack-assistant-interface["Slack assistant interface"] --> multi-model-assistant-core["Multi-model assistant core"]
  multi-model-assistant-core --> tool-capability-layer["Tool capability layer"]
  multi-model-assistant-core --> internal-knowledge-bases["Internal knowledge bases"]
  multi-model-assistant-core --> privacy-guardrails["Privacy guardrails"]
  feedback-mechanism["Feedback mechanism"] --> usage-analytics-and-education["Usage analytics and education"]
  usage-analytics-and-education --> production-use-case-pipeline["Production use-case pipeline"]

Components

Data flow

Employees interact with Toqan.ai in Slack or other assistant surfaces, choose from text, code, image, voice, document, or data-analysis capabilities, and receive grounded responses when internal knowledge bases are relevant. Users provide feedback such as thumbs-up or Pinocchio reports, and teams stress-test promising workflows until they are confident enough to move them into normal engineering practice or productized applications.

Why this architecture

Prosus chose a broad internal assistant rather than a top-down use-case program because useful applications emerged organically across portfolio companies. Slack access lowered friction, multi-model/tool support covered many task types, internal knowledge grounding improved relevance, and no-retention/no-learning controls made experimentation safer at enterprise scale.

Models used: Toqan.ai · Commercial LLMs · Open-source LLMs · In-house fine-tuned models · Image generation · Voice generation · Code creation tools

Scale

• Prosus serves one quarter of the global population across over 100 countries.
  “serving one quarter of the global population with our consumer internet services in over 100 countries”article, stated
• The assistant was used by approximately 13,000 colleagues across 24 companies.
  “the assistant is utilized by approximately 13,000 colleagues across 24 companies.”article, stated
• Prosus ran over 20 practical field experiments before the broad assistant launch.
  “Between 2020 and 2021, we conducted over 20 practical field experiments”article, stated
• Toqan integrates over 20 models and tools.
  “for a total of over 20 models and tools.”article, stated
• Engineering tasks account for about 50% of assistant usage.
  “the assistant is utilized for engineering-related tasks in approximately 50% of instances”article, stated
• More than 81% of users reported productivity increases above 5-10%.
  “over 81% of users reporting productivity increases of more than 5%-10%.”article, stated

Alternatives considered (roads not taken)

Run a top-down use-case discovery program.

Rejected because: The strongest applications emerged bottom-up in collaborative channels across companies.

Deploy a narrow single-model chatbot.

Rejected because: Prosus needed many task types and model options, including code, documents, images, voice, and data analysis.

Let experiments stay as informal assistant usage.

Rejected because: Teams stress-tested use cases and then graduated validated ones into production practices and products.

Patterns

model-routing (assistant platform)

Toqan combines commercial, open-source, and in-house fine-tuned models for different assistant capabilities.

rag-retrieval (grounding)

The assistant accesses internal knowledge bases to provide grounded company-specific responses.

privacy-preserving-processing (enterprise trust)

No-learning and no-retention policies protect data from future model training.

eval-driven-development (quality feedback)

The assistant tracks positive and negative feedback, including hallucination-specific Pinocchio reports, to guide improvements.

tool-use-orchestration (assistant evolution)

The assistant is evolving toward selecting agents for web browsing, code execution, API connections, and complex tasks.

Tradeoffs

Deploy a general-purpose assistant for discovery.

Chose Broad Toqan.ai access across employees and companies over A small set of centrally mandated applications.

Rationale: Bottom-up stress testing revealed use cases that could later graduate into production.

Expose many models and tools through one assistant.

Chose Commercial, open-source, fine-tuned, multimodal, document, data, and code capabilities over A single-purpose Q&A bot.

Rationale: The assistant was meant to support exploration across diverse team needs and task types.

Treat bad responses as manageable rather than eliminable.

Chose Feedback, model improvements, prompt education, and oversight over Waiting for perfect hallucination-free models.

Rationale: The article states bad responses cannot be eliminated entirely but can be reduced and managed.

Gotchas (what bit them)

Hallucinations need explicit feedback channels.

Root cause: A broad assistant will produce unreliable or fabricated answers, especially early in deployment.

Resolution: Track Pinocchio feedback, improve models/prompts, and educate users on better prompting.

Central teams cannot predict every high-value use case.

Root cause: Prosus companies found similar and unexpected uses across different functions and contexts.

Resolution: Use a broad assistant to enable collective discovery and let patterns emerge from real usage.

Enterprise adoption requires data-protection trust.

Root cause: Employees need to experiment across sensitive business functions without training future models on their data.

Resolution: Implement no-learning/no-retention guardrails and training programs.

Evaluation

Prosus evaluates adoption and quality through interaction analysis, user interviews, productivity surveys, A/B testing for selected tasks, first-help-resource usage, and explicit positive/negative feedback including Pinocchio hallucination reports.

Metrics: Pinocchio hallucination-feedback rate · Productivity increase reported by users · A/B-tested time reduction · Engineering-task usage share · First-help-resource usage share · Positive and negative feedback

Outcomes

• Over 81% of users reported productivity increases greater than 5-10%.
• Pinocchio hallucination feedback fell from nearly 10% to around 1.5%.
• Teams graduated assistant-tested ideas into products such as Genie, compr.ai, Kodie, and simulation RolePlay.

Prosus deployed Toqan.ai to about 13,000 colleagues across 24 companies, within a group serving one quarter of the global population in over 100 countries.