Real-Time Multilingual Chat Translation at Scale

roblox · Media & Entertainment · 2024 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Real-time multilingual chat translation at platform scale

Roblox needed real-time chat translation across all combinations of 16 supported languages for a global platform, while meeting conversational latency, domain-specific language quality, and trust-and-safety requirements.

Constraints:

• The system had to translate between any combination of 16 languages, avoiding an explosion of 256 separate models.
• Latency had to be about 100 milliseconds to preserve chat flow.
• The system had to handle more than 5,000 chats per second and more than 70 million daily active users.
• Roblox-specific slang and abbreviations needed domain adaptation beyond generic commercial translation.
• Translated text had to pass through trust and safety controls like other chat text.

Architecture

Roblox built a unified transformer-based multilingual translation LLM with mixture-of-experts behavior for all supported language pairs. The production system checks a request cache, uses dynamic batching on model servers, adds an embedding cache between encoders and decoders, and sends both original and translated messages to devices. Training combines open-source data, Roblox in-experience data, human-labeled chats, slang translations, quality-estimation labels, iterative back translation, distillation, quantization, model compilation, and trust-and-safety integration.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["In-experience text chat"] --> B["Translation API"]
  B --> C["RCC request cache"]
  C --> D["Model server"]
  D --> E["Unified multilingual translation model"]
  D --> F["Embedding cache"]
  B --> G["Trust and safety systems"]
  H["Training data pipeline"] --> E
  H --> I["Translation quality estimation model"]
  I --> E
  J["User feedback tool"] --> H

Components

Data flow

A chat message, source language, and target language pass through RCC. If a translation is cached, the backend is skipped. Otherwise the request goes to backend model servers with dynamic batching. The unified model translates across the requested language pair, embedding cache improves multi-target efficiency, and the API sends original plus translated text to the recipient device. The backend applies additional translation logic and trust/safety screening.

Why this architecture

A unified model avoids operating hundreds of language-pair models and lets related languages share training signal. Serving optimizations, caching, distillation, and quantization make that model fast enough for live chat while trust-and-safety integration keeps translated text within platform policy.

Models used: Unified transformer-based translation LLM · Translation quality estimation model · Multilingual language model

Scale

• Roblox supports direct translation among 16 languages.
  “direct translation between any combination of the 16 languages we currently support”article, stated
• Translations are displayed with approximately 100 millisecond latency.
  “These translations are displayed in real time, with latency of approximately 100 milliseconds”article, stated
• Roblox reports more than 70 million daily active users.
  “Roblox is home to more than 70 million daily active users all over the world and growing.”article, stated
• The platform has more than 15 million active experiences.
  “Manually translating every conversation happening across more than 15 million active experiences”article, stated
• The system needed to support more than 5,000 chats per second.
  “low latency is critical to support more than 5,000 chats per second.”article, stated
• The teacher model had roughly 1 billion parameters and was reduced to fewer than 650 million parameters.
  “The resulting chat translation model has roughly 1 billion parameters.”article, stated

  “reduce the size of the model to fewer than 650 million parameters”article, stated

Alternatives considered (roads not taken)

Build one model per language pair

Rejected because: Sixteen languages would require 256 models and would be harder to train, serve, and maintain.

Serve the roughly 1B-parameter model directly

Rejected because: The large model was too resource-intensive and slow for real-time chat at platform scale.

Use only off-the-shelf translation quality metrics

Rejected because: Roblox wanted reference-free quality assessment across accuracy, fluency, and incorrect references.

Patterns

fine-tuning-adaptation (model-adaptation)

Roblox trained and optimized a domain-specific multilingual translation model using open data, in-experience translations, human labels, slang, and back translation.

cost-aware-routing (operations)

The serving path uses request caching, dynamic batching, embedding caching, distillation, quantization, compilation, and a smaller model for latency/resource efficiency.

eval-driven-development (quality)

Roblox built a custom reference-free translation quality estimation model to classify error types and improve the translation model.

human-in-the-loop-gate (quality)

Human evaluators translate trending/slang terms, and Roblox plans user feedback to flag mistranslations and suggest better translations.

safety-policy-enforcement (safety)

The translation backend integrates with Roblox trust and safety so translated text is screened for policy-violating content.

Tradeoffs

Build a custom unified translation model.

Chose Single multilingual model with expert specialization over Commercial APIs or separate pairwise models.

Rationale: A custom model could optimize for Roblox-specific language, all language pairs, latency, and maintainability.

Send both translated and original messages to devices.

Chose Recipient sees native-language translation with original available over Translated-only chat display.

Rationale: The UI defaults to seamless translation while preserving user access to the sender's original message.

Use reference-free custom quality estimation.

Chose Human-labeled error model over ground-truth comparison only over Off-the-shelf metrics requiring reference translations.

Rationale: Reference translations are impractical at Roblox scale, so the quality model evaluates source and machine translation directly.

Gotchas (what bit them)

A naive per-language-pair model strategy would explode operational complexity.

Root cause: Sixteen supported languages create 256 possible directed language-pair models.

Resolution: Roblox built one unified transformer model with expert specialization for language groups.

Real-time serving makes large high-quality translation models operationally impractical without compression.

Root cause: The roughly 1B-parameter model was too resource-intensive and slow for chat latency and throughput targets.

Resolution: Roblox used student-teacher distillation, quantization, compilation, and serving optimizations to shrink the model below 650M parameters.

Generic translation data misses Roblox-specific slang and platform terms.

Root cause: Terms like obby, afk, and lol are domain-specific and change over time.

Resolution: Human evaluators translated popular/trending terms, and Roblox plans regular updates with in-experience examples and slang.

Evaluation

Roblox uses a custom translation quality estimation model trained on human-labeled error types/scores, internal metrics comparing Roblox content performance, human slang translation, and planned user feedback loops.

Metrics: Translation quality without reference translation · Accuracy: additions, omissions, mistranslations · Fluency: punctuation, spelling, grammar · Incorrect references · Critical/major/minor error severity · Latency around 100 milliseconds · Throughput above 5,000 chats per second

Outcomes

• Roblox deployed real-time AI chat translation across all 16 supported languages.
• The model supports direct language-pair translation with approximately 100 millisecond latency.
• The optimized model serves at platform scale with more than 5,000 chats per second target/support.
• The system integrates translated chat with trust and safety review.
• The architecture sets up future multimodal and voice translation expansion.

Roblox cites 16 supported languages, 256 pairwise combinations avoided, more than 70 million daily active users, more than 15 million active experiences, approximately 100ms latency, more than 5,000 chats per second, roughly 1B-parameter teacher model, and fewer than 650M parameters after optimization.