Scaling Trust and Safety Using LLMs at Tinder

tinder · Tech · year n/a · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: trust and safety text classification

Tinder needed to detect many trust-and-safety violations in high-volume realtime profile interactions while adversarial behavior changed quickly enough to degrade traditional rules and models.

Constraints:

• Profile interactions happen at huge realtime volume, making direct API calls to frontier LLMs too costly and slow.
• Violation types are numerous and adversarial, including hate speech, promotion, catfishing, and scams.
• Training data has to resemble Tinder's own distribution rather than generic synthetic examples.
• Production serving needs confidence scores, maintainability, and fast adapter updates when behavior shifts.

Architecture

Tinder used large API LLMs such as GPT-4 to mine and label likely violation examples from internal analytics data, manually verified ambiguous mined labels, fine-tuned open-source LLMs with LoRA/QLoRA adapters, and served thousands of small adapters on a base model through LoRAX so each trust-and-safety violation type could be deployed with low marginal serving cost.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Internal analytics data"] --> B["Heuristic candidate gate"]
  A --> C["GPT-4 mining workflow"]
  B --> C
  C --> D["Manual verification"]
  D --> E["Fine-tuning pipelines"]
  E --> F["LoRA adapter store"]
  F --> G["LoRAX adapter serving"]
  B --> G
  G --> H["Trust and safety enforcement"]

Components

Data flow

Internal analytics data is gated by heuristics toward likely candidates, GPT-4-style models mine and label examples, human reviewers verify or fix labels, fine-tuning pipelines create LoRA adapters on open-source base models, and LoRAX serves many adapters on shared GPU-backed base models for realtime trust-and-safety violation detection.

Why this architecture

Tinder used frontier models for data mining because they could bootstrap hard examples, but used fine-tuned open-source models and LoRA adapter serving in production because realtime scale, maintainability, confidence scoring, and fast adaptation mattered more than raw API-model convenience.

Models used: GPT-4 · Llama 3 · Mistral · LoRA adapters · QLoRA adapters · 7B parameter models

Scale

• Tinder has huge realtime profile-interaction volume.
  “Tinder has a huge real-time volume of profile interactions”transcript, stated
• LoRAX can serve thousands of fine-tuned models on a single GPU.
  “allows users to efficiently serve thousands of fine-tuned models on a single GPU.”transcript, stated
• Tinder supported tens of QPS with roughly 100 ms latency on 7B models using A10 GPUs.
  “We can support on 7 billion parameter models uh tens of QPS on 100-ish milliseconds of latency on A10 GPUs.”transcript, stated
• Fine-tuning development could accelerate from months to weeks.
  “we can actually develop accelerate the model development process from months to weeks.”transcript, stated

Alternatives considered (roads not taken)

Directly call GPT-4/API LLMs in production detection

Rejected because: Realtime Tinder volume did not scale in cost, latency, or throughput with external API calls.

Use purely synthetic few-shot training examples

Rejected because: Synthetic examples risked not matching Tinder's true platform distribution.

Full fine-tunes of smaller base models

Rejected because: PEFT/LoRA on larger base models was likely better for classification and cheaper to serve.

Horizontally scale separate models for every violation type

Rejected because: LoRAX made marginal adapter serving virtually free on the same base model and small GPU set.

Patterns

fine-tuning-adaptation (model-adaptation)

Tinder fine-tuned open-source LLMs with LoRA and QLoRA adapters for many trust-and-safety classification tasks.

human-in-the-loop-gate (data-quality)

GPT-4 mined likely examples, but humans verified labels, fixed mislabels, and judged ambiguous trust-and-safety cases before training.

safety-policy-enforcement (safety)

The model family detects and enforces against many safety violations such as hate speech, promotion, catfishing, and scams.

model-routing (serving)

Tinder used heuristics to restrict expensive LLM mining calls and reduce throughput for high-frequency production domains.

Tradeoffs

Use GPT-4 for data mining but not production moderation.

Chose GPT-4-assisted mining plus fine-tuned local production models over direct GPT-4/API moderation in production.

Rationale: The API path helped bootstrap data, but Tinder's realtime production volume required lower cost, latency, and throughput control.

Adopt LoRA/QLoRA adapter fine-tuning.

Chose parameter-efficient adapters on larger base models over full fine-tunes of smaller models.

Rationale: Adapters are small, quick to train, work on limited GPUs, and serve well with LoRAX.

Use LoRAX for many violation models.

Chose joint adapter serving on shared GPU-backed base models over dedicated horizontally scaled deployments per violation type.

Rationale: The marginal cost of another adapter on the same base model is virtually zero.

Gotchas (what bit them)

Generic synthetic data could miss Tinder's true harm distribution.

Root cause: Few-shot synthetic examples are platform-agnostic and may not resemble real internal data.

Resolution: Mine internal analytics data with GPT-4 and then manually verify labels.

Production performance degrades as bad actors adapt.

Root cause: Trust-and-safety is adversarial; blocked spam waves incentivize attackers to change behavior.

Resolution: Use LLM generalization and maintain fine-tuning control so adapters can be updated as production behavior shifts.

Adapter sprawl could become a serving problem.

Root cause: There are many violation types, each potentially needing its own fine-tuned model behavior.

Resolution: Serve many megabyte-scale LoRA adapters jointly with LoRAX on a shared base model.

Evaluation

The talk describes production suitability in terms of realtime throughput, latency, confidence outputs, performance degradation, and manual verification of mined training labels.

Metrics: tens of QPS · 100-ish ms latency · confidence/probability distribution · model degradation curve · manual verification of ambiguous labels

Outcomes

• Fine-tuned open-source LLMs could beat few-shot GPT-4 on some downstream text-detection tasks.
• LoRAX enabled thousands of fine-tuned adapters on one GPU.
• Tinder could support tens of QPS with about 100 ms latency on 7B models and A10 GPUs for some use cases.

The system targeted huge realtime profile-interaction volume and many trust-and-safety violation types.