Domain-Adapted Foundation Models for Enterprise-Scale LLM Deployment

linkedin · Tech · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise-scale domain-adapted foundation models

LinkedIn needed lower-cost, lower-latency, domain-specific GenAI models for a 1B+ member platform where proprietary models were strong but expensive, slow, and less tuned to LinkedIn's Economic Graph and professional workflows.

Constraints:

• The platform serves 1B+ members and 69 million companies.
• GPT-4 enabled early experiences but latency and cost were concerns.
• Models needed to follow instructions, generalize to professional tasks, and comply with responsible AI principles.
• Hiring Assistant evaluation calls needed both speed and accuracy at high call volume.
• The model family needed training, inference, and evaluation inside a repeatable pipeline.

Architecture

LinkedIn built EON, a family of domain-adapted foundation models. EON adapts open-source/third-party foundation models with LinkedIn Economic Graph data, multi-task instruction tuning with reasoning traces, prompt simplification, synthetic safe outputs, preference/safety alignment via RLHF and DPO, and an on-prem Kubernetes training pipeline covering preprocessing, training, offline inference, and evaluation. EON-8B then powers key parts of Hiring Assistant's candidate/job evaluation agent.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  linkedin-economic-graph-data["LinkedIn Economic Graph data"] --> training-data-pipeline["Training data pipeline"]
  training-data-pipeline --> multi-task-instruction-tuning["Multi-task instruction tuning"]
  multi-task-instruction-tuning --> preference-and-safety-alignment["Preference and safety alignment"]
  preference-and-safety-alignment --> eon-models["EON models"]
  eon-models --> evaluation-suite["Evaluation suite"]
  eon-models --> hiring-assistant-evaluation-agent["Hiring Assistant evaluation agent"]
  hiring-assistant-evaluation-agent --> recruiter-workflow["Recruiter workflow"]

Components

Data flow

LinkedIn collects high-quality diverse training data, including Economic Graph/domain data and synthetically generated safety examples. The EON pipeline preprocesses data, trains candidate open-source LLMs, runs offline inference, evaluates quality/safety/cost, and deploys the selected EON model into product flows such as Hiring Assistant's evaluation agent.

Why this architecture

EON occupies a middle ground between proprietary frontier models and base open models: it adapts smaller models to LinkedIn's domain to lower cost/latency while preserving instruction-following, safety, and generalization. The pipeline is end-to-end so candidate models can be repeatedly trained, evaluated, and deployed into high-volume product agents.

Models used: EON-8B · EON-70B · Llama 3.1-8B · Llama-3-8B-instruct · GPT-4 · GPT-4o · GPT-4o mini · Mixtral · Mistral · InBart

Scale

• LinkedIn serves 1B+ members and 69 million companies.
  “LinkedIn's 1B+ members and 69 million companies”article, stated
• EON training data totaled around 200M tokens.
  “The high-quality, diverse, and de-duped training data totaled around 200M tokens.”article, stated
• Prompt simplification reduced prompt size by 30%.
  “prompt simplification strategies to standardize complex human-written prompts, which led to a 30% reduction in prompt size.”article, stated
• EON-8B was 75x cost effective versus GPT-4 and 6x versus GPT-4o.
  “We found the EON-8B model (a domain-adapted Llama 3.1-8B variant) to be 75x and 6x cost effective in comparison to GPT-4 and GPT-4o respectively”article, stated
• Nearly 90% of Hiring Assistant LLM calls come from the evaluation agent.
  “Nearly 90% of large language model (LLM) calls in the hiring assistant flow come from the evaluation agent.”article, stated

Alternatives considered (roads not taken)

Use GPT-4 for scaled LinkedIn GenAI features

Rejected because: GPT-4 accelerated early launches but had latency and cost concerns at LinkedIn scale.

Use base open-source models without domain adaptation

Rejected because: EON adaptation improved professional-domain tasks and Hiring Assistant matching beyond base Llama.

Rely only on raw benchmark scores

Rejected because: LinkedIn treats open-source/GPT-judge metrics as directional and uses human evaluation/product benchmarks where needed.

Patterns

fine-tuning-adaptation (model-ops)

LinkedIn domain-adapted open-source foundation models with multi-task instruction tuning and reasoning traces.

model-routing (model-ops)

LinkedIn selected and evaluated multiple SOTA foundation models before domain adapting them for EON.

llm-as-judge (evaluation)

GPT-4 variants judged model quality when reliable reference labels were absent.

multi-agent-orchestration (agent)

Hiring Assistant breaks recruiter queries into multiple steps through an orchestration layer, including an evaluation agent.

Tradeoffs

Invest in EON domain-adapted models

Chose LinkedIn-tuned smaller foundation models over Serving GPT-4/GPT-4o for every use case.

Rationale: Domain adaptation improved cost efficiency and specialized performance at LinkedIn scale.

Use reasoning traces and multi-task instructions

Chose Reasoning-augmented multi-task tuning over Single-task fine-tuning only.

Rationale: EON needed to generalize across many LinkedIn-specific professional tasks.

Standardize prompts

Chose Prompt simplification over Complex human-written prompts.

Rationale: Prompt simplification reduced token size and standardized model behavior.

Gotchas (what bit them)

Proprietary model quality did not remove latency and cost pressure.

Root cause: LinkedIn's platform scale makes GPT-4-style serving expensive and latency-sensitive.

Resolution: Build smaller domain-adapted models that retain quality on professional tasks.

Task-specific domain tuning can harm generalization and safety.

Root cause: Fine-tuning can reduce out-of-box safety and generalization if data is narrow or unbalanced.

Resolution: Use diverse data, reasoning-augmented multi-task instruction tuning, synthetic safety outputs, and alignment steps.

Hiring Assistant's evaluation agent dominates LLM call volume.

Root cause: Candidate-job matching requires parsing extensive candidate and job context at high scale.

Resolution: Use a fast, accurate domain-adapted model for the evaluation agent.

Evaluation

LinkedIn evaluated EON with open-source frameworks, LinkedIn benchmarks, GPT-4 judge variants, safety benchmarks, cost-efficiency measurements, human annotations, and product metrics for Hiring Assistant.

Metrics: ARC, MuSR, IFEval, BFCL · LinkedIn benchmarks · GPT-4-as-judge where labels were absent · Safety compliance · GPU cost efficiency versus GPT-4/GPT-4o · Candidate-job-requirements matching accuracy

Outcomes

• Built EON domain-adapted foundation models for LinkedIn use cases.
• EON-8B achieved 75x cost efficiency compared with GPT-4 and 6x compared with GPT-4o.
• Hiring Assistant evaluation improved candidate-job matching by 4% over GPT-4o mini and 30% over Llama-3-8B-instruct.
• EON supports a roadmap toward more complex language agents and multi-step planning/reasoning.

The case operates at LinkedIn platform scale: 1B+ members, 69M companies, around 200M training tokens, and nearly 90% of Hiring Assistant LLM calls in the evaluation agent.