Pre-training and Deploying Small Language Models for Edge Devices

liquid_ai · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Small language model training and edge deployment

Liquid AI needed to build language models that could run on phones, cars, and other edge devices where memory, latency, privacy, and connectivity constraints make normal large-model deployment patterns impractical.

Constraints:

• Edge models are memory-bound by device hardware and therefore have lower knowledge capacity than large server-side models.
• The useful production target is narrow task performance, such as data extraction and tool use, rather than broad general-purpose chat.
• On-device inference is latency-sensitive and requires high throughput on real target hardware.
• Small reasoning models can fall into doom loops on complex tasks unless post-training explicitly addresses repetition.
• Deployment choices must handle offline, privacy-regulated, and long-context limitations without assuming remote large-model access.

Architecture

Liquid AI treats edge models as a separate production architecture rather than scaled-down large models. The stack combines an LFM2 hybrid architecture with short-convolution blocks and grouped-query attention, real-device profiling on CPU and mobile hardware, extended pretraining on 28 trillion tokens, task-specific SFT, preference alignment, reinforcement learning with verifiable rewards, and deployment patterns that add tools or recursive language-model environments to compensate for low knowledge capacity and short context.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  target-edge-hardware["Target edge hardware"] --> on-device-profiling-loop["On-device profiling loop"]
  on-device-profiling-loop["On-device profiling loop"] --> target-edge-hardware["Target edge hardware"]
  on-device-profiling-loop["On-device profiling loop"] --> lfm2-edge-model["LFM2 edge model"]
  lfm2-edge-model["LFM2 edge model"] --> on-device-profiling-loop["On-device profiling loop"]
  lfm2-edge-model["LFM2 edge model"] --> pretraining-corpus["Pretraining corpus"]
  pretraining-corpus["Pretraining corpus"] --> lfm2-edge-model["LFM2 edge model"]
  lfm2-edge-model["LFM2 edge model"] --> post-training-pipeline["Post-training pipeline"]
  post-training-pipeline["Post-training pipeline"] --> lfm2-edge-model["LFM2 edge model"]
  post-training-pipeline["Post-training pipeline"] --> doom-loop-mitigation["Doom-loop mitigation"]
  doom-loop-mitigation["Doom-loop mitigation"] --> post-training-pipeline["Post-training pipeline"]
  lfm2-edge-model["LFM2 edge model"] --> tool-access-layer["Tool access layer"]
  tool-access-layer["Tool access layer"] --> lfm2-edge-model["LFM2 edge model"]

Components

Data flow

A target edge use case starts with hardware constraints such as phone or in-car deployment. Liquid profiles candidate model operators on those devices, chooses an efficient architecture with short convolutions and a smaller embedding share, pretrains the small model on a very large token budget, narrows SFT around the intended tasks, then uses preference alignment and RL to remove doom-loop behavior. At deployment time, the small model runs locally for low latency or privacy-sensitive workloads and can call tools such as web search or Python-backed recursive environments when its memorized knowledge or context window is insufficient.

Why this architecture

Liquid chose an edge-specific architecture because small models are not merely reduced large models: their memory footprint, task focus, latency needs, and doom-loop failure modes require dedicated architecture profiling, narrow training, and external tool support.

Models used: LFM 2.5 350M · LFM 2.5 1.2B thinking model · LFM2 architecture · LLM jury · Qwen 3.5 0.8B · Gemma 3 270M

Scale

• Liquid's model family spans roughly 350 million to 24 billion parameters.
  “we mostly focus on edge models for on device deployment. And as you can see here we have models from 350 million parameters to 24 billion parameters.”transcript, stated
• Gemma 3 270M allocates about 63% of parameters to embeddings, limiting effective parameter use.
  “Gemma 3 270M is mostly an embedding layer. It's 63% of the total parameters. And even Gemma 2.5 0.8B it's it's still like 29% of the parameters.”transcript, stated
• LFM2 reduces embedding overhead so roughly 90% of parameters are effective parameters.
  “the embedding layer is actually a lot smaller compared to the others. It's like 90% of the parameters so we have more effective parameters which is great.”transcript, stated
• The 350M model was pre-trained on 28 trillion tokens.
  “we pre-trained a 350 million parameter model on 28 trillion tokens.”transcript, stated
• Preference-alignment data generation starts from roughly one million prompts and six rollouts per prompt.
  “we start with prompt like 1 million samples to give you a rough idea, and then we use the policy model, the model that we want to train with temperature sampling, and we just generate five rollouts.”transcript, stated
• The LFM 2.5 1.2B thinking model had roughly 15-16% doom-loop ratio after pretraining, while Qwen 3.5 0.8B reasoning mode was cited as over 50%.
  “after pre-training, the doom loop ratio that we calculated across like a lot of benchmarks was about 15%. Um or even 16%.”transcript, stated

  “Gwen 3.5 0.8B in reasoning mode, you will see a lot a lot a lot of doom loops, like over 50% of doom loops”transcript, stated

Alternatives considered (roads not taken)

Treat edge models as scaled-down versions of bigger models

Rejected because: Liquid argues that edge models have unique memory, latency, task-focus, and failure-mode constraints that require a specialized stack.

Keep large teacher vocabularies and embedding-heavy student models

Rejected because: Large embedding layers consume memory without adding the same reasoning or knowledge capacity as effective parameters.

Choose architecture operators from theory alone

Rejected because: Liquid found the useful operators by implementing and profiling them on the actual CPU and mobile hardware targets.

Use broad supervised fine-tuning for general chat behavior

Rejected because: The presenter says small-model SFT works best when it is narrow and focused on the production task.

Rely on SFT alone to remove doom looping

Rejected because: The measured doom-loop ratio barely moved after SFT, while preference alignment and RL substantially reduced it.

Patterns

latency-optimization (deployment)

Liquid optimized architecture operators for edge inference by profiling short-convolution and attention alternatives directly on target hardware.

fine-tuning-adaptation (model-adaptation)

The post-training stack uses narrow SFT, preference alignment, and reinforcement learning around the specific edge tasks the model will perform.

tool-use-orchestration (agent-architecture)

The deployment strategy equips small models with tools such as web search so they can compensate for limited memorized knowledge.

eval-driven-development (quality)

The training process uses benchmarks, doom-loop ratios, LLM jury scoring, and RL reward signals to decide whether training stages are actually improving production behavior.

context-engineering (context-management)

The case treats long-context limits as a deployment-environment design problem, proposing recursive language-model environments and Python shortcuts instead of relying only on the base model.

data-quality-loop (model-adaptation)

RL failures are used as feedback to add missing cold-start examples to the SFT mixture before retrying the later training stage.

Tradeoffs

Optimize small models for targeted edge tasks rather than general-purpose chat.

Chose Narrow task specialization for data extraction, tool use, and similar edge capabilities over A broad general-purpose chatbot objective.

Rationale: Small models have limited knowledge capacity, so task focus lets them be excellent at specific production uses instead of average across everything.

Train far beyond older compute-optimal guidance for a 350M model.

Chose 28 trillion-token pretraining for LFM 2.5 350M over Stopping near Chinchilla-style compute optimality.

Rationale: Liquid observed continued improvement with more pretraining tokens and small models are cheap enough to make extended training practical.

Use real-device profiling to choose model operators.

Chose Gated short convolution blocks identified through target-hardware profiling over Architecture selection based on theoretical appeal or inherited large-model blocks.

Rationale: The production objective was latency and memory on actual edge hardware, where short convolutions measured faster than alternatives.

Use tools and recursive environments to compensate for small-model limits.

Chose Agentic deployment with web search, reliable tool use, and Python-backed recursive environments over Expecting the local model to memorize facts and handle long context unaided.

Rationale: External tools and environment design let edge models handle knowledge and context limits while preserving local latency and privacy benefits.

Deploy small local models when edge constraints dominate.

Chose Local small models for no-internet, latency-sensitive, and privacy-regulated workflows over Remote large-model calls for every workload.

Rationale: In-car, latency-sensitive, finance, and healthcare contexts make local execution more suitable than depending on remote large models.

Gotchas (what bit them)

Embedding-heavy distilled models waste scarce edge memory.

Root cause: Small student models inherited huge teacher vocabularies, so a large share of their parameters sat in embeddings rather than effective reasoning capacity.

Resolution: Liquid reduced embedding share in LFM2 so more of the same memory footprint contributes to effective parameters.

Architectural choices that look good theoretically may not be fastest on real edge devices.

Root cause: The production bottleneck is actual target hardware throughput and memory behavior, not just model-family design.

Resolution: Liquid implemented and profiled operators on CPUs and phones before choosing gated short convolution blocks.

Small reasoning models can repeat indefinitely on tasks beyond their capability.

Root cause: Doom looping is especially likely when small model size, reasoning mode, and complex tasks coincide.

Resolution: Preference alignment rejects bad repetitive rollouts, and RL with verifiable rewards plus n-gram penalties further suppresses the behavior.

SFT alone did not remove doom-loop behavior.

Root cause: The failure mode was not adequately represented or corrected by supervised examples alone.

Resolution: Liquid used DPO and reinforcement learning stages where measured doom-loop ratios dropped sharply.

Knowledge and context limitations become hallucination risks if the model is used as a closed book.

Root cause: Memory-bound small models store less knowledge and have weaker long-context capacity than large models.

Resolution: The deployment strategy adds external tools and recursive environments so the model can retrieve facts and offload state.

Evaluation

Liquid evaluates the models through production-relevant benchmarks for knowledge, instruction following, data extraction, and tool use; real-device profiling on CPUs and mobile hardware; and doom-loop ratios across training stages.

Metrics: GPQA Diamond knowledge benchmark · IFEval instruction-following benchmark · Case report data-extraction benchmark · BFCL and Tau2Bench tool-use benchmarks · Throughput and memory on AMD Ryzen Max Plus 395 and Samsung Galaxy S25 Ultra · Doom-loop ratio after pretraining, SFT, DPO, and RL

Outcomes

• LFM2 models achieved higher throughput and lower memory usage than comparable models on profiled CPU and phone targets.
• LFM 2.5 350M improved over the previous LFM2 350M across knowledge, instruction-following, data-extraction, and tool-use benchmarks.
• Preference alignment and reinforcement learning reduced doom looping from roughly 15-16% after pretraining to nearly nonexistent after RL.
• The deployment strategy positioned small edge models as useful for offline, latency-sensitive, and privacy-regulated agentic workloads.

The case spans models from 350M to 24B parameters, a 350M model trained on 28T tokens, profiling on AMD Ryzen Max Plus 395 and Samsung Galaxy S25 Ultra, and doom-loop comparisons involving LFM 2.5 1.2B and Qwen 3.5 0.8B.