Fine-tuned LLM Deployment for Automotive Customer Engagement

impel · Automotive · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: automotive customer engagement

Impel needed better accuracy, customization, security, scaling control, and cost predictability for its Sales AI product, whose third-party LLM provider had per-token costs and limited fine-tuning despite automotive-specific customer interactions.

Constraints:

Sales AI needed to support summarization, follow-up generation, and response personalization across dealership communications.
The prior third-party LLM had cost pressure as transaction volumes grew.
Fine-tuning limitations prevented full use of Impel's proprietary automotive data.
Production deployment needed automatic scaling for variable inference demand.
The model needed secure processing within Impel's AWS account.

Architecture

Impel fine-tuned a Meta Llama model for automotive retail tasks using SageMaker Studio notebooks on ml.p4de.24xlarge instances, LoRA fine-tuning, PyTorch, and torchtune. It optimized the model with Activation-Aware Weight Quantization, deployed inference through SageMaker LMI containers on ml.g6e.12xlarge endpoints with four NVIDIA GPUs, and used SageMaker automatic scaling to serve Sales AI features such as summarization, personalized replies, and follow-up generation.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  sales-ai-application["Sales AI application"] --> fine-tuned-meta-llama-model["Fine-tuned Meta Llama model"]
  sagemaker-studio-training-environment["SageMaker Studio training environment"] --> fine-tuned-meta-llama-model
  fine-tuned-meta-llama-model --> awq-optimization-layer["AWQ optimization layer"]
  awq-optimization-layer --> sagemaker-lmi-endpoint["SageMaker LMI endpoint"]
  sagemaker-lmi-endpoint --> automatic-scaling-policy["Automatic scaling policy"]
  sagemaker-lmi-endpoint --> sales-ai-application
  human-evaluation-process["Human evaluation process"] --> fine-tuned-meta-llama-model
  human-evaluation-process --> sales-ai-application

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Sales AI application`	Serves dealership customer engagement through summarization, follow-up generation, and response personalization.	`Fine-tuned Meta Llama model`
`SageMaker Studio training environment`	Runs managed notebooks for LoRA fine-tuning with PyTorch and torchtune.	`Fine-tuned Meta Llama model`
`Fine-tuned Meta Llama model`	Provides automotive-domain language generation tailored to Sales AI.	`SageMaker Studio training environment`, `AWQ optimization layer`, `Sales AI application`
`AWQ optimization layer`	Reduces model size and improves inference performance before deployment.	`Fine-tuned Meta Llama model`, `SageMaker LMI endpoint`
`SageMaker LMI endpoint`	Hosts optimized large-model inference on GPU-backed ml.g6e.12xlarge instances.	`AWQ optimization layer`, `Automatic scaling policy`, `Sales AI application`
`Automatic scaling policy`	Scales serving containers based on concurrent requests to meet variable production demand.	`SageMaker LMI endpoint`
`Human evaluation process`	Grades incumbent and fine-tuned model outputs across core Sales AI use cases.	`Fine-tuned Meta Llama model`, `Sales AI application`

Data flow

Automotive customer interaction data is anonymized and used to fine-tune a public foundation model. The optimized model is quantized with AWQ, packaged through SageMaker LMI containers, deployed on GPU-backed SageMaker endpoints, and called by Sales AI to summarize engagements, personalize dealership responses, and generate follow-ups while automatic scaling adjusts serving capacity.

“Using anonymized customer interaction data, Impel fine-tuned a publicly available foundation model, resulting in several key improvements.”article, stated

“In production, Impel deployed inference endpoints on ml.g6e.12xlarge instances, powered by four NVIDIA GPUs and high memory capacity”article, stated

Why this architecture

Impel chose SageMaker-hosted fine-tuning because domain-specific automotive data improved output quality while hosted pricing, in-account processing, quantization, LMI containers, and autoscaling addressed cost, security, operational control, and production traffic variability.

“Two key factors drove Impel to transition from their existing LLM provider: the need for model customization and cost optimization at scale.”article, stated

“By deploying a fine-tuned Meta Llama model on SageMaker, Impel achieved the following:”article, stated

Models used: Meta Llama · LoRA · Activation-Aware Weight Quantization · SageMaker LMI containers · PyTorch · torchtune · Amazon SageMaker AI

Scale

Sales AI operates all day through email and text customer engagement.
“Sales AI uses generative AI to provide instant responses around the clock to prospective customers through email and text.”article, stated
Training used ml.p4de.24xlarge instances.
“notebooks running on ml.p4de.24xlarge instances.”article, stated
Production endpoints used ml.g6e.12xlarge instances with four NVIDIA GPUs.
“ml.g6e.12xlarge instances, powered by four NVIDIA GPUs and high memory capacity”article, stated
The fine-tuned model improved personalized replies from 73% to 86%, summarization from 70% to 83%, and follow-up generation from 59% to 92%.
“personalized replies improved from 73% to 86% accuracy, conversation summarization increased from 70% to 83%, and follow-up message generation showed the most significant gain, jumping from 59% to 92% accuracy.”article, stated

Alternatives considered (roads not taken)

Continue using the third-party LLM provider.

Rejected because: Per-token pricing became cost-prohibitive and customization limits prevented fuller use of proprietary data.

“Their previous solution’s per-token pricing model became cost-prohibitive as transaction volumes grew, and limitations on fine-tuning prevented them from fully using their proprietary data”article, stated

Use a general-purpose LLM without domain adaptation.

Rejected because: The fine-tuned model improved automotive terminology understanding and Sales AI feature accuracy.

“noticeable improvements in understanding automotive-specific terminology and generating personalized responses.”article, stated

Deploy without managed autoscaling.

Rejected because: Production traffic was variable and needed scaling based on concurrent requests while optimizing cost.

“automatically scale serving containers based on concurrent requests, which helped meet variable production traffic demands while optimizing for cost.”article, stated

Patterns

`fine-tuning-adaptation` (domain-specific automotive LLM)

Impel fine-tuned Meta Llama on anonymized automotive customer interactions to improve Sales AI accuracy.

“Using anonymized customer interaction data, Impel fine-tuned a publicly available foundation model”article, stated

`cost-aware-routing` (hosted inference economics)

Impel moved from per-token third-party costs to hosted pricing and SageMaker scaling to improve cost predictability.

“Cost predictability through hosted pricing, mitigating per-token charges”article, stated

`latency-optimization` (production LLM inference)

The team used AWQ, LMI containers, endpoint sizing, and latency/throughput tests to validate production serving.

“For model optimization, Impel applied AWQ techniques to reduce model size and improve inference performance.”article, stated

`safety-policy-enforcement` (proprietary data security)

Processing proprietary data inside Impel's AWS account reduced reliance on external APIs and improved security control.

“Enhanced data security through in-house processing within their AWS accounts”article, stated

Tradeoffs

Self-host a fine-tuned domain model.

Chose Fine-tuned Meta Llama on SageMaker over Third-party general LLM provider.

Rationale: The SageMaker approach improved customization, cost predictability, security, and operational control.

“Greater control of model training and customization, leading to 20% improvement across core features”article, stated

Use LoRA and AWQ rather than full heavyweight serving.

Chose LoRA fine-tuning plus AWQ optimization over Unoptimized large-model deployment.

Rationale: LoRA adapts the model cost-effectively and AWQ reduces model size while improving inference performance.

“Impel used LoRA fine-tuning, an efficient and cost-effective technique to adapt LLMs for specialized applications”article, stated

“For model optimization, Impel applied AWQ techniques to reduce model size and improve inference performance.”article, stated

Coordinate with AWS teams across sprints.

Chose Joint model evaluation, benchmarking, scaling, and instance selection over Isolated deployment by Impel alone.

Rationale: The work required technical, cost, scaling, and operational alignment for launch.

“collaborated over multiple sprints leading up to the fine-tuned Sales AI launch date to review model evaluations, benchmark SageMaker performance, optimize scaling strategies, and identify the optimal SageMaker instances.”article, stated

Gotchas (what bit them)

Per-token pricing became hard to control as volumes grew.

Root cause: The prior third-party model charged by token and Sales AI transaction volumes increased.

Resolution: Impel moved to SageMaker-hosted pricing and autoscaled endpoints.

“per-token pricing model became cost-prohibitive as transaction volumes grew”article, stated

The previous provider limited model customization.

Root cause: Impel could not fully use proprietary automotive customer data for model improvement.

Resolution: They fine-tuned a Meta Llama model on anonymized customer interaction data.

“limitations on fine-tuning prevented them from fully using their proprietary data for model improvement.”article, stated

Production readiness needed latency and throughput validation beyond quality metrics.

Root cause: A customer engagement product must sustain real production experience, not just benchmark output quality.

Resolution: The team measured latency and throughput with SigV4-signed HTTP requests before production.

“Impel measured latency and throughput to validate the model’s production readiness.”article, stated

Evaluation

Impel used structured human evaluations across core Sales AI use cases, compared incumbent and fine-tuned outputs, and validated latency and throughput for production readiness.

Metrics: 20% improvement across core features · Personalized replies from 73% to 86% · Conversation summarization from 70% to 83% · Follow-up generation from 59% to 92% · Latency · Throughput

“Structured human evaluations revealed enhancements in critical customer interaction areas”article, stated

“Using awscurl for SigV4-signed HTTP requests, the team confirmed these improvements in real-world performance metrics”article, stated

Outcomes

Impel reported 20% improved accuracy across core Sales AI features.
Personalized replies, summarization, and follow-up generation all improved versus the incumbent provider.
The deployment improved security, reduced third-party dependency, and gave Impel more operational control.

The article reports all-day customer engagement, variable production traffic, ml.g6e.12xlarge endpoints with four NVIDIA GPUs, and automatic scaling based on concurrent requests.

“The new model exhibited a 20% increase in accuracy across core features”article, stated

“resulting in 20% improved accuracy across core features including response personalization, conversation summarization, and follow-up generation, while achieving better security and operational control.”summary, stated