Expert-in-the-Loop Generative AI for Marketing Content and Product Descriptions

stitch_fix · E-commerce · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Marketing copy and product-description generation

Stitch Fix needed to scale marketing headlines and product descriptions while preserving the quality, brand voice, product truthfulness, and human nuance required for client-facing fashion content.

Constraints:

• Copywriters traditionally had to write new headlines for every ad asset.
• Product detail pages need accurate, detailed, compelling descriptions that build trust and support SEO.
• Inventory-scale PDP descriptions span hundreds of thousands of styles.
• Generic few-shot prompting produced limited-quality product descriptions.
• Natural language tone and sentiment require human expertise.
• Generated copy must sound natural, align with brand guidelines, and make truthful product statements.

Architecture

Stitch Fix used a two-track expert-in-the-loop system. Ad headlines combine latent style understanding, word embeddings, style keyword selection, GPT-3 few-shot generation, and copywriter review. Product descriptions use expert-written prompt/completion examples to fine-tune a pre-trained language model, then experts evaluate and improve outputs through QA and feedback loops.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  outfit-image-and-style-input["Outfit image and style input"] --> latent-style-and-embedding-layer["Latent style and embedding layer"]
  latent-style-and-embedding-layer --> gpt-3-few-shot-headline-generator["GPT-3 few-shot headline generator"]
  gpt-3-few-shot-headline-generator --> human-copywriter-gate["Human copywriter gate"]
  product-attributes["Product attributes"] --> expert-example-dataset["Expert example dataset"]
  expert-example-dataset --> fine-tuning-job["Fine-tuning job"]
  fine-tuning-job --> fine-tuned-product-description-model["Fine-tuned product-description model"]
  product-attributes --> fine-tuned-product-description-model
  fine-tuned-product-description-model --> generated-pdp-descriptions["Generated PDP descriptions"]
  generated-pdp-descriptions --> human-copywriter-gate
  generated-pdp-descriptions --> blind-quality-evaluation["Blind quality evaluation"]
  blind-quality-evaluation --> qa-feedback-loop["QA feedback loop"]
  human-copywriter-gate --> qa-feedback-loop
  qa-feedback-loop --> fine-tuning-job

Components

Data flow

For headlines, outfit images and style keywords are embedded into latent style space, nearest style terms feed GPT-3, and copywriters edit generated headlines. For PDP descriptions, product attributes become prompts, expert-written descriptions become completions, a base model is fine-tuned, generated descriptions are blind-evaluated, and expert feedback informs future iterations.

Why this architecture

The architecture matches task complexity: few-shot GPT-3 plus human editing works for short ad headlines, while product descriptions require task-specific fine-tuning because they are longer, more factual, more scalable, and more tightly bound to brand voice and product attributes.

Models used: GPT-3 · Few-shot learning · Latent style understanding · Word embeddings · Fine-tuned pre-trained language model

Scale

• The headline system has generated all ad headlines for Facebook and Instagram campaigns.
  “used the expert-in-the-loop approach for generating all ad headlines for Facebook and Instagram campaigns”article, stated
• Product descriptions were needed for hundreds of thousands of inventory styles.
  “Writing descriptions for hundreds of thousands of styles in inventory is a daunting task for human copywriters alone”article, stated
• The task-specific dataset was built from several hundred expert-written product descriptions.
  “human experts write several hundred high-quality product descriptions”article, stated

Alternatives considered (roads not taken)

Manual copywriting as the main scaling strategy.

Rejected because: Writing new ad copy and PDP descriptions manually was too time-consuming and costly at Stitch Fix's content scale.

Few-shot GPT-3 for product descriptions.

Rejected because: The few-shot strategy that worked for headlines generated generic, limited-quality product descriptions.

Fully automated generation without experts.

Rejected because: Human experts were needed to judge nuance, expression, brand fit, appropriateness, and truthfulness.

Patterns

human-in-the-loop-gate (copy quality assurance)

Human copywriters review and edit generated headlines and define what high-quality product-description output should look like.

fine-tuning-adaptation (product-description generation)

Stitch Fix fine-tuned a base language model using expert-written product descriptions and product attributes.

eval-driven-development (blind content evaluation)

The team compared algo-generated descriptions with written descriptions in a blind evaluation and used regular QA checks to improve output.

data-quality-loop (expert feedback)

Expert insights about wording and brand fit feed back into ongoing algorithm and output improvements.

Tradeoffs

How to accelerate marketing-copy production.

Chose AI-generated candidates with copywriter edits over Human writers starting every asset from scratch.

Rationale: The system saves copywriter time while keeping expert control over style and brand alignment.

How to customize PDP descriptions.

Chose Fine-tune on expert-written prompt/completion examples over Generic base model with few-shot prompting alone.

Rationale: Fine-tuning captured Stitch Fix's language, style, and template while retaining scalability.

How to balance scalability and client-facing quality.

Chose Expert-in-the-loop algorithmic generation over Purely algorithmic generation or purely human generation.

Rationale: The hybrid approach combines algorithmic efficiency with human quality judgment.

Gotchas (what bit them)

Ad-headline generation is not the same problem as product-description generation.

Root cause: Descriptions require detailed, truthful, product-specific copy, while headlines can be shorter and style-driven.

Resolution: Use few-shot GPT-3 for headlines but fine-tune on expert-written examples for product descriptions.

Generated client-facing text can be fluent but subtly inappropriate.

Root cause: Algorithms often miss tone, sentiment, and nuanced expression choices.

Resolution: Keep human experts in the loop to choose expressions for the algorithm to model and approve.

Brand guidelines and truthfulness are requirements, not cosmetic preferences.

Root cause: Product copy can damage trust if it sounds wrong or makes unsupported claims.

Resolution: Define quality as original, unique, natural, compelling, truthful, and brand-aligned, then review against those criteria.

Evaluation

Stitch Fix used blind product-description comparisons, copywriter review, and ongoing quality assurance checks to assess whether generated content met quality and brand requirements.

Metrics: Blind quality scores · Copywriter time savings · Brand alignment · Truthfulness · Regular QA findings

Outcomes

• All Facebook and Instagram campaign headlines use the expert-in-the-loop headline approach.
• The fine-tuned product-description solution achieved higher quality scores than written descriptions in blind evaluation.
• Copywriters reported significant time and effort savings.
• The approach created an iterative feedback loop between experts and algorithms.

Stitch Fix reports campaign-wide headline generation and product-description scaling for hundreds of thousands of inventory styles.