LLM-Powered Product Attribute Extraction from Unstructured Marketplace Data

etsy · E-commerce · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Marketplace product attribute extraction

Etsy needed to convert unstructured titles, descriptions, and listing photos into structured product attributes across a highly diverse marketplace so search filters, comparisons, and discovery systems could work without adding seller friction.

Constraints:

• Etsy has more than 100 million items from more than 5 million sellers.
• Unique handmade listings lack global SKUs and standardized product-attribute mappings.
• Most sellers provide unstructured data because structured fields are optional.
• Query-time processing of long descriptions and images creates latency concerns.
• Human ground-truth labeling is expensive, slow, and error-prone across thousands of categories.

Architecture

Etsy built an LLM attribute-extraction pipeline that curates listing context, prompts LLMs to extract attributes in parallel, routes requests through LiteLLM across regions, parses responses into Pydantic dataclasses, validates them with business logic, and exports structured outputs to filestores, database tables, and search systems. Evaluation combines human-labeled ground truth, LLM-generated silver labels reviewed by domain experts, performance regression checks, tracing, logging, metrics, and alerting.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Context["Listing context builder"] --> Prompts["Parallel attribute prompts"]
  Prompts --> Router["LiteLLM regional router"]
  Router --> Validation["Pydantic validation layer"]
  Validation --> PostProcess["Post-processing job"]
  PostProcess --> Sinks["Search and data sinks"]
  Evaluation["Evaluation and observability loop"] --> Prompts
  Evaluation --> Router

Components

Data flow

Seller-provided titles, descriptions, photos, taxonomy rules, category-specific extraction rules, and few-shot examples are assembled into JSON context. The context is injected into prompts for parallel attribute extraction, routed through LiteLLM, parsed into Pydantic dataclasses, validated, post-processed, and exported to downstream search and data systems.

Why this architecture

Etsy used LLMs because traditional classifiers and sequence taggers struggled with sparse long-tail attributes, while question-answering transformers still required expensive application-specific training data. The architecture focuses on context engineering, validation, and monitoring because Etsy needed scalable extraction without trusting raw LLM output directly.

Models used: foundational LLMs · high-performance, state-of-the-art LLMs · more scalable LLM

Scale

• Etsy's marketplace has more than 100 million items from more than 5 million sellers.
  “Our marketplace is home to more than 100 million special items made, handpicked and designed by more than 5 million sellers.”article, stated
• The attribute space spans millions of creative items across thousands of categories.
  “With millions of creative items across thousands of categories – many of which are unique – it’s difficult to accurately capture all possible product attributes”article, stated
• The pipeline was designed to extract attributes across millions of listings.
  “our next focus was to build a scalable pipeline that could extract attributes across millions of listings while maintaining confidence in the LLM output.”article, stated
• Complete attribute coverage rose from 31% to 91% in target categories.
  “In target categories, we’ve increased the number of listings with complete attribute coverage from 31% to 91%.”article, stated

Alternatives considered (roads not taken)

Seller-required structured attributes

Rejected because: Making every field required would increase seller friction and undermine flexibility for unique items.

Search-time processing of descriptions and photos

Rejected because: Long context and image processing at query time would hurt latency for buyer search.

Traditional supervised classification and sequence tagging

Rejected because: Sparse long-tail attributes and multi-attribute extraction made them hard to scale.

Human-only ground-truth labeling

Rejected because: Human annotation was slow, costly, and itself error-prone across thousands of categories.

Patterns

context-engineering (prompting)

Etsy explicitly curated listing titles, descriptions, images, few-shot examples, taxonomy logic, and category rules into the prompt context for extraction.

structured-output-contract (validation)

LLM outputs are parsed into Pydantic dataclasses and validated with both type checks and business logic before downstream export.

eval-driven-development (evaluation)

The team compares sample inferences against ground truth, tracks precision and recall, and terminates the pipeline when metrics drift from baselines.

human-in-the-loop-gate (quality)

Domain experts review silver labels and iterate on prompts before Etsy uses them for larger evaluation datasets.

observability-tracing (operations)

Etsy logs inference errors, exposes error metrics, alerts on failure thresholds, and samples traces in Honeycomb.

model-routing (infrastructure)

LiteLLM routes requests to different regions to improve parallelization and avoid single-region dependency.

Tradeoffs

Infer attributes instead of requiring sellers to fill every field

Chose LLM extraction from unstructured listing data over mandatory seller-provided structured attributes.

Rationale: Etsy could improve search and comparison without increasing listing friction for unique handmade items.

Precompute structured data

Chose distill unstructured product information into structured attributes before search use over feeding long text and images directly into search at query time.

Rationale: Every millisecond matters in search, so direct query-time interpretation was too slow.

Use silver-label evaluation with expert review

Chose LLM-generated labels reviewed by domain experts over exclusive reliance on third-party human annotation.

Rationale: This reduced labeling cost and time while keeping expert oversight in the loop.

Continue on individual inference errors

Chose log failures and alert on thresholds over fail the entire pipeline for any individual error.

Rationale: At marketplace scale, partial inference failures are expected and should be managed through observability rather than total aborts.

Gotchas (what bit them)

Unstructured product data is too slow to interpret at query time.

Root cause: Long descriptions and images are useful but expensive to process inside search request latency.

Resolution: Precompute structured attributes from unstructured listing data.

Human annotation is not a perfect evaluation source.

Root cause: Annotators make mistakes and labeling thousands of listings becomes slow and expensive.

Resolution: Use LLM-generated silver labels reviewed by Etsy domain experts.

Individual inference failures cannot halt a marketplace-scale batch pipeline.

Root cause: Failures can come from code bugs, permissions, transient errors, quota limits, safety filters, and other causes.

Resolution: Log individual errors, surface metrics, and alert only when failures exceed thresholds.

Low error rates can still hide model-performance regressions.

Root cause: Third-party model behavior or prompt changes may degrade extraction quality without obvious pipeline errors.

Resolution: Run sample inferences against ground truth, compare precision and recall with baselines, and terminate on significant drift.

Evaluation

Etsy evaluates attribute extraction with human-annotated datasets, silver labels reviewed by domain experts, precision, recall, Jaccard index, baseline comparisons, pre-production sample inference, tracing, logging, metrics, alerting, and automatic termination when metrics drift.

Metrics: precision · recall · Jaccard index · complete attribute coverage from 31% to 91% · search-filter engagement · post-click conversion rate

Outcomes

• Complete attribute coverage increased from 31% to 91% in target categories.
• Adding LLM-inferred attributes to search filters increased buyer engagement with relevant filters.
• Overall post-click conversion rate increased.
• LLM-inferred color attributes powered color swatches on search results pages.

The implementation targets Etsy's more than 100 million items, more than 5 million sellers, and thousands of categories, with reported target-category coverage improving from 31% to 91%.