Automating Video Ad Classification with GenAI

mediaradar_|_vivvix · Media & Entertainment · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Advertising intelligence and video ad classification

MediaRadar | Vivvix needed to classify video ads and identify products at much higher volume than manual operators and prior queue-based infrastructure could support.

Constraints:

• the classification task spans over 6 million unique products
• manual operators could not scale with rapidly increasing ad volume
• the prior SQS setup polled only 10 messages at a time and threatened SLAs
• video ads require preprocessing such as fingerprinting, transcription, translation, and OCR
• model experimentation and serving costs had to remain practical at thousands of assets per day

Architecture

MediaRadar | Vivvix moved ad-processing workflows onto Databricks and Spark Structured Streaming. Video ad data is ingested into the Databricks Data Intelligence Platform, preprocessed with duplicate fingerprinting, Whisper-style transcription/translation, and OCR, then classified through a dual-layer approach: GenAI identifies products in ads and those predictions are compared with MediaRadar | Vivvix's own classification models. Databricks selects the best combined match, supports model training visibility, and scales processing with Ray clusters and Structured Streaming.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  ad-data-sources[Ad data sources] --> databricks-data-intelligence-platform[Databricks Data Intelligence Platform]
  spark-structured-streaming[Spark Structured Streaming] --> databricks-data-intelligence-platform
  databricks-data-intelligence-platform --> preprocessing-pipelines[Preprocessing pipelines]
  preprocessing-pipelines --> ray-cluster[Ray cluster]
  databricks-data-intelligence-platform --> genai-product-identifier[GenAI product identifier]
  databricks-data-intelligence-platform --> existing-classification-models[Existing classification models]
  genai-product-identifier --> best-match-selector[Best-match selector]
  existing-classification-models --> best-match-selector
  best-match-selector --> databricks-data-intelligence-platform
  unity-catalog[Unity Catalog] --> databricks-data-intelligence-platform

Components

Data flow

Video-ad data enters Databricks, where preprocessing extracts duplicate fingerprints, transcriptions/translations, and OCR text. A Ray cluster scales video processing. GenAI generates product predictions from the ad, existing classification models produce their own matches, and the Databricks platform compares the combined predictions to select the best product classification. Structured Streaming keeps ingestion continuous and removes manual polling bottlenecks.

Why this architecture

The team needed a unified data and model platform because their prior SQS/manual workflow could not keep up with extreme product-classification scale. Databricks, Spark Structured Streaming, and Ray handled ingestion and processing throughput, while the dual-layer GenAI plus existing-model approach improved classification accuracy without discarding the company's existing models.

Models used: OpenAI GPT-3.5 · Whisper · fine-tuned in-house ML model · GenAI product identification model

Scale

• The classification space includes over 6 million unique products.
  “The nature of their task — classifying over 6 million unique products — posed “an extreme classification problem,” as Dong-Hwi Kim, MediaRadar | Vivvix’s Senior Machine Learning (ML) Engineer, put it.”article, stated
• The prior SQS setup was limited to polling 10 messages at a time.
  “Prior to Databricks, MediaRadar | Vivvix faced bottlenecks with their previous setup using Amazon Simple Queue Service (SQS), where manual polling for data and a limit of 10 messages at a time severely hindered the company’s ability to meet service level agreements (SLAs).”article, stated
• Manual classification depended on hundreds of operators.
  “Hundreds of operators were tasked with watching ads and recording detailed information on their elements, which was incredibly time-consuming and could not keep pace with the rapid increase in volume.”article, stated
• Processing volume increased from about 800 to 2,000 creatives per hour.
  “Before, we were able to do at most around 800 creatives an hour. With our move to Databricks, we’re classifying about 2,000 an hour.”article, stated
• The workflow processes thousands of creative assets daily.
  “This decision allowed them to process thousands of creative assets daily without incurring prohibitive costs. ## Serving clients better with a 150% increase in hourly throughput”article, stated
• Model experimentation work fell from about two days to about half a day.
  “What would normally take two days’ worth of work now takes maybe half a day,” Kim remarked. In addition, Databricks enabled MediaRadar | Vivvix to implement a more agile and adaptable development process.”article, stated

Alternatives considered (roads not taken)

Continue with SQS manual polling

Rejected because: The 10-message polling limit hindered SLA compliance.

Scale manual operators with ad volume

Rejected because: Ad spend nearly doubled and adding people to match volume was not feasible.

Use the in-house fine-tuned model alone

Rejected because: The model lacked enough training data for millions of products.

Use a more expensive GenAI model for every asset

Rejected because: The team chose GPT-3.5 to balance performance and expense while processing thousands of assets daily.

Patterns

batch-offline-processing (data)

Spark Structured Streaming and Databricks automate high-throughput ad ingestion and classification workflows.

model-routing (modeling)

The classification pipeline combines GenAI predictions with existing classification models and selects the best match.

cost-aware-routing (operations)

The team selected GPT-3.5 for cost-effective daily creative-asset processing.

observability-tracing (operations)

Databricks centralizes visibility into data ingestion, transformation, training, and model performance.

data-quality-loop (data)

Preprocessing pipelines fingerprint duplicates and extract transcript/OCR signals to keep a large product database cleaner for classification.

Tradeoffs

Move from SQS/manual polling to Spark Structured Streaming

Chose continuous Databricks streaming ingestion over manual polling from SQS in batches of 10 messages.

Rationale: Streaming removed a throughput bottleneck and improved SLA confidence.

Combine GenAI with existing classifiers

Chose dual-layer best-match selection over depending only on an in-house classifier or only on GenAI.

Rationale: The existing classifier alone lacked coverage, while combined predictions let Databricks select the best product match.

Use GPT-3.5 during high-volume model serving experiments

Chose OpenAI GPT-3.5 over more expensive models for every creative asset.

Rationale: The use case needed to process thousands of creative assets daily without prohibitive cost.

Gotchas (what bit them)

The prior queue architecture could not meet throughput requirements.

Root cause: Manual polling from SQS was limited to 10 messages at a time.

Resolution: The team moved to Spark Structured Streaming and Databricks for automated continuous ingestion.

Manual ad classification could not scale with market volume.

Root cause: Hundreds of people had to watch ads and record details while ad spend and volume grew rapidly.

Resolution: The workflow automated video processing and classification with Databricks, Spark, Ray, GenAI, and existing models.

A fine-tuned in-house model alone was insufficient.

Root cause: The product taxonomy was too large and diverse for the available training data.

Resolution: The team used a dual-layer approach that compared GenAI product identification with existing classification model outputs.

Open data access was hard to reconcile with customer security needs.

Root cause: The existing environment made it difficult to enforce who could see what.

Resolution: The team planned to move to Unity Catalog for access governance.

Evaluation

The case evaluates the migration through throughput, workflow-time reduction, SLA confidence, model experimentation speed, and operational visibility in Databricks.

Metrics: 2,000 creatives per hour after Databricks versus 800 before · thousands of ads per hour processed · model experimentation and testing reduced from two days to half a day · thousands of creative assets processed daily · millions of product categories

Outcomes

• Hourly creative classification throughput rose from about 800 to about 2,000.
• Manual intervention and administrative overhead were reduced.
• Model experimentation and testing became faster.
• Databricks improved visibility into data transformation, training, and model performance.

The case centers on a 6-million-product classification space, thousands of daily creative assets, and thousands of ads per hour after the Databricks migration.