AI-Assisted Activity Onboarding in Travel Marketplace

getyourguide · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.2 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Travel marketplace supplier onboarding

GetYourGuide suppliers spent too long creating activities, and content quality issues hurt traveler understanding and conversion.

Constraints:

Architecture

GetYourGuide added a generative-AI layer to its supplier activity onboarding wizard. Suppliers paste existing content into a dedicated AI input page; the model generates longer free-text sections and fills structured fields; suppliers review the output in the normal product-creation flow. Because ordinary A/B analysis was confounded by marketplace skew, the rollout also depended on a custom permutation testing framework and UI changes that clarified what the AI page would and would not do.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  activity-creation-wizard["activity creation wizard"]
  ai-input-box["AI input box"]
  llm-content-generator["LLM content generator"]
  structured-field-filler["structured field filler"]
  supplier-review-ui["supplier review UI"]
  permutation-testing-framework["permutation testing framework"]
  marketplace-metrics["marketplace metrics"]
  activity-creation-wizard --> ai-input-box
  ai-input-box --> llm-content-generator
  llm-content-generator --> structured-field-filler
  llm-content-generator --> supplier-review-ui
  structured-field-filler --> supplier-review-ui
  activity-creation-wizard --> marketplace-metrics
  marketplace-metrics --> permutation-testing-framework
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
activity creation wizard Existing 16-step supplier onboarding workflow for descriptions, photos, availability, pricing, and locations. AI input box, supplier review UI, marketplace metrics
AI input box Supplier-facing input where providers paste existing content such as website copy. activity creation wizard, LLM content generator
LLM content generator Generates free-text descriptions and structured field suggestions from pasted supplier content. AI input box, structured field filler, supplier review UI
structured field filler Autofills transport, location tags, and other structured onboarding fields across eight key steps. LLM content generator, supplier review UI
supplier review UI Review step where activity providers inspect generated content rather than manually filling every field. activity creation wizard, LLM content generator, structured field filler
permutation testing framework Experiment-analysis framework for supply-side and demand-side marketplace impact under skewed groups. marketplace metrics, activity creation wizard
marketplace metrics Completion, content-quality, traveler-performance, and conversion metrics used to judge launch impact. permutation testing framework, activity creation wizard

Data flow

A supplier enters the existing activity-creation wizard and pastes content from an external source into the AI input box. The generative model produces descriptions and structured fields, pre-filling eight key onboarding steps for supplier review. Experiment telemetry and marketplace outcomes flow into a custom permutation-testing framework so the team can decide whether the AI-assisted onboarding improved both supplier completion and traveler-facing performance.

4 supporting quotes
“Our activity onboarding process involves going through a 16-step product creation wizard where activity providers add various information such as descriptions, photos, availability, pricing, and locations.”article, implied
“our full-scale rollout of the feature enables activity providers to paste their existing content (such as from their own website) into a designated input box.”article, stated
“We then do some AI magic to generate both the longer free-text content sections, like the full description, as well as fill out many of the structured fields”article, stated
“The AI auto-completes 8 key steps, allowing activity providers to review content in minutes, reducing time spent on manual data entry.”article, stated

Why this architecture

The product problem spanned supplier effort, content quality, traveler clarity, and marketplace measurement. The team therefore kept AI inside the existing wizard with clearer expectations, human review, and a purpose-built experiment framework rather than treating generation quality as a standalone model metric.

4 supporting quotes
“activity providers were sometimes spending up to an hour manually creating a new product with our wizard.”article, stated
“the content our activity providers create can contain missing or contradictory information and might not always have our traveler-friendly tone of voice.”article, stated
“We clarified that the AI page was a step within the normal product creation wizard, with the left-side menu/progress bar visible.”article, stated
“A significant achievement also came in the form of building an experimentation framework that allowed us to measure both the supply side and the demand-side impact of the tool.”article, stated

Models used: LLM

Scale

Alternatives considered (roads not taken)

Standard A/B experiment analysis

Rejected because: Marketplace supplier skew meant the treatment and control groups were not comparable before the experiment.

“Our planned standard A/B experiment analysis approach would not hold under the given circumstances. We learned that our experiment groups A and B showed significant differences in both traveler — and supplier-side metrics”article, stated

Shipping after the first full-scale experiment

Rejected because: The primary metric decreased, so the team closed the experiment and redesigned the UX/measurement approach.

“With our primary success metric showing a decrease, we closed the experiment to A and did not launch the feature.”article, stated

Broad AI promise without tight product scope

Rejected because: The team explicitly warns that AI output depends on input quality, algorithm design, platform support, and practical constraints.

“The correctness and suitability of AI-generated results depend on factors like input data quality and algorithm design. While outputs may be technically correct, they might not meet what our platform could support.”article, stated

Patterns

structured-output-contract (content-operations)

The system fills structured onboarding fields such as transport and locations, not only prose descriptions.

“We then do some AI magic to generate both the longer free-text content sections, like the full description, as well as fill out many of the structured fields, like what kind of transport is used and what locations should be tagged.”article, stated

eval-driven-development (experimentation)

The team had to create a novel experiment approach because supplier-side AI variants affect traveler-side outcomes.

“One of these challenges was evaluating the experiment’s success.”article, implied

Tradeoffs

How much to promise from AI content generation.

Chose adaptable but precise scope over broad over-promising.

Rationale: The team explicitly warns against scope creep and unsupported AI output promises.

“One common pitfall is scope creep—underestimating the limitations of AI and over-promising what AI can realistically achieve in the marketplace.”article, stated

Gotchas (what bit them)

The first full-scale experiment failed because marketplace AI measurement was nonstandard.

Root cause: Suppliers can be assigned variants, but travelers cannot see AI and non-AI versions of the same product independently.

Resolution: Iterate the UI/UX and use permutation testing to evaluate downstream marketplace effects.

“B variants, travellers couldn't be separately assigned to these variants.”article, stated

AI outputs were hard to debug because correctness depended on input quality, algorithm design, and platform support.

Root cause: The generated result could be technically correct while still unsuitable for the marketplace workflow.

Resolution: Keep scope precise and evaluate both supplier-side output quality and traveler-side marketplace impact.

“Additionally, evaluating AI outputs is challenging because these systems often act as black boxes, making it hard to discern how decisions are made and what to tweak for the final product.”article, stated

Evaluation

GetYourGuide used experiment redesign, permutation testing, completion rates, content quality, and traveler engagement metrics.

Metrics: activity completion rate · content quality · onboarding time 14 minutes · traveler engagement

“The second rollout successfully increased activity completion rates, improved content quality, and reduced onboarding time to as little as 14 minutes, ultimately achieving positive impacts on both supplier efficiency and traveler engagement metrics.”summary, stated

Outcomes

Not a fully autonomous agent; marked non-agentic AI assistance.

“The second rollout successfully increased activity completion rates, improved content quality, and reduced onboarding time to as little as 14 minutes, ultimately achieving positive impacts on both supplier efficiency and t”summary, stated