AI-Powered Travel Assistant for Rail and Coach Platform

trainline · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Rail and coach travel support

Trainline needed to support customers during travel disruptions and long-tail journey questions at a scale that phone or web support could not cover, without trapping users in a deterministic chatbot that only handled a narrow FAQ set.

Constraints:

• customers ask unpredictable travel questions, from refunds to pets, motorbikes, live train position, and station disruptions
• the assistant must operate across ticket terms, curated content, realtime train APIs, station data, and refund workflows
• rail support must be available at large customer scale and across changing live journey contexts
• answers must be factually grounded and safe enough for travel disruption scenarios
• human handoff is required for sensitive or unsupported cases

Architecture

Trainline built a production travel assistant around a single central orchestrator agent working in a loop with tools. The first proof of concept used a vector database, terms-and-conditions tool, and mock refund endpoint to test whether the agent could route questions correctly. The production system expanded to information retrieval over about 700,000 Trainline-curated pages, realtime train and station APIs, terms and conditions, refund/action tools, guardrail checks at input and output, contextual grounding checks, a retrieval-judge LLM, clean prompts, and seamless human handoff for sensitive or unsupported situations.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  free-text-chat-ui["free-text-chat-ui"] --> central-orchestrator-agent["central-orchestrator-agent"]
  free-text-chat-ui["free-text-chat-ui"] --> human-support-handoff["human-support-handoff"]
  central-orchestrator-agent["central-orchestrator-agent"] --> terms-and-conditions-tool["terms-and-conditions-tool"]
  central-orchestrator-agent["central-orchestrator-agent"] --> information-retrieval-service["information-retrieval-service"]
  central-orchestrator-agent["central-orchestrator-agent"] --> realtime-train-api["realtime-train-api"]
  central-orchestrator-agent["central-orchestrator-agent"] --> refund-action-tool["refund-action-tool"]
  central-orchestrator-agent["central-orchestrator-agent"] --> guardrail-layer["guardrail-layer"]
  central-orchestrator-agent["central-orchestrator-agent"] --> human-support-handoff["human-support-handoff"]
  information-retrieval-service["information-retrieval-service"] --> curated-content-vector-store["curated-content-vector-store"]
  information-retrieval-service["information-retrieval-service"] --> retrieval-judge-llm["retrieval-judge-llm"]
  guardrail-layer["guardrail-layer"] --> human-support-handoff["human-support-handoff"]
  evaluation-and-monitoring-harness["evaluation-and-monitoring-harness"] --> central-orchestrator-agent["central-orchestrator-agent"]

Components

Data flow

A customer opens free-text chat and asks a travel question. The central orchestrator receives the conversation and decides which tool to call: terms and conditions, information retrieval, realtime train data, station data, refund/action flows, or human handoff. The retrieval service summarizes only relevant context back to the orchestrator. Guardrail checks run on the way in and out, including contextual grounding against tool-provided information. If the assistant cannot help safely, the chat is handed to a human with the conversation context.

Why this architecture

Trainline chose a fully agentic orchestrator because customers needed flexible support for a long tail of unpredictable travel questions rather than a deterministic decision tree. Separation of concerns kept tools and services maintainable. Guardrails, retrieval judging, and human handoff reduced safety risk, while realtime APIs allowed the assistant to answer live journey questions rather than static FAQs.

Models used: LLM · RAG · vector database · LLM-as-judge · Braintrust · AWS inference routing

Scale

• Trainline sells tickets in 40 countries.
  “we actually sell tickets in in 40 countries.”transcript, stated
• Trainline has more than 27 million customers, including 18 million in the UK.
  “we now have over 27 million customers including 18 million in the UK.”transcript, stated
• The proof of concept took two months and production followed three months later.
  “we built a PC in two months to do this we validated it we did some human evaluation and then from there we went on and we got it into production 3 months after that.”transcript, stated
• Production launched around February 2025.
  “we went from zero to delivery into production in front of our customers in in roughly February 2025.”transcript, stated
• The retrieval corpus expanded to about 700,000 pages of Trainline-curated material.
  “We since expanded that to about 700,000 pages of train line curated material that we have on our website already built for SEO.”transcript, stated
• The assistant sees about 300,000 conversations per month in testing.
  “We're in boots testing now. We've got 300,000 people having conversations every month.”transcript, stated
• Average latency improved from about 12 seconds to about five to six seconds.
  “the response time was about 12 seconds”transcript, stated

  “we've managed to get latency actually down to I think last time I checked the average was about five to six seconds”transcript, stated

Alternatives considered (roads not taken)

Deterministic chatbot or decision tree

Rejected because: Users expected unique answers and would lose trust if the assistant only handled a narrow subset of questions.

Human support for every UK customer

Rejected because: Trainline could not put 18 million UK customers at the end of the phone, so AI offered 24/7 scalable support.

Static FAQ/terms-only assistant

Rejected because: The system needed realtime train position, station, disruption, and action tools, not just terms and conditions.

Optimize only for fastest response

Rejected because: The team accepted several seconds of latency because a correct answer was more valuable than a wrong instant response.

Patterns

multi-agent-orchestration (agent architecture)

A central loop orchestrator manages input, output, and tool calls, with tools that can themselves be agents or APIs.

rag-retrieval (retrieval)

Trainline processes hundreds of thousands of curated pages into a vector database for assistant retrieval.

llm-as-judge (evaluation)

A specialized LLM judges retrieved information relevance before summarized context reaches the orchestrator, and another simulator helps test realtime ticket scenarios.

guardrails-io-filtering (safety)

The assistant uses input/output guardrails, contextual grounding, prompts, tool validation checks, and human escalation.

human-in-the-loop-gate (safety)

Unsupported or sensitive cases are handed to human agents inside the same chat context.

Tradeoffs

Expose open free-text chat.

Chose Unprompted open chat over Heavily guided prompts and button flows.

Rationale: Open chat built trust that the assistant was not just a decision tree and generated valuable product discovery data.

Launch with an agent-plus-tools architecture early.

Chose Single orchestrator agent loop from the start over Wait for agent patterns to mature or build only deterministic workflows.

Rationale: The team needed to validate whether LLM reasoning could choose tools and handle diverse customer situations at scale.

Prioritize correctness over lowest latency.

Chose Five-to-six second average response with better answer quality over One-second incorrect answers.

Rationale: For travel support, a slightly slower correct answer is more useful than an instant wrong answer.

Gotchas (what bit them)

Open-ended travel chat can loop or call tools indefinitely.

Root cause: A central agent loop may keep reasoning and trying tools if it cannot satisfy an objective.

Resolution: Use guardrails, validation checks, prompt directives, and human handoff rather than relying only on the loop.

Latency is dominated by reasoning calls.

Root cause: Each reasoning call waits for model output before the next tool or step can run.

Resolution: Optimize inference routing with AWS and manage the user's expectation with product UX while preserving answer quality.

Evaluation is harder than normal model testing.

Root cause: A general-purpose assistant can receive a million and one questions tied to realtime ticket and journey context.

Resolution: Combine human evaluation, red teaming, semantic checks, monitoring, and synthetic realtime ticket simulations.

Evaluation

Trainline evaluates the assistant through human evaluation, small-scale automated semantic checks, third-party and internal red teaming, groundedness/relevance/helpfulness/consistency principles, Braintrust monitoring, and a synthetic realtime ticket simulator that generates customer-like queries against current train-ticket context.

Metrics: groundedness · factuality · relevance · helpfulness · consistency · toxicity/safety · latency · human handoff quality · synthetic realtime ticket scenario pass/fail

Outcomes

• Trainline went from proof of concept to production in about five months.
• The assistant launched to customers around February 2025.
• The corpus grew from FAQ/terms material to about 700,000 curated pages plus realtime train data.
• The system handles around 300,000 conversations per month in testing.
• Latency improved from about 12 seconds to about five or six seconds.

Trainline serves more than 27 million customers across 40 countries, including 18 million in the UK, and the assistant now operates over hundreds of thousands of monthly conversations.