Building a Property Question-Answering Chatbot to Replace 8-Hour Email Responses with Instant AI-Powered Answers

agoda · E-commerce · 2024 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Travel-property question answering

Agoda needed travelers to get immediate answers to specific property questions without scanning long property pages or waiting for properties to respond through inbox messaging.

Constraints:

Property pages contain lots of information that is hard to pinpoint quickly
Many property inquiries through messaging were unanswered
Average property response time was eight hours
Answers needed to be property-specific and current
The chatbot needed guardrails for out-of-scope and unsupported questions
The interface needed to work across desktop, mobile web, and native app surfaces

Architecture

Agoda built Property AMA Bot as a three-layer chatbot system: an AIDE chatbot client/micro-frontend, an LLM API orchestration layer, and ChatGPT. When users ask property-specific questions, the client sends the query to the LLM API, which fetches current property data from Agoda's Property API, builds prompts with role, scope, topic guardrails, and context, calls ChatGPT, and returns an instant answer through the same reusable client across web, mobile web, and app.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  traveler[Traveler] --> aide-chatbot-client[AIDE Chatbot Client]
  aide-chatbot-client --> llm-api[LLM API]
  llm-api --> property-api[Property API]
  llm-api --> prompt-guardrails[Prompt Guardrails]
  prompt-guardrails --> chatgpt[ChatGPT]
  llm-api --> chatgpt
  chatgpt --> llm-api
  llm-api --> aide-chatbot-client
  evaluation-labeling[Evaluation Labeling] --> llm-api
  evaluation-labeling --> experimentation-platform[Experimentation Platform]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Traveler`	User asking property-specific questions before booking.	`AIDE Chatbot Client`
`AIDE Chatbot Client`	Reusable micro-frontend/NPM chatbot client rendered across Agoda surfaces.	`Traveler`, `LLM API`
`LLM API`	Backend orchestration layer connecting the chatbot client, Agoda services, and ChatGPT.	`AIDE Chatbot Client`, `Property API`, `Prompt Guardrails`, `ChatGPT`
`Property API`	Agoda service that provides current property details used as context.	`LLM API`
`Prompt Guardrails`	Prompt rules for role, reference context, out-of-scope queries, input topics, and unsupported questions.	`LLM API`, `ChatGPT`
`ChatGPT`	Language model that generates property answers from the enriched prompt.	`LLM API`, `Prompt Guardrails`
`Evaluation Labeling`	Data-science labeling of chatbot interactions used to refine prompts and algorithms.	`LLM API`, `Experimentation Platform`
`Experimentation Platform`	In-house A/B testing platform for comparing prompts or models against business outcomes.	`Evaluation Labeling`

Data flow

The user asks a property question in the AIDE client. The client passes the request to the LLM API. The LLM API fetches property information from PAPI, constructs an enriched prompt with guardrails and in-context property data, sends it to ChatGPT, and returns the generated answer to the chatbot client. Labeled responses and A/B tests feed prompt, data-input, and model iteration.

4 supporting quotes

“When a user poses a question, the chatbot client passes this request to LLM API. The API acts as a conduit, ensuring the question is parsed and relayed efficiently to ChatGPT for processing with context that is fetched from Agoda services.”article, stated

“For property-AMA, we connect to Property API (PAPI) to fetch property information that enriches the chatbot’s knowledge base. This integration ensures that when users inquire about properties, the information they receive is comprehensive and current.”article, stated

“The chatbot fetches relevant property data from Property API, enhancing responses with specific details. This ensures answers are accurate and contextually informed.”article, stated

“By applying these labels to chatbot interactions, we gain valuable insights that enable developers to refine prompts”article, stated

Why this architecture

Agoda used a context-fetching LLM API instead of relying on ChatGPT's stored knowledge because property answers must be current and specific to the selected accommodation. Packaging the client as a reusable micro-frontend let the same chatbot ship consistently across surfaces, while guardrails and labeling/A-B testing gave the first production AI chatbot a path to reliability improvement.

4 supporting quotes

“This integration ensures that when users inquire about properties, the information they receive is comprehensive and current.”article, stated

“Additionally, the chatbot is accessible across platforms, including desktop, mobile web, and the Agoda app, ensuring a seamless experience wherever users interact with Agoda.”article, stated

“This ensures that if a user asks something not allowed, the bot will not provide an answer. Press enter or click to view image in full size Press enter or click to view image in full size”article, stated

“By integrating the chatbot with our in-house experimentation platform, we can measure key metrics such as bookings, user engagement, and other business-critical outcomes to evaluate the success or failure of each prompt.”article, stated

Models used: ChatGPT

Scale

Only 55% of property questions received responses from properties.
“Statistics show that 80% of properties allow an inbox messaging feature. But only 55% of questions actually receive a response from properties. We recognized the need for a faster, more reliable solution to address these challenges and enhance the user experience.”article, stated
Agoda reported average property-message response time of eight hours.
“users to ask properties directly, it often falls short: many inquiries remain unanswered, and responses often take an average of eight hours. This delay makes it difficult for users to get the timely information they need to decide which property best suits their”article, stated
Eighty percent of properties allow inbox messaging.
“Statistics show that 80% of properties allow an inbox messaging feature. But only 55% of questions actually receive a response from properties. We recognized the need for a faster, more reliable solution to address these challenges and enhance the user experience.”article, stated
The bot returns property-specific information in seconds.
“bot ensures users have access to property-specific information in seconds, making the decision-making process easier and more reliable. Additionally, the chatbot is accessible across platforms, including desktop, mobile web, and the Agoda app, ensuring a seamless experience wherever users”article, stated

Alternatives considered (roads not taken)

Continue relying on property inbox messaging

Rejected because: Too many inquiries went unanswered and response times were too slow for booking decisions.

“Customer Messaging platform allows users to ask properties directly, it often falls short: many inquiries remain unanswered, and responses often take an average of eight hours. This delay makes it difficult for users to get the timely information they need to decide which property best suits”article, stated

“We recognized the need for a faster, more reliable solution to address these challenges and enhance the user experience. ## Introducing the Property AMA Bot”article, stated

Make users search property pages manually

Rejected because: Common questions required sifting through long descriptions and delayed booking confidence.

“Our property pages provide a wealth of details about accommodations, but for many users, pinpointing specific answers can be a challenge. Common questions such as, “Does this hotel offer free breakfast?” or “Is there a bar in the hotel?” often require users to sift through long pages of information.”article, stated

“The Property AMA Bot is designed to simplify the way users access property-related information. Instead of searching through detailed property descriptions or waiting hours for a response from a property, users can now ask the bot directly and receive an instant reply.”article, stated

Answer from LLM knowledge alone

Rejected because: The bot needed current, property-specific data from Agoda services.

“The API acts as a conduit, ensuring the question is parsed and relayed efficiently to ChatGPT for processing with context that is fetched from Agoda services. For property-AMA, we connect to Property API (PAPI) to fetch property information that enriches the chatbot’s knowledge base.”article, stated

“The chatbot fetches relevant property data from Property API, enhancing responses with specific details. This ensures answers are accurate and contextually informed.”article, stated

Patterns

`rag-retrieval` (property-context)

The bot retrieves current property data from Agoda's Property API before asking ChatGPT to answer.

“fetch property information that enriches the chatbot’s knowledge base.”article, stated

“enhancing responses with specific details.”article, stated

`guardrails-io-filtering` (scope-control)

Prompt rules classify allowed topics and prevent answers to unsupported or out-of-scope questions.

“Input topic guardrail — to check if the user’s question is allowed based on our provided topic list.”article, stated

“The bot attempting to answer with information we haven’t provided”article, stated

`eval-driven-development` (chatbot-quality)

Agoda labels chatbot responses and uses A/B tests to refine prompts, models, and business metrics.

“Our data science plays a key role in evaluating the question-and-answer data”article, stated

“A/B testing, which allows us to compare different prompts or models”article, stated

`prompt-versioning` (prompt-iteration)

Prompt engineering controls role, reference, scope, unsupported questions, and answer behavior.

“Prompt engineering is key to making the P-AMA chatbot work effectively.”article, stated

“It involves creating prompts that guide the chatbot’s responses for a smooth user experience.”article, stated

Tradeoffs

Use a property-data-backed LLM API

Chose ChatGPT with current PAPI context over direct property messaging or bare LLM answers.

Rationale: This gave instant answers while grounding responses in current Agoda property data.

“Instead of searching through detailed property descriptions or waiting hours for a response from a property”article, stated

“information they receive is comprehensive and current.”article, stated

Constrain the bot to property-related questions

Chose topic guardrails and unsupported-question handling over open-ended general travel assistant behavior.

Rationale: The first production chatbot needed reliability for property decision support before broader agentic expansion.

“While the Property AMA Bot currently focuses on property-related queries”article, stated

“Our roadmap includes the introduction of AI Agents to expand its capabilities beyond property questions.”article, stated

Gotchas (what bit them)

Instant answers are only useful if the bot stays in scope.

Root cause: Users can ask unrelated, unsupported, or under-contextualized questions.

Resolution: Agoda added input-topic guardrails and unsupported-question handling.

“Customers asking questions unrelated to the property”article, stated

“Customers asking questions the bot can’t answer”article, stated

Static property-page content does not meet the need for targeted answers.

Root cause: Users struggle to find narrow answers in detailed property descriptions.

Resolution: The bot answers specific property questions with context fetched from PAPI.

“Common questions such as, “Does this hotel offer free breakfast?” or “Is there a bar in the hotel?””article, stated

“ensures answers are accurate and contextually informed.”article, stated

Cross-platform deployment needs reusable frontend architecture.

Root cause: The same chatbot needed to work across website, mobile web, and native app.

Resolution: Agoda built the client as a reusable chatbot client/micro-frontend.

“the chatbot is accessible across platforms, including desktop, mobile web, and the Agoda app”article, stated

“ensuring a seamless experience wherever users interact with Agoda.”article, stated

Evaluation

Agoda evaluates the bot through response labeling by data science, prompt/model A/B tests on an in-house experimentation platform, and business metrics such as bookings and user engagement.

Metrics: answer labels · accuracy · reliability · bookings · user engagement · business-critical outcomes · prompt comparison results

“developers to refine prompts, adjust data inputs, and optimize algorithms, ultimately improving the chatbot’s accuracy and reliability.”article, stated

“we can measure key metrics such as bookings, user engagement, and other business-critical outcomes”article, stated

Outcomes

Agoda built its first production AI chatbot for property-specific questions.
The bot replaced hours-long property-message waits with instant responses in seconds.
The same chatbot client works across desktop, mobile web, and Agoda app surfaces.
The system grounds answers with PAPI property data and scope guardrails.
Evaluation combines labels, prompt/model experiments, and business outcome tracking.

The original messaging channel had 80% property participation, 55% response rate, and eight-hour average response time; Property AMA returns property-specific answers in seconds across desktop, mobile web, and app surfaces.

3 supporting quotes

“our first AI chatbot that provides instant and accurate answers to property-related questions.”article, stated

“users have access to property-specific information in seconds”article, stated

“This combination results in precise and personalized responses to user queries”article, stated