Multi-Agent Property Investment Advisor with Continuous Evaluation

prophero · Finance · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Property investment advisory

PropHero needed a Spanish/Australian property-investment advisor that could answer multi-turn investment questions with grounded advice, continuous monitoring, and controlled model cost.

Constraints:

• Advice had to be contextually relevant and grounded across an investment journey.
• The system needed to monitor context relevance, groundedness, and goal accuracy in real time.
• Foundation models had to be selected strategically to reduce cost.
• The architecture had to support Spanish FAQs, property guides, conversation history, and observability traces.

Architecture

PropHero built a four-layer Amazon Bedrock architecture: data foundation with DynamoDB, RDS/PostgreSQL, and S3; a LangGraph multi-agent layer in Lambda; Bedrock Knowledge Bases for semantic FAQ search; and continuous evaluation with CloudWatch, EventBridge, Lambda, and QuickSight dashboards. Agents include router, general, advisor, settlement, and response roles.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  api-gateway["API Gateway"]
  langgraph-agents["LangGraph Agents"]
  bedrock-knowledge-bases["Bedrock Knowledge Bases"]
  conversation-store["Conversation Store"]
  continuous-evaluation-layer["Continuous Evaluation Layer"]
  quicksight-dashboards["QuickSight Dashboards"]
  api-gateway --> langgraph-agents
  langgraph-agents --> bedrock-knowledge-bases
  langgraph-agents --> conversation-store
  langgraph-agents --> continuous-evaluation-layer
  bedrock-knowledge-bases --> langgraph-agents
  conversation-store --> langgraph-agents
  continuous-evaluation-layer --> quicksight-dashboards

Components

Data flow

A user conversation enters through API Gateway. A router agent classifies the query and sends it to a general, advisor, or settlement agent. Bedrock Knowledge Bases retrieve relevant Spanish FAQ or investment-guide context, a response agent formats the answer, and EventBridge/Lambda evaluation runs after conversation completion.

Why this architecture

PropHero separated data, agents, evaluation, and application access because the advisor needed grounded conversational advice plus continuous quality monitoring. Model selection and evaluation were designed into the architecture, not bolted on after deployment.

Models used: Amazon Bedrock foundation models

Scale

• The system achieved 90% goal accuracy.
  “The system achieved 90% goal accuracy, reduced customer service workload by 30%, lowered AI costs by 60%”summary, stated
• Customer service workload fell by 30%.
  “reduced customer service workload by 30%”summary, stated
• Optimal model selection lowered AI costs by 60%.
  “lowered AI costs by 60% through optimal model selection”summary, stated
• More than 50% of users and 70% of paid users engaged with the advisor.
  “enabled over 50% of users (70% of paid users) to actively engage with the AI advisor.”summary, stated

Alternatives considered (roads not taken)

Use a single model choice for all agents

Rejected because: PropHero selected models strategically across agents to reduce AI costs.

Treat evaluation as an offline afterthought

Rejected because: The production design integrated real-time evaluation into the architecture.

Patterns

multi-agent-orchestration (architecture)

Router, general, advisor, settlement, and response agents specialize in property-investment conversation stages.

rag-retrieval (knowledge)

Bedrock Knowledge Bases provide semantic search over FAQ-style property investment content.

eval-driven-development (quality)

The system continuously evaluates context relevance, groundedness, and goal accuracy.

cost-aware-routing (optimization)

Different foundation models are selected for different agents to lower cost.

Tradeoffs

Use multiple specialized agents

Chose Router, general, advisor, settlement, and response agents over One property-advice chatbot.

Rationale: The investment journey spans onboarding, advice, settlement support, and response formatting.

Integrate continuous evaluation

Chose CloudWatch, EventBridge, Lambda, and QuickSight evaluation layer over Manual occasional review only.

Rationale: Real-time groundedness and goal-accuracy monitoring were core production requirements.

Gotchas (what bit them)

Advisory systems need real-time groundedness and relevance monitoring.

Root cause: Property investment conversations are multi-turn and advice-like, so ungrounded answers can mislead users.

Resolution: PropHero integrated evaluation for context relevance, response groundedness, and goal accuracy.

Model choice materially affects production cost.

Root cause: Different agent tasks do not need the same foundation model capability.

Resolution: The architecture strategically selects foundation models per agent.

Evaluation

PropHero evaluates context relevance, response groundedness, and goal accuracy in real time.

Metrics: 90% goal accuracy · 30% customer-service workload reduction · 60% AI cost reduction · over 50% user engagement · 70% paid-user engagement

Outcomes

• PropHero delivered a property-investment advisor for Spanish and Australian consumers.
• The advisor achieved reported goal accuracy, workload, cost, and engagement improvements.
• Continuous evaluation became a first-class production component.

The case reports 90% goal accuracy, 30% workload reduction, 60% cost reduction, and more than 50% user engagement.