Unified Data Foundation for AI-Fueled Mortgage and Home Ownership Platform

rocket · Finance · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Mortgage AI data foundation

Rocket needed to replace fragmented, siloed mortgage and customer data with a unified governed data foundation that could support ML models, agentic AI, acquisitions, and real-time business intelligence.

Constraints:

• Rocket served 1 in 6 mortgages nationwide.
• Data was fragmented across 12+ OLTP systems and multiple schemas.
• The foundation initially consolidated 10+ petabytes, later growing to 30+ petabytes.
• Data scientists, analysts, applications, and AI systems needed one governed source of truth.
• Some orchestration choices such as Step Functions could become expensive at high frequency.

Architecture

Rocket built a unified S3 data lake using Apache Iceberg and Parquet, with ingestion, processing, and consumption layers. Data lands raw and immutable, is processed into cleansed and conformed zones, and is exposed as standardized data products such as Customer 360, Mortgage 360, and Transaction 360 through APIs. The platform supports 210+ automated ML models and agentic AI applications.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  source_oltp_systems["Source OLTP systems"] --> ingestion_layer["Ingestion layer"]
  ingestion_layer --> s3_iceberg_data_lake["S3 Iceberg data lake"]
  s3_iceberg_data_lake --> processing_layer["Processing layer"]
  processing_layer --> data_product_apis["Data product APIs"]
  data_product_apis --> analytics_and_bi["Analytics and BI"]
  data_product_apis --> ml_model_platform["ML model platform"]
  data_product_apis --> agentic_ai_applications["Agentic AI applications"]
  ml_model_platform --> agentic_ai_applications

Components

Data flow

OLTP systems, services, and streams land data into S3 in raw Parquet/Iceberg form. Event-driven and scheduled processing through EMR, Glue, Flink, Lambda, and Step Functions cleanses and conforms data into business data products. APIs then serve Customer 360, Mortgage 360, and Transaction 360 to analysts, applications, data scientists, ML models, and AI agents.

Why this architecture

Rocket needed AI systems and data scientists to use the same governed datasets as production applications, so the data lake and standardized data products replaced duplicated team-specific sources of truth.

Models used: 210+ machine learning models · agentic AI applications

Scale

• Rocket handles 1 in 6 mortgages nationwide.
  “Rocket Companies serves as America's number one mortgage provider, handling 1 in 6 mortgages nationwide”summary, stated
• The platform consolidated 10+ PB initially and grew to 30+ PB.
  “consolidating 10+ petabytes of data initially (growing to 30+ petabytes at the time of the presentation)”summary, stated
• The data came from 12+ OLTP systems.
  “from 12+ different OLTP systems into one standardized environment.”summary, stated
• Rocket ran 210+ ML models in full automation.
  “enabled 210+ machine learning models running in full automation”summary, stated
• Mortgage approval time dropped from weeks to under 8 minutes.
  “reduced mortgage approval times from weeks to under 8 minutes”summary, stated

Alternatives considered (roads not taken)

Continue siloed product data systems

Rejected because: Data fragmentation created multiple sources of truth and blocked AI strategy.

Transform data separately for every use case

Rejected because: Rocket adopted transform once, serve many.

Let every team build its own customer data view

Rejected because: Rocket standardized Customer 360, Mortgage 360, and Transaction 360.

Patterns

batch-offline-processing (data foundation)

Rocket processes raw, cleansed, and conformed data products for many downstream consumers.

tool-use-orchestration (agentic BI)

Agentic AI applications use standardized data products for real-time business intelligence.

data-quality-loop (governed data products)

The platform standardizes ownership, cleanliness, and golden-source data for ML and applications.

Tradeoffs

Use open table formats on S3.

Chose Apache Iceberg and Parquet over Proprietary or siloed data stores.

Rationale: Open formats made data accessible to varied tools and downstream AI/analytics consumers.

Create curated 360 data products.

Chose Customer 360, Mortgage 360, Transaction 360 over Team-specific duplicated data views.

Rationale: Standardized data products prevent duplication and conflicting sources of truth.

Gotchas (what bit them)

AI models can diverge from production behavior if trained on different data.

Root cause: Data scientists and applications may consume different datasets when data is fragmented.

Resolution: Serve analysts, apps, and ML from the same governed data products.

Step Functions can become costly at very high workflow frequency.

Root cause: Lightweight workflows may run thousands of times per minute.

Resolution: Monitor orchestration cost and adjust architecture when needed.

Evaluation

Rocket evaluates the data foundation through automation scale, mortgage approval speed, model count, acquisition impact, and conversion/recapture metrics.

Metrics: 10+ PB initially, 30+ PB at presentation time. · 12+ OLTP source systems. · 210+ automated ML models. · Approval time from weeks to under 8 minutes. · 20% refinance pipeline increase. · 3x industry recapture rate. · 10% conversion lift. · 9-point banker follow-up improvement.

Outcomes

• Rocket consolidated fragmented mortgage data into a governed S3/Iceberg lake.
• The platform supports 210+ automated ML models.
• Mortgage approval times fell from weeks to under 8 minutes.
• Acquisition integration improved refinance and conversion metrics.

Rocket handles 1 in 6 US mortgages and consolidated tens of petabytes from 12+ systems.