Scaling Foundation Models for Predictive Banking Applications

nubank · Finance · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: predictive banking foundation models

Nubank needed to introduce billion-parameter foundation models into regulated banking decision engines without discarding existing governance, data infrastructure, and production ML baselines.

Constraints:

Nubank serves more than 100 million customers and critical decisions occur across all users monthly.
Existing production models rely on governed tabular ML pipelines.
Transformer workloads require heterogeneous GPU clusters, high-quality sequence data, and leakage-free evaluations.

Architecture

Nubank and Hyperplane integrated foundation models into the existing Nubank AI Platform. They preserve data governance and deployment components, build new sequence-data ingestion/validation, use Ray to scale heterogeneous GPU clusters, train billion-parameter transformer and DNN architectures on raw sequential transaction data, and compare challengers against tabular ML baselines before production approval.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Production sequence data"] --> B["Data validation and enrichment"]
  B --> D["Foundation model training"]
  C["Ray heterogeneous clusters"] --> D
  D --> E["Benchmark evaluation loop"]
  E --> F["Existing AI Platform"]
  F --> G["Production decision engines"]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Production sequence data`	Transaction and non-transaction histories used as raw sequential model inputs.	`Data validation and enrichment`
`Data validation and enrichment`	Ensures sequence data quality, monitors production safety, and avoids leakage.	`Production sequence data`, `Foundation model training`
`Ray heterogeneous clusters`	Scales GPU/heterogeneous infrastructure for billion-parameter training.	`Foundation model training`
`Foundation model training`	Transformer and DNN training over customer transaction sequences.	`Data validation and enrichment`, `Ray heterogeneous clusters`, `Benchmark evaluation loop`
`Benchmark evaluation loop`	Compares foundation-model challengers to tabular ML baselines with matched labels and data.	`Foundation model training`, `Existing AI Platform`
`Existing AI Platform`	Nubank's model governance, data infrastructure, and deployment platform.	`Benchmark evaluation loop`, `Production decision engines`
`Production decision engines`	Critical banking decisions that use approved predictive models.	`Existing AI Platform`

Data flow

Production sequence data is ingested, validated, enriched, and monitored; Ray-backed heterogeneous clusters train billion-parameter transformer models over customer transaction histories; challenger models are evaluated against replicated tabular baselines using the same labels and inputs; approved models are deployed into production decision pipelines.

3 supporting quotes

“train and deploy billion-parameter models across several predictive modeling use cases.”article, stated

“We leverage Ray to enable our small infra team to scale out heterogeneous clusters”article, stated

“Models in Production: Get model approval and deploy them into production pipelines to serve customers.”article, stated

Why this architecture

Nubank constrained the foundation-model program to use existing governed inputs and deployment paths, so performance gains could be measured fairly against production baselines while preserving regulated-finance guardrails.

3 supporting quotes

“Nubank’s years of experience in data infrastructure, model governance, and model deployment are essential guardrails for productionizing these complex models in a highly regulated environment.”article, stated

“By changing as few levers as possible, we fairly replicate baselines and measure delta performance improvements.”article, stated

“all model inputs are already validated and monitored in production.”article, stated

Models used: billion-parameter transformer models · large-scale DNNs · linear models · gradient-boosted trees · tabular ML baselines

Scale

Nubank serves more than 100 million customers.
“ML Engineers to train billion-parameter models on all 100+ million Nubank customers and their transaction histories.”article, stated
Models ingest billions of labels and thousands of transactions per label.
“our models ingest O(billions) of labels and O(thousands) transactions per label.”article, stated
Final foundation models process trillions of tokens during training.
“our final Foundation Models process O(trillions) tokens during training.”article, stated
The first eight months produced a +1.20% average AUC lift across four benchmark tasks.
“we present the average AUC lift of our models across four benchmark tasks over the first several months.”article, stated
The +1.20% AUC lift is 2-3x a mature model's typical annual lift.
“a +1.20% AUC lift is 2~3x the lift for a mature model’s typical annual release.”article, stated

Alternatives considered (roads not taken)

Replace Nubank's existing platform components

Rejected because: Existing governance, deployment, and data infrastructure were critical regulated-finance guardrails.

“Components where Nubank has expertise would not be touched.”article, stated

Use new unvalidated data sources immediately

Rejected because: Initial experiments intentionally used the same raw sequence sources as baselines to isolate model impact.

“these engagements begin by leveraging only the same raw sequence data sources as in the baseline model”article, stated

Continue only tabular ML

Rejected because: Foundation models can capture behavioral signals not available through aggregated tabular features.

“capture behavioral signals that are impossible to detect using traditional aggregated feature approaches.”article, stated

Patterns

`fine-tuning-adaptation` (model-development)

Nubank trains billion-parameter transformer foundation models for banking prediction tasks.

“This blog post outlines how platform and product teams worked together over the first eight months of this effort to train and deploy billion-parameter models across several predictive modeling use cases. We break down our journey into three sections:”article, stated

`eval-driven-development` (evaluation)

Foundation-model challengers are evaluated against replicated production tabular baselines to measure delta improvements.

“With this structure in place, we onboard product use cases and train Foundation Model-based challengers against Tabular ML baselines. Building a tight, reliable experiment iteration loop is paramount when introducing significant modeling changes to an existing product.”article, stated

`observability-tracing` (operations)

The sequence-data team extends monitoring and tooling so new sequence data can be safely deployed in production.

“this team also works to develop additional tooling to ensure that sequence data can be deployed safely in production.”article, stated

Tradeoffs

Initial experiment isolation

Chose same raw sequence sources as baselines over new data plus new model simultaneously.

Rationale: This isolates model impact and keeps inputs already validated and monitored.

“This ensures that all model inputs are already validated and monitored in production.”article, stated

Platform integration

Chose reuse existing AI Platform guardrails over standalone foundation-model stack.

Rationale: Financial-industry productionization needs existing governance and deployment controls.

“Moreover, by leveraging years of Nubank’s expertise in governing and productionizing model innovations, these new projects can focus on optimizing metrics within a constrained yet ambitious modeling scope.”article, stated

Gotchas (what bit them)

Foundation models can overwhelm established production discipline.

Root cause: Large models, less structured data, and new clusters add complexity.

Resolution: Keep existing governance/deployment components and change as few levers as possible.

“As models get larger and data less structured, it is easy to get lost in the new complexity. Our aforementioned approach enables us to build and measure progress incrementally and in a principled manner.”article, stated

Sequence-data leakage would invalidate banking model evaluation.

Root cause: High-value predictive decisions depend on clean temporal and transaction data.

Resolution: Create a dedicated team to ingest, validate, enrich, and monitor sequence data.

“As with all ML innovations, immense effort is required behind the scenes to ensure that the data fed into our models is of high quality and free of leakages. We formed a dedicated team to ingest, validate, and enrich Nubank’s enormous trove of rich transaction (and non-transaction) data sources.”article, stated

Transformer workloads need different infrastructure than tabular models.

Root cause: CPU clusters are enough for many linear/GBDT models, but transformer training and serving need heterogeneous GPU clusters.

Resolution: Use Ray to scale heterogeneous clusters for foundation-model workloads.

“Large-scale linear and gradient-boosted tree models can be run on CPU clusters. However, training and deploying transformer-based architectures require large heterogeneous clusters.”article, stated

Evaluation

Nubank evaluates challengers against tabular ML baselines with matched labels and features, tracks AUC lift across benchmark tasks, and requires model approval before production deployment.

Metrics: average AUC lift · benchmark task performance · baseline replication · model approval · production deployment readiness

“Replicate baseline metrics using the same labels and tabular features as those in production models to ensure that all data is correct and evaluations are fair.”article, stated

“Note that a +1.20% AUC lift is 2~3x the lift for a mature model’s typical annual release. Furthermore, we achieve this lift without adding any new data source, relying solely on signals already in the baseline models but represented as tabular features.”article, stated

Outcomes

Nubank deployed large-scale transformer-based sequence models in several key decision engines.
The first eight months delivered a +1.20% average AUC lift across four benchmark tasks.
Foundation models process trillions of training tokens across 100+ million customers and their transaction histories.

The source is an engineering blog by Nubank/Hyperplane and provides architecture plus early performance metrics.

“This work has enabled the deployment of large-scale transformer-based sequence models in several key decision engines.”article, stated