Large Bank LLMOps Implementation: Lessons from Deutsche Bank and Others

various · Finance · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Generative AI adoption in regulated banking

Large banks wanted to apply generative AI to document processing, research, engineering productivity, compliance monitoring, procedure summarization, risk models, and eventually customer-facing experiences, but had to move under banking trust, model-risk management, regulatory scrutiny, data residency, cost governance, and business-case uncertainty.

Constraints:

• banks process hundreds of thousands of unstructured customer documents daily
• use cases must satisfy risk, compliance, security, and data-residency policies
• regulators are still forming expectations for generative AI
• customer-facing use cases carry bias, hallucination, discrimination, and accountability risks
• business sponsors cannot always commit to savings before pilots prove value
• cost and ROI need FinOps-style governance for generative AI
• employee roles and skills will change materially as AI adoption expands

Architecture

The banks describe a governed adoption model rather than a single application. Deutsche Bank chose Google Cloud partly because AI capabilities were part of the platform, then collected use cases bottom-up, clustered them into categories, and tested whether a capability could generalize across a category. Document AI became a banking-fit service with required controls and a 'concept of one' service-oriented architecture to avoid duplicate capabilities. Discover describes a center-of-excellence/advisory program with business, HR, risk, strategy, and technology partners, governed under model risk management, starting with internal and non-customer-facing use cases such as compliance monitoring, co-pilots, GitLab engineering productivity, RAG procedure summarization, underwriting letters, and marketing content generation.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Use_case_portfolio["Use-case portfolio"] --> Pilot_measurement_process["Pilot measurement process"]
  Cloud_AI_platform["Cloud AI platform"] --> Banking_controlled_Document_AI_service["Banking-controlled Document AI service"]
  Banking_controlled_Document_AI_service --> Concept_of_one_service_model["Concept of one service model"]
  Use_case_portfolio --> Research_report_automation["Research report automation"]
  Use_case_portfolio --> Engineering_productivity_tooling["Engineering productivity tooling"]
  Use_case_portfolio --> RAG_procedure_summarization["RAG procedure summarization"]
  Compliance_monitoring["Compliance monitoring"] --> Model_risk_management["Model risk management"]
  Pilot_measurement_process --> Model_risk_management
  COE_advisory_group["COE advisory group"] --> Use_case_portfolio
  COE_advisory_group --> Model_risk_management
  COE_advisory_group --> Employee_reskilling_program["Employee reskilling program"]
  Model_risk_management --> Regulator_engagement["Regulator engagement"]
  Regulator_engagement --> COE_advisory_group
  Business_sponsors["Business sponsors"] --> COE_advisory_group
  Risk_Tech_strategy_partners["Risk Tech strategy partners"] --> COE_advisory_group

Components

Data flow

Business teams propose generative-AI use cases, the bank clusters them into categories, sponsors pilots, applies model-risk and compliance review, and deploys proven use cases through controlled platform services. Document workflows feed unstructured customer documents into a banking-controlled Document AI service. Research workflows automate data collection for analysts. Compliance and procedure workflows use RAG-style summarization and monitoring. Governance flows run in parallel: COE/advisory groups include risk, technology, strategy, business, and HR partners; regulators are kept informed; customer-facing use cases wait until controls and regulatory clarity mature.

Why this architecture

The banks need a portfolio and governance architecture because generative AI affects nearly every task but banking adoption cannot be a free-for-all. Category pilots and shared services keep experimentation grounded in business problems; model-risk governance and regulator engagement keep risk controlled; internal-first use cases reduce customer-facing accountability risk; and top-down sponsorship plus reskilling handles organizational change.

Models used: Google Document AI · Vertex AI · Gemini · RAG · co-pilot tools · GitLab engineering productivity solution

Scale

• Deutsche Bank receives hundreds of thousands of unstructured customer documents daily.
  “we are still having a huge amount of unstructured documents that we are receiving on a daily basis and this is hundreds of thousands of Doc ments”transcript, stated
• A Deutsche Bank study suggested at least a third of roles would change or disappear/emerge in some form.
  “at least a third of our roles will change in certain way or form uh or will uh completely disappear and new roles will emerge”transcript, stated
• Research analysts reportedly spend 80% of time on data collection and 20% digesting/building reports.
  “he said 80% of the work is data collection and 20% is digesting and building the report”transcript, stated
• The panel cites 41% of GitHub code being AI-generated in the prior 10 months.
  “on GitHub 41% of the code is already AI generated and that's in last 10 months”transcript, stated
• Discover expected a formal regulator update around May or June.
  “our first formal um update uh with some actual detail about what we're doing it's probably May or June”transcript, stated

Alternatives considered (roads not taken)

Ignore generative AI or wait too long

Rejected because: Panelists argue the technology is already being embedded by vendors and creates competitive risk if banks stay on the sidelines.

Let every line of business build duplicate AI capabilities

Rejected because: Deutsche Bank wanted one reusable service for a dedicated use case to avoid redundant capabilities.

Start with customer-facing autonomous banker use cases

Rejected because: The banks view accountability and regulatory constraints as too immature for replacing humans in customer-facing agent banking.

Adopt AI without regulator education

Rejected because: Regulators are still learning and can slow adoption if controls or industry failures raise concern.

Patterns

safety-policy-enforcement (governance)

Banks govern generative-AI use cases through model risk management, policy compliance, regulator dialogue, and controls for security, data residency, bias, hallucination, and customer impact.

human-in-the-loop-gate (governance)

The banks frame near-term generative AI as augmenting human workers rather than replacing accountable humans, especially in regulated contexts.

eval-driven-development (evaluation)

The adoption model uses pilots to measure what is possible, which obstacles exist, and whether a capability generalizes across a use-case category.

rag-retrieval (retrieval)

Discover discusses a RAG implementation for procedure summarization in banking operations.

cost-aware-routing (operations)

The panel calls for FinOps-like cost governance for generative AI so proliferation does not outrun ROI.

Tradeoffs

Start with internal efficiency use cases.

Chose internal augmentation and operational efficiency over immediate customer-facing autonomous banking.

Rationale: Internal use cases reduce accountability and regulatory risk while teams build controls and expertise.

Use top-down sponsorship plus bottom-up use-case discovery.

Chose executive sponsorship, advisory groups, and business-proposed use cases over uncoordinated grassroots experiments or executive-only strategy.

Rationale: Business teams bring real opportunities, while top-down sponsorship and cross-functional governance move pilots into production.

Fit generic Document AI to banking controls.

Chose Google Document AI adapted for banking over plain OCR or unmanaged generic document AI.

Rationale: Plain OCR was not fit for purpose, and banking needed controls plus a reusable service model.

Govern GenAI under model risk management.

Chose model-risk management and regulator updates over separate informal innovation governance.

Rationale: Banking trust and regulatory scrutiny require established control frameworks before scaling.

Gotchas (what bit them)

A solution-first AI program can chase savings without a real problem owner.

Root cause: Early hype can produce expectations of large savings before a business sponsor can commit to outcomes.

Resolution: Return to real business problems and measure pilot value before scaling.

Uncoordinated bank AI adoption can create duplicate capabilities.

Root cause: Large banks historically build redundant tools across business lines.

Resolution: Use a service-oriented 'concept of one' model for shared AI services such as document processing.

Regulated industries can be slowed down by another institution's AI failure.

Root cause: If one bank creates biased, harmful, or uncontrolled output, regulators may impose horizontal reviews and new best practices across the industry.

Resolution: Bring regulators along, demonstrate control environments, and avoid customer-facing high-risk use cases until controls mature.

Generative AI cost can proliferate through SaaS providers and tools before ROI is clear.

Root cause: AI capabilities can be embedded everywhere without clear control over who uses them, for what use case, and at what cost.

Resolution: Apply FinOps-style governance to GenAI usage, cost, and ROI.

Data residency can constrain the value of global generative-AI models.

Root cause: Jurisdictional requirements may prevent holistic customer data use across countries.

Resolution: Plan for data-residency constraints and regulator dialogue as first-class platform requirements.

Evaluation

Evaluation is framed as governed pilots and business impact rather than one benchmark. The banks assess whether use cases are possible, what obstacles exist, what ROI and cost profile they have, and whether controls satisfy risk partners and regulators before production expansion.

Metrics: pilot feasibility · obstacles discovered · business sponsorship · cost and ROI · production readiness · regulatory control fit · research analyst data-collection time · document processing volume · engineering productivity · risk model data expansion

Outcomes

• Deutsche Bank adapted Google Document AI with banking controls for high-volume unstructured document processing.
• The banks identified research report automation, engineering productivity, risk models, compliance monitoring, RAG procedure summarization, underwriting letters, and marketing content as use cases.
• Discover had a compliance use case in production for state regulation monitoring.
• The adoption model includes COE/advisory governance, model risk management, regulator engagement, top-down sponsorship, and employee reskilling.
• Panelists expected customer-facing use cases to expand later as controls and regulatory clarity mature.

This is a panel with multiple institutions, not a single deployment report. It includes concrete scale indicators: hundreds of thousands of daily documents, hundreds of research analysts, 80% analyst time on data collection, a third of roles expected to change, and formal regulator updates planned.