Enterprise LLM Deployment with Multi-Cloud Data Platform Integration

databricks · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise LLM deployment on governed data platforms

Databricks described large enterprises struggling to move LLM prototypes into high-risk production paths because their data estates span clouds, warehouses, vendors, governance boundaries, and teams that do not all have GenAI expertise.

Constraints:

• Enterprise data is fragmented across clouds, vendors, services, and acquired systems.
• High-risk financial or reputational use cases require deterministic behavior from probabilistic systems.
• Governance must cover data, models, tools, queries, and agent permissions.
• Quality has to be measured granularly enough for production confidence.
• Latency also functions as a cost proxy for GenAI systems.

Architecture

Databricks frames production LLM systems as governed, evaluated, composable workflows over enterprise data. The FactSet example decomposed a large English-to-FQL prompt into a multi-node process so each task could be tuned separately, improving accuracy and latency. The broader platform story pairs Mosaic AI, governed data/model/tool access, vector or feature stores, MLflow evals, and Claude on Databricks for controlled agentic systems.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  enterprise-data-estate["Enterprise data estate"] --> databricks-data-platform["Databricks data platform"]
  databricks-data-platform --> vector-and-feature-stores["Vector and feature stores"]
  databricks-data-platform --> governance-layer["Governance layer"]
  governance-layer --> composable-agent-workflow["Composable agent workflow"]
  governance-layer --> claude-on-databricks["Claude on Databricks"]
  vector-and-feature-stores --> composable-agent-workflow["Composable agent workflow"]
  claude-on-databricks --> composable-agent-workflow["Composable agent workflow"]
  composable-agent-workflow --> evaluation-system["Evaluation system"]
  composable-agent-workflow --> application-output["Application output"]
  evaluation-system --> composable-agent-workflow["Composable agent workflow"]

Components

Data flow

Enterprise data is governed inside the Databricks data stack, injected into LLM workflows through vector or feature stores, routed through tools/agents or multi-step chains, evaluated with MLflow/custom judges, and deployed with controls over data, models, tools, and queries.

Why this architecture

Databricks argues that high-risk enterprise LLM systems need governed data/model/tool access plus granular evaluation because single-prompt prototypes do not provide enough determinism, quality measurement, or operational control.

Models used: Claude · Mosaic AI · MLflow · Databricks · Vector store · Feature store

Scale

• Databricks describes itself as a multicloud platform with tens of thousands of customers.
  “a leading data  platform, multicloud or crosscloud data platform,   uh tens of thousands of customers”transcript, stated
• FactSet's initial English-to-FQL prototype reached 59% accuracy with about 15 seconds of latency.
  “They ended up  with 59% accuracy in about 15 seconds of latency.”transcript, stated
• The decomposed workflow improved FactSet to 85% accuracy in six seconds of latency.
  “we got them to 85% accuracy   in six seconds of latency. at 85% accuracy.”transcript, stated
• Databricks previously produced a 450-page analyst response to 180 questions.
  “we ended up writing a 450 page document, right?  They had 180 questions for us”transcript, stated
• A Block customer example reported 40 to 50% weekly user adoption increase and 8 to 10 hours saved per week.
  “40 to 50% weekly user adoption increase 8  to 10 hours saved per week by using this.”transcript, stated

Alternatives considered (roads not taken)

Use a one-click RAG or massive-prompt approach for query translation.

Rejected because: The FactSet prototype was only slightly better than a coin flip and had high latency.

Leave a complex prompt as one undifferentiated task.

Rejected because: Databricks decomposed the prompt to tune performance at each step.

Rely on governance without measuring quality.

Rejected because: The speaker says governance is incomplete without evaluation for high-risk use cases.

Use earlier open-source or non-Anthropic models for questionnaire automation.

Rejected because: The results became shippable without touching only after switching to Claude.

Patterns

prompt-chaining (workflow)

FactSet's large prompt was decomposed into a multi-node, multi-step process so each step could be tuned.

tool-use-orchestration (orchestration)

Databricks describes agents choosing among tools, queries, parameterized functions, and nested decision trees.

eval-driven-development (evaluation)

Databricks emphasizes measuring quality at granular levels before high-risk production use.

privacy-preserving-processing (governance)

The platform governs data, model, tool, and query access for regulated organizations.

latency-optimization (optimization)

The FactSet workflow reduced latency from about 15 seconds to six seconds while improving accuracy.

Tradeoffs

Decompose FactSet's prompt into a workflow.

Chose Multi-node chain with step-level tuning. over A single large prompt plus vector database..

Rationale: Step-level tuning improved both accuracy and latency.

Treat agents as governed principals.

Chose Control access to data, models, tools, and queries. over Letting prototype agents access broad enterprise systems by default..

Rationale: High-risk enterprise use requires tight, low-grain control.

Pair Claude with Databricks for governed enterprise use.

Chose Claude available inside Databricks instances across Azure, AWS, and GCP. over Waiting for external technology access outside the governed platform..

Rationale: The platform gives regulated teams access to frontier models with their existing controls.

Gotchas (what bit them)

One-click RAG can look plausible while remaining too weak for production.

Root cause: A massive prompt plus vector database delivered only 59% accuracy for FactSet.

Resolution: Decompose the prompt into tunable workflow steps and measure accuracy/latency.

Enterprise GenAI work gets blocked by fragmented data expertise.

Root cause: Data warehouses, streaming systems, and domain experts are spread across different teams.

Resolution: Bring AI capabilities onto the governed data platform rather than isolating them from the data estate.

Governed access alone does not prove production readiness.

Root cause: Teams may claim a system is good without quantifying accuracy.

Resolution: Measure quality and hill-climb performance with evals.

Evaluation

The talk frames evaluation as quantified accuracy, latency/cost measurement, granular question-space testing, MLflow evals, and LLM judges for higher-risk systems.

Metrics: FactSet accuracy 59% to 85% · FactSet latency 15 seconds to six seconds · Global logistics system 85% accurate · Granular question-space coverage · Safe score/guardrail-style response checks

Outcomes

• FactSet improved from 59% accuracy and 15 seconds latency to 85% accuracy and six seconds latency.
• FactSet became comfortable showing the system to existing customers.
• Databricks used Claude to draft analyst questionnaire responses close to final draft quality.
• The Block/Goose example reports 40 to 50% weekly adoption increase and 8 to 10 hours saved per week.

Databricks discusses tens of thousands of customers and production use across banks, hospitals, logistics, FactSet, Databricks internal workflows, and Block.