LLM-Powered Upskilling Assistant in Steel Manufacturing

gerdau · Other · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Manufacturing data platform and workforce upskilling

Gerdau needed to replace a fragmented data ecosystem that limited real-time analytics, governance, business adoption, and future AI applications, including an early LLM assistant for employee re/upskilling.

Constraints:

• The existing homegrown and open-source tools were complex, disconnected, and hard to manage.
• Users needed Python and Spark proficiency to work with the data platform.
• The prior platform lacked real-time data processing for digital twin use cases.
• Gerdau needed fine-grained access control, lineage, and compliance-grade governance.
• Incorrect or late data can cause costly steel-production failures because chemical composition is critical.
• The source provides little technical detail about the LLM assistant itself.

Architecture

Gerdau consolidated its data ecosystem onto the Databricks Data Intelligence Platform. Delta Lake provides the storage foundation, Delta Sharing supports internal and external data exchange, Photon accelerates processing, Unity Catalog supplies governance and lineage, and Power BI gives business teams governed analytics access. This unified data foundation enabled advanced analytics, predictive maintenance, classification workloads, generative AI exploration, and one early LLM assistant for re/upskilling.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Platform["Databricks Data Intelligence Platform"] --> Delta["Delta Lake"]
  Delta --> Sharing["Delta Sharing"]
  Delta --> Photon["Photon"]
  Platform --> Catalog["Unity Catalog"]
  Catalog --> BI["Power BI integration"]
  Platform --> AI["AI application layer"]
  Catalog --> AI

Components

Data flow

Data sources and workflows are unified on Databricks, stored and processed through Delta Lake, shared through Delta Sharing, accelerated by Photon, governed through Unity Catalog, and surfaced to business users through Power BI and analytics/ML workflows. The same governed platform then supports future AI applications such as predictive maintenance, image/text classification, generative AI, and the re/upskilling assistant.

Why this architecture

The architecture prioritizes a governed unified data foundation before deeper AI deployment. Gerdau's fragmented toolchain created cost, maintenance, governance, and adoption barriers, so the platform first standardizes data storage, sharing, lineage, access control, and analytics access; the LLM upskilling assistant and future predictive/classification/generative AI uses build on that foundation.

Models used: LLMs

Scale

• Gerdau has a 123-year history.
  “With a history spanning 123 years, Gerdau is Brazil’s largest steel producer”article, stated
• Gerdau has operations in many countries and over 30,000 employees.
  “Gerdau has operations in many countries and over 30,000 employees.”article, stated
• Data processing time dropped from 1.5 hours to 12 minutes.
  “Gerdau has reduced their average data processing time from 1.5 hours to 12 minutes”article, stated
• Data processing costs fell by 40%.
  “the adoption of Databricks and consolidation of Gerdau’s various tools has resulted in a remarkable 40% cost reduction for data processing”article, stated
• Streaming-solution development costs fell by 80%.
  “and 80% in new developments for streaming solutions.”article, stated
• More than 300 global data users were onboarded.
  “This was evidenced by the rapid onboarding of over 300 new global data users, including operations in other countries such as Peru and the United States.”article, stated

Alternatives considered (roads not taken)

Continue with homegrown open-source data tooling

Rejected because: The toolchain was flexible but disconnected, hard to manage, and difficult for users to adopt without Python and Spark expertise.

Keep separate data stores and team-specific databases

Rejected because: Duplicate databases created inconsistencies, inaccuracies, collaboration barriers, and higher total cost of ownership.

Deploy advanced AI without first fixing data governance

Rejected because: The source frames governance, lineage, and access controls as prerequisites for scaling consistently and meeting compliance/security standards.

Patterns

data-quality-loop (data)

Gerdau's platform consolidation improves data consistency, governance, lineage, and accuracy so downstream analytics and AI workflows use trusted data.

privacy-preserving-processing (governance)

Unity Catalog provides access controls, lineage, and access segregation so data and AI use can meet compliance and security requirements.

latency-optimization (serving)

Photon and the unified platform reduce processing latency for daily data workflows that feed analytics and AI initiatives.

cost-aware-routing (operations)

The platform consolidation reduces operational and development costs by replacing multiple manual/disparate systems with a unified data environment.

Tradeoffs

Replace a flexible homegrown stack with a managed unified platform

Chose Databricks Data Intelligence Platform over continuing the open-source/homegrown ecosystem.

Rationale: The open-source stack offered flexibility, but the unified platform reduced complexity, improved adoption, and added governance capabilities.

Prioritize governed data foundations before deeper GenAI expansion

Chose lineage, access controls, source-of-truth data, and platform adoption over building AI applications on fragmented, inconsistent data systems.

Rationale: The platform creates the reliability, access, and compliance basis needed for advanced analytics, ML, and generative AI.

Gotchas (what bit them)

Flexible open-source data stacks can become adoption barriers.

Root cause: The homegrown tools required Python and Spark proficiency from users and were hard for engineering teams to maintain.

Resolution: Consolidate tooling into a single user-friendly platform that lowers the barrier to data and AI workflows.

Digital twin and manufacturing optimization use cases stall without real-time processing.

Root cause: The old platform could not offer real-time data processing capabilities.

Resolution: Adopt a unified platform with faster processing and support for real-time data handling.

Manufacturing decisions are sensitive to data correctness and timing.

Root cause: Steel production depends on correct chemical composition data at the right time.

Resolution: Use a unified source of truth, governance, and lineage to reduce inconsistencies and improve reliability.

Evaluation

The source reports platform impact through processing-time, cost, user-adoption, and user-feedback metrics; it does not describe a detailed evaluation harness for the LLM upskilling assistant.

Metrics: data processing time · data processing cost reduction · streaming development cost reduction · global data users onboarded · ML model development effort reduction

Outcomes

• Average data processing time dropped from 1.5 hours to 12 minutes.
• Data processing costs were reduced by 40%.
• New development costs for streaming solutions fell by 80%.
• Over 300 global data users were onboarded across countries including Peru and the United States.
• A user reported that ML model creation takes 30% less effort.
• The unified platform enabled early LLM and generative AI applications, including a re/upskilling assistant.

Gerdau operates in many countries with over 30,000 employees, produces long and special steel globally, and processes sustainability/manufacturing data across a large industrial footprint.