Autonomous Semiconductor Manufacturing with Multi-Modal LLMs and Reinforcement Learning

samsung · Tech · 2023 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Semiconductor manufacturing process control

Samsung is trying to move toward autonomous fab operations where AI can sense equipment behavior, analyze multimodal production data, and eventually support control decisions across extremely complex chip-manufacturing processes.

Constraints:

• advanced-node semiconductor manufacturing can involve close to 1,000 process steps
• profitability requires roughly 80% yield or better
• sensor trace data arrives every second and is too large to inspect manually
• equipment, chambers, and sensors use inconsistent names across tools
• time-series foundation models were not yet accurate enough for process control

Architecture

Samsung frames Autonomous Fab around sensing, analysis, and control. The architecture combines equipment trace data, event logs, process histories, engineering notes, knowledge graphs, RAG/LLM assistants, multimodal foundation models, time-series models, OES/defect-image analysis, and reinforcement learning for future control. Production use already includes a multimodal engineering assistant and knowledge-graph/LLM alignment of sensor names, while process-control use remains gated by time-series accuracy and data-standardization work.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  fab-equipment-sensors[Fab equipment sensors] --> trace-data-pipeline[Trace data pipeline]
  trace-data-pipeline --> time-series-foundation-models[Time-series foundation models]
  trace-data-pipeline --> knowledge-graph[Knowledge graph]
  event-and-history-stores[Event and history stores] --> knowledge-graph
  knowledge-graph --> rag-assistant[RAG assistant]
  rag-assistant --> engineer-users[Engineer users]
  multimodal-foundation-models[Multimodal foundation models] --> oes-analyzer[OES analyzer]
  multimodal-foundation-models --> defect-detector[Defect detector]
  trace-data-pipeline --> multimodal-foundation-models
  oes-analyzer --> engineer-users
  defect-detector --> engineer-users
  time-series-foundation-models --> process-control-layer[Process-control layer]
  fab-equipment-sensors --> process-control-layer
  process-control-layer --> reinforcement-learning-controller[Reinforcement learning controller]

Components

Data flow

Manufacturing equipment emits second-by-second trace data plus event, maintenance, inspection, OES, and image data. Knowledge graphs and LLMs align inconsistent sensor and chamber naming, RAG-style assistants retrieve historical fixes and suggestions, multimodal models analyze text/images/tables/time-series signals, and control candidates such as reinforcement learning are explored only after sensing and forecasting are trustworthy enough.

Why this architecture

Samsung uses a layered multimodal architecture because fab control is not a single-text problem: it requires sensor traces, event logs, maintenance history, images, spectroscopy, wafer context, and domain knowledge. Knowledge graphs and RAG help make messy engineering knowledge usable now, while time-series models and RL are explored cautiously because control requires high accuracy at critical transitions.

Models used: LLMs · RAG systems · multimodal foundation models · time-series foundation models · PatchTST · TimeLLM · TimeGPT · reinforcement learning

Scale

• Advanced logic processes can require close to 1,000 process steps.
  “for example five nanometer technology in terms of logic how many process steps can you guys guess yeah close to 1,000”transcript, stated
• Yield needs to be at least about 80% to be profitable.
  “controlled optimized properly so we are talking about yield at least 80% to be profitable is really don't way to go so”transcript, stated
• Trace signals are sampled every second.
  “equipment Behavior the excess is the time so usually one second every second there are points going up and down the”transcript, stated
• One percent naming errors compound across 100 chambers and 1,000 steps.
  “99% time is accurate and you miss only 1% error then imagine that there are 100 different chambers and then there's thousand different steps”transcript, stated

Alternatives considered (roads not taken)

Manual step/window statistics and domain-expert configuration

Rejected because: It requires heavy manual expertise and cannot scale across all signals and steps.

Strict human-enforced sensor naming discipline

Rejected because: Even rare naming mistakes create huge accumulated alignment work across chambers and process steps.

Use time-series foundation models directly for live control

Rejected because: The models are not yet accurate enough at microsecond/critical transition behavior for control.

Patterns

rag-retrieval (knowledge)

The engineering assistant retrieves historical error context and suggestions from manufacturing knowledge sources.

tool-use-orchestration (manufacturing)

The system orchestrates sensor traces, event histories, metrology, images, and knowledge graphs rather than a single data source.

data-quality-loop (data)

Knowledge-graph alignment repairs inconsistent sensor naming instead of relying only on perfect human data entry.

fine-tuning-adaptation (modeling)

Samsung is exploring semiconductor-specific training/fine-tuning for foundation models that have not seen industrial time-series data.

human-in-the-loop-gate (operations)

The autonomous-fab framing explicitly augments engineers rather than removing them from control decisions.

Tradeoffs

Use AI to augment fab engineers

Chose autonomous-fab assistance with humans behind the scenes over replacing process engineers outright.

Rationale: The stated goal is to make engineers more powerful while freeing them from manual knobs and repetitive analysis.

Adopt knowledge graphs for alignment

Chose automatic relationship-based alignment over manual naming-rule enforcement alone.

Rationale: A one-percent naming error becomes large across many chambers and steps.

Gate control uses until prediction quality improves

Chose diagnosis/forecasting exploration before live process control over premature closed-loop control with current time-series models.

Rationale: Control requires accurate forecasts in the critical windows where current models lag.

Gotchas (what bit them)

Second-by-second trace signals are too large and complex for manual monitoring.

Root cause: Even a small etch recipe emits many sensor readings across windows and steps.

Resolution: Samsung combines summarization/windowing today with multimodal and time-series models for richer analysis.

Sensor names are not consistently aligned across chambers and tools.

Root cause: Humans make small naming errors and different perspectives need different relationships.

Resolution: Knowledge graphs and LLMs align equivalent sensors automatically.

Time-series foundation models are not yet control-ready.

Root cause: Industrial microsecond-level process-control data differs from what general time-series models have seen.

Resolution: Samsung treats them as an active collaboration and research area before using them for control.

Manufacturing data cannot simply be shared externally for model development.

Root cause: Fab data is sensitive and every fab/vendor may use different formats.

Resolution: Samsung describes building its own cloud and emphasizes data standardization.

Evaluation

Samsung evaluates by comparing manufacturing usefulness: engineer-assistant answers, sensor-alignment quality, OES/defect detection usefulness, time-series forecast accuracy in critical windows, and whether models can support advanced process control.

Metrics: close to 1,000 process steps for 5nm logic · 80% yield profitability target · second-by-second trace data · 100 chambers and 1,000 steps compounding naming errors · microsecond-level time-series control requirements

Outcomes

• Samsung already has an LLM/RAG-style engineering assistant for equipment-error diagnosis.
• Knowledge graph plus LLM alignment addresses sensor-naming inconsistency across chambers and wafers.
• Multimodal analysis supports OES and defect-detection use cases.
• Time-series foundation models and reinforcement learning remain active paths toward future autonomous control rather than finished production control.

The transcript provides concrete operating context: roughly 1,000 steps for advanced logic, 80% yield requirements, second-by-second trace data, and data-standardization/privacy constraints.