Automating Enterprise Workflows with Foundation Models in Healthcare

various · Healthcare · 2023 · source Agentic: yes · promotional bias: low · summary-only (no fetched source) · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Healthcare enterprise workflow automation

Stanford researchers wanted to automate healthcare enterprise workflows in systems such as Epic EHR without requiring brittle API integration or traditional RPA scripts.

Constraints:

• healthcare workflows involve sensitive patient data
• enterprise workflows use complex dynamic UIs
• traditional RPA struggles with variations and backend/API access
• long-horizon workflows require validation and auditability
• source cache is unavailable, so this extraction is summary-only

Architecture

The aCLAr system uses a Demonstrate, Execute, Validate framework. In Demonstrate, multimodal foundation models observe human task demonstrations and generate standard operating procedures. In Execute, the model navigates enterprise UIs visually, including virtualized Epic EHR environments, without backend integration. In Validate, the system self-monitors, checks for errors, and produces audit trails for completed workflows.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  demonstration-observer[Demonstration observer] --> sop-generator[SOP generator]
  sop-generator --> ui-execution-model[UI execution model]
  ui-execution-model --> epic-ehr-interface[Epic EHR interface]
  ui-execution-model --> validation-monitor[Validation monitor]

Components

Data flow

Human demonstrations are passively observed and converted into natural-language SOPs with GPT-4 visual understanding. During execution, the model uses SOP context and UI observations to operate the application. Validation layers monitor progress, correct errors, and record an audit trail.

Why this architecture

A visual, demonstration-based architecture lets the system operate in enterprise applications where backend access may be unavailable or undesirable. SOP generation translates human demonstrations into reusable task specifications, while validation compensates for long-horizon workflow risk.

Models used: GPT-4 · multimodal foundation models

Scale

• The summary says 92% of jobs require digital skills.
  “• Enterprise workflows are central to the modern economy with 92% of jobs requiring digital skills • Workers spend approximately 3 hours per day on repetitive tasks”summary, stated
• Workers spend about three hours per day on repetitive tasks.
  “• Workers spend approximately 3 hours per day on repetitive tasks • Current RPA solutions have significant limitations:”summary, stated
• The Demonstrate phase reportedly reached 95% workflow-understanding accuracy.
  “• Achieves 95% accuracy in workflow understanding metrics ### Execute Phase • Models interact with UIs in a human-like manner”summary, stated

Alternatives considered (roads not taken)

Traditional RPA

Rejected because: The summary says current RPA solutions have significant limitations for these workflows.

API or backend integration

Rejected because: The system was designed to work through visual UI interaction without API integration.

Patterns

tool-use-orchestration (agent-architecture)

The model interacts with enterprise UI tools visually instead of relying on APIs.

observability-tracing (operations)

The Validate phase produces audit trails and supports self-monitoring.

human-in-the-loop-gate (workflow)

The system learns from human demonstrations and escalates oversight where needed.

Tradeoffs

Use visual UI interaction

Chose foundation-model UI navigation over API integration.

Rationale: Visual interaction is more flexible in enterprise systems where APIs are absent, inaccessible, or insufficient.

Generate SOPs from demonstrations

Chose demonstration-derived natural-language SOPs over hand-coded procedural scripts.

Rationale: Human demonstrations supply task knowledge without requiring developers to encode every workflow step.

Gotchas (what bit them)

Long-horizon UI workflows remain difficult.

Root cause: Workflow variations, context windows, and dynamic UIs complicate execution.

Resolution: The summary calls for enhanced error handling, better long-workflow handling, and validation layers.

The primary failure mode is inaction.

Root cause: The system may fail to proceed rather than taking an incorrect action.

Resolution: Validation and self-monitoring layers are used to detect and correct failures.

Evaluation

The summary reports workflow-understanding accuracy, a healthcare Epic EHR demonstration, and validation/audit behavior rather than broad production metrics.

Metrics: workflow understanding accuracy · successful Epic EHR workflow automation · validation and audit trail generation · execution speed

Outcomes

• The system demonstrated automation of a healthcare workflow in Epic EHR.
• The Demonstrate phase reportedly achieved 95% workflow-understanding accuracy.
• The implementation is slower than humans today but has parallelization opportunities.

Summary-only evidence gives broad labor-context numbers but limited deployment-scale data.