Rapid Integration of Advanced AI Models through Modular Architecture and Workflow Orchestration

harvey · Legal · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Legal AI platform model integration and workflow orchestration

Harvey needed to translate OpenAI's new Deep Research capability into a secure, reliable legal AI product quickly, despite sparse documentation, long-running streamed API behavior, and compatibility requirements with Harvey's workflow and transparency systems.

Constraints:

• The Deep Research API was brand new and not in base LLM knowledge.
• Streaming mode was a long-running operation with more than 20 dynamic output types.
• Legal workflows require trust, citations, transparency, and user oversight.
• The integration needed to fit Harvey's Workflow Engine, thinking states, and citation system.

Architecture

Harvey integrated Deep Research through its AI-native platform architecture. A modular Workflow Engine composes model systems and agent behavior from low-code AI building blocks; thinking states expose plans, intermediate results, and user interventions; the citation engine incorporates Deep Research URL citations; and engineers used AI-assisted coding with full streamed-output logs in pydantic and JSON formats as context for humans and coding assistants. This let Harvey incorporate Deep Research less than 12 hours after OpenAI's API release.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Deep Research API"] --> B["Streamed API output logs"]
  B --> C["AI coding assistant"]
  C --> D["Engineering iteration loop"]
  D --> E["Workflow Engine"]
  A --> E
  E --> F["AI building blocks"]
  E --> G["Thinking states"]
  E --> H["Citation engine"]
  E --> I["Harvey product surfaces"]
  G --> J["Harvey users"]
  H --> J
  I --> J

Components

Data flow

The Deep Research API emits long-running streamed events. Engineers dump those events into pydantic and JSON logs, use them as shared context for humans and AI coding assistants, implement compatible Workflow Engine blocks and thinking states, pass Deep Research actions through Harvey's orchestration layer, and attach Deep Research URL citations into Harvey's citation system so workflow steps can be traced and verified.

Why this architecture

Harvey could integrate a new model capability quickly because its platform had already invested in modular AI-native primitives: Workflow Engine orchestration, composable building blocks, thinking states, citations, and AI-assisted development practices. Those abstractions turned a brand-new API into another tool/workflow component rather than a one-off product rewrite.

Models used: OpenAI Deep Research · AI-assisted coding tools

Scale

• Deep Research was integrated less than 12 hours after OpenAI released the API.
  “less than 12 hours after it was released by OpenAI.”article, stated
• The API streaming mode had more than 20 dynamic output types.
  “Streaming mode for the API was a long-running operation with 20+ dynamic output types”article, stated
• The platform includes multiple existing product surfaces such as Workflows and Workflow Builder.
  “This engine underpins both their Workflows product and their newly released Workflow Builder”summary, stated

Alternatives considered (roads not taken)

Wait for stable docs and broad LLM knowledge before integrating

Rejected because: Harvey wanted to understand and use the API immediately despite little documentation.

Treat Deep Research as an isolated feature

Rejected because: The integration needed compatibility with Workflow Engine and thinking states.

Hide model reasoning and intermediate work from users

Rejected because: Harvey treats transparency and user intervention as core legal-workflow requirements.

Patterns

tool-use-orchestration (orchestration)

The Workflow Engine lets model systems compose blocks and lets agents call building blocks as tools.

context-engineering (context)

Engineers dumped streamed API outputs into pydantic and JSON logs as context for humans and AI coding tools.

human-in-the-loop-gate (workflow)

Thinking states let users follow, evaluate, intervene, tweak parameters, and rerun actions.

observability-tracing (trust)

Harvey incorporated Deep Research URL citations into its citation system so workflow steps could be traced and verified.

Tradeoffs

Build on modular platform primitives.

Chose Workflow Engine and composable AI building blocks over One-off Deep Research integration.

Rationale: The modular architecture reduced development time and minimized regression risk.

Expose thinking states to users.

Chose Visible plans, intermediate results, and user interventions over Opaque long-running model execution.

Rationale: Legal users need to evaluate intermediate work and course-correct before final output.

Use AI-assisted development with explicit API context.

Chose Engineer-facilitated AI coding loop grounded in event logs over Manual integration without AI tools or relying on model prior knowledge.

Rationale: The API was new, so the coding assistant needed concrete streamed-output context.

Gotchas (what bit them)

New AI APIs may be absent from model knowledge and documentation.

Root cause: Deep Research had just launched, so base LLMs and online docs did not fully describe it.

Resolution: Use full streamed-output logs as context for humans and AI coding assistants.

Streaming APIs can hide complex dynamic behavior.

Root cause: The streaming mode was long-running and had more than 20 output types.

Resolution: Explore event logs iteratively and adapt them into Harvey's Workflow Engine and thinking states.

Legal AI users need verification, not just generated answers.

Root cause: Complex legal work requires trust and auditability.

Resolution: Incorporate URL citations and thinking states so each step can be traced, verified, and corrected.

Evaluation

Harvey's reported validation centers on integration readiness: event-log exploration, compatibility with internal Workflow Engine/thinking states, successful citation incorporation, internal testing, and planned early access.

Metrics: Less than 12-hour integration time · 20+ dynamic output types handled · URL citations incorporated · Internal testing · Early access readiness

Outcomes

• Harvey integrated Deep Research less than 12 hours after OpenAI released the API.
• Deep Research citations were incorporated into Harvey's citation system.
• The integration reinforced the value of AI-native architecture and iterative development culture.
• The feature was undergoing internal testing with broader early access planned.

The source reports a less-than-12-hour integration, 20+ dynamic streaming output types, multiple product surfaces under Workflow Engine, and a staged release from internal testing to early access.