Evolving AI Coding Agent Workflows from Research-Plan-Implement to CRISPY

humanlayer · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: AI coding-agent workflow methodology

HumanLayer needed to improve its Research-Plan-Implement workflow because long monolithic prompts, unreadable plan files, magic words, and poor human-agent alignment produced inconsistent production coding-agent results.

Constraints:

• RPI prompts were widely used, from small startups to enterprise teams.
• Frontier models could only follow roughly 150-200 instructions consistently.
• An 85-instruction planning prompt plus project instructions and tools overloaded adherence.
• Thousand-line plans were as much work to review as code and could diverge from implementation.
• Teams needed leverage without outsourcing engineering judgment or skipping code review.

Architecture

HumanLayer replaced RPI with CRISPY-style staged planning: questions, research, design, structure outline, plan, work tree, implement, and pull request. Instead of one 85+ instruction mega-prompt, the workflow uses smaller prompts under about 40 instructions, explicit control flow outside prompts, design discussions, structure outlines, vertical plans, and human alignment checkpoints before implementation.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  question-stage["Question stage"] --> research-stage["Research stage"]
  research-stage --> design-discussion["Design discussion"]
  design-discussion --> structure-outline["Structure outline"]
  structure-outline --> plan-stage["Plan stage"]
  plan-stage --> work-tree-stage["Work tree stage"]
  work-tree-stage --> implementation-stage["Implementation stage"]
  implementation-stage --> question-stage

Components

Data flow

A task starts with questions, then research, design discussion, structure outline, plan, work tree, implementation, and PR. Each stage narrows the next artifact, the human reviews shorter design/outline docs for alignment, control flow routes work between focused prompts, and deep review is saved for the generated code rather than enormous plans.

Why this architecture

HumanLayer moved control flow out of prompts because models were overloaded by monolithic instructions. Shorter artifacts create alignment earlier, vertical planning provides testable checkpoints, and engineers keep ownership by reading code rather than trusting large generated plans.

Models used: Claude Code · Claude Opus 4.5 · RPI prompts · CRISPY workflow · Focused prompt stages · Human code review

Scale

• About 10,000 people had grabbed HumanLayer's open-source RPI prompts.
  “There's probably 10,000 people who have gone to our open source and grabbed our prompts”transcript, stated
• HumanLayer worked with thousands of engineers from startups through enterprise.
  “we've basically worked with thousands of engineers from tiny startups all the way”transcript, stated
• The speaker estimates frontier LLMs follow about 150-200 instructions with good consistency.
  “about 150 to 200 instructions with like good consistency.”transcript, stated
• The old planning prompt had 85 or more instructions; the new stage prompts are under 40.
  “single mega prompt with 85 instructions.”transcript, stated
• A thousand-line plan often corresponds to roughly a thousand lines of code within about 10%.
  “a thousand line plan tends to be about a thousand lines of code within 10% or so”transcript, stated

Alternatives considered (roads not taken)

Keep Research-Plan-Implement as a monolithic workflow.

Rejected because: The old prompt had too many instructions, depended on magic words, and produced plans people did not reliably read.

Review long plan files deeply.

Rejected because: Long plans can be as much work as code and still diverge when implementation begins.

Use prompts for control flow.

Rejected because: Classification and external control flow are more reliable than asking one prompt to branch across many actions.

Patterns

prompt-chaining (CRISPY workflow)

HumanLayer decomposes a large planning prompt into focused stages with fewer instructions and clearer handoffs.

human-in-the-loop-gate (alignment checkpoints)

Engineers review design discussions and structure outlines before code, then reserve deep review for actual code.

eval-driven-development (prompt/process validation)

HumanLayer tried prompting and evals against horizontal planning behavior, then adjusted workflow structure when prompts were insufficient.

context-compaction (shorter artifacts)

Structure outlines compress plan review into shorter interface-like documents before full implementation.

Tradeoffs

Read code instead of long plans.

Chose Deep review on generated code over Deep review of thousand-line plans.

Rationale: Plans can diverge from implementation, so the same review effort is better spent on code.

Split planning into design, outline, and plan.

Chose Short staged alignment artifacts over One giant planning artifact.

Rationale: Shorter docs let teammates re-steer before code exists and make later review lighter.

Use vertical plans.

Chose End-to-end slices with checkpoints over Horizontal layer-by-layer plans.

Rationale: Vertical slices create checkpoints where the human or model can verify behavior before continuing.

Gotchas (what bit them)

Magic words hide workflow requirements.

Root cause: Users had to know to ask for open questions and outline before planning to get good behavior.

Resolution: Make questions and outlines explicit workflow stages.

Overloaded prompts reduce adherence.

Root cause: An 85-instruction planning prompt plus project files, system prompt, tools, and MCP exceeds reliable instruction following.

Resolution: Split into smaller prompts under about 40 instructions.

Horizontal plans delay feedback until too much code exists.

Root cause: The model plans database, services, API, and frontend in separate layers, leaving no working checkpoint.

Resolution: Use vertical plans with testable slices and pauses for fixes.

Evaluation

HumanLayer evaluates the workflow through prompt adherence, plan/code review burden, horizontal-plan failure behavior, human-agent alignment quality, code review/rework time, and production responsibility for generated code.

Metrics: Instruction count per prompt · Prompt adherence · Plan length · Structure outline length · Code review/rework time · Human alignment before code · Vertical checkpoint success

Outcomes

• RPI was replaced by a staged CRISPY-style process for research and planning.
• Instruction-heavy prompts were split into smaller stages under about 40 instructions.
• Design discussions and structure outlines became alignment artifacts before code.
• The methodology emphasizes reading code, not outsourcing engineering judgment to plans.

The talk cites 10,000 prompt users, thousands of engineers, 150-200 reliable instructions, an 85-instruction old prompt, less-than-40-instruction new prompts, and thousand-line plans matching roughly thousand-line code changes.