Evolution from Open-Ended LLM Agents to Guided Workflows

lindy.ai · Tech · 2024 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: no-code business agent workflow reliability

Lindy.ai needed to make no-code agents reliable and understandable after an early open-ended prompt-plus-tools design proved too unconstrained for production business workflows.

Constraints:

Lindy 1.0 relied on a giant prompt and tools, leaving users to hope the model used them correctly.
Most users do not have a mental model for turning text prompts into agent data structures.
Business workflows require reliable API integrations, permissions, and predictable action ordering.
Memory needs pruning because not all stored facts are useful.
User feedback and confirmations need to improve future behavior without making every workflow fully autonomous.

Architecture

Lindy 2.0 moved agents onto visual rails. Users build workflows with explicit triggers, actions, structured-output fields, model choices per node, permissions, and confirmation gates. Each node can act as a small agent with access to prior ledger context, while the overall workflow remains closer to deterministic if-this-then-that software. Memory, cross-agent calls, and a poor-man's RLHF loop use user corrections to retrieve better examples later.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  visual-workflow-builder["Visual workflow builder"] --> workflow-trigger["Workflow trigger"]
  workflow-trigger --> mini-agent-nodes["Mini agent nodes"]
  mini-agent-nodes --> append-only-ledger["Append-only ledger"]
  append-only-ledger --> confirmation-and-feedback-loop["Confirmation and feedback loop"]
  confirmation-and-feedback-loop --> memory-and-example-retrieval["Memory and example retrieval"]
  memory-and-example-retrieval --> mini-agent-nodes
  api-integrations["API integrations"] --> mini-agent-nodes

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Visual workflow builder`	No-code UI for configuring triggers, actions, models, permissions, and prompts.	`Workflow trigger`
`Workflow trigger`	Starts a Lindy workflow from an event such as email, meeting, Zendesk ticket, or schedule.	`Mini agent nodes`
`Mini agent nodes`	Per-node LLM calls that extract fields, generate replies, summarize meetings, or perform workflow actions.	`Append-only ledger`
`Append-only ledger`	Carries accumulated workflow context to later nodes.	`Confirmation and feedback loop`
`Confirmation and feedback loop`	Lets users approve, deny, edit, and save corrections as future examples.	`Memory and example retrieval`
`Memory and example retrieval`	Stores useful user corrections and memories for retrieval into future workflow context.	`Mini agent nodes`
`API integrations`	Deterministic tool/API actions for email, calendar, Slack, Google, GitHub, and other connectors.	`Mini agent nodes`

Data flow

A workflow trigger fires, deterministic steps and mini-agent nodes execute in order, each node inherits context from an append-only ledger, structured output fills fields, optional confirmation gates capture user edits, those edits can be embedded and retrieved for future tasks, and API integrations perform the actual business action.

“every node keeps appending to that ledger and every filled node inherits all the context built up by all the previous nodes.”article, stated

“this node here, this append row node is a mini agent. It can see everything that just happened.”article, stated

Why this architecture

Lindy chose guided workflows because rails improve reliability and usability. Instead of asking a model to infer when to call a knowledge base or API, workflow nodes make required actions explicit, while the LLM is used narrowly where interpretation or generation is actually useful.

“the more you can put your agent on Rails, one, the more reliable it's going to be, obviously, but two, it's also going to be easier to use for the user”article, stated

“we talk about using as much AI as necessary and as little as possible.”article, stated

Models used: Claude 3.5 Sonnet · GPT-4 Turbo · Structured output calls · Vector database examples · Workflow nodes · API integrations

Scale

Lindy started when context windows were about 4,000 tokens.
“we started building this when the context windows were like 4,000 tokens.”article, stated
Lindy 2.0 had been rebuilt over about six months and exposed to users for about two months.
“We called it Lindy 2.0. I would say we started working on that six months ago.”article, stated
Some workflows can be represented as two steps, such as email reservation confirmation to spreadsheet row.
“It's two steps. It's like, okay, I receive an email. If it is a reservation confirmation, I have this filter here.”article, stated

Alternatives considered (roads not taken)

A giant prompt field with tools.

Rejected because: Users had to rely on the model to invoke the right tool at the right time.

“the first version of Lindy was like just a giant prompt and a bunch of tools.”article, stated

Pure text-to-agent interface.

Rejected because: Many user prompts are ambiguous, and GUI workflows give users a clearer structure for configuring agents.

“60 or 70% of the prompts that people type don't mean anything.”article, stated

Use computer use instead of APIs.

Rejected because: Flo argues API use is far more reliable and performant for actions that APIs can handle.

“anything that can be done with an API must be done by an API or should be done by an API for a very long time.”article, stated

Patterns

`tool-use-orchestration` (workflow nodes)

Lindy turns agent behavior into explicit triggers and action nodes rather than open-ended tool selection.

“you can actually have the trigger, which is Zendesk ticket received, have the knowledge base consult, which is always there, and then have the agent.”article, stated

`structured-output-contract` (node fields)

Workflow nodes fill structured fields such as reservation date, amount, and travelers.

“Each of these fields is a prompt.”article, stated

`human-in-the-loop-gate` (confirmation)

Users can require confirmation before a workflow step executes, then approve, deny, or edit the output.

“ask me for confirmation before you actually execute this step.”article, stated

`agent-memory` (cross-execution memory)

Lindy stores and prunes memories and retrieves prior user corrections for future tasks.

“Lindy can just manage her own memory and decide when she's remembering things between executions.”article, stated

Tradeoffs

Constrain agents through workflows.

Chose Visual rails and explicit actions over Maximum open-ended model autonomy.

Rationale: Narrow workflow boxes improve reliability and make the setup easier for users.

“put Shoggoth in a box and make it a very small, the minimal viable box.”article, stated

Use APIs for actions.

Chose API integrations over Computer-use-style UI control where an API exists.

Rationale: The team sees API use as much more reliable and performant for business workflows.

“when you're talking about modules and magnitude of differences in terms of performance and reliability, which is what we are talking about when we're talking about API use versus computer use”article, stated

Ask for incremental permissions.

Chose Least permissions needed at a given moment over Requesting broad connector access upfront.

Rationale: Granular authorization improves control but can make setup more painful.

“The general philosophy is we ask for the least amount of permissions needed at any given moment.”article, stated

Gotchas (what bit them)

Open-ended prompts made tool use unreliable.

Root cause: The model might not invoke a required knowledge base or action even when the user intended it.

Resolution: Make required actions explicit workflow nodes.

“You're praying the LLM gods that they will actually invoke your knowledge base”article, stated

Memory can fill with low-quality facts.

Root cause: Assistants may save trivial or useless facts unless memory is actively pruned.

Resolution: Prefer fewer high-quality memories over thousands of low-value ones.

“I'd much rather have high-quality facts that it accepts.”article, stated

Customer-support agents can hallucinate external links.

Root cause: The support Lindy generated a nonexistent tutorial link instead of acknowledging missing material.

Resolution: Human checking caught the email before the customer opened it; guidelines and feedback can be updated.

“Lindy responded to the email. It's like, oh, absolutely, here's a link. And we were like, what? Like, what kind of link did you send?”article, stated

Evaluation

Lindy uses product constraints and user feedback more than a single described formal eval suite. The article mentions an internal eval tool, Braintrust consideration, confirmation gates, task history, logs, human checking, and a poor-man's RLHF loop that stores edits as future examples.

Metrics: Workflow reliability · Human confirmation edits · Task history correctness · Support hallucination incidents · Memory quality · Connector/action reliability

“our eval tool breaks all the time and it's a nightmare to maintain.”article, stated

“when you make a change, we are then saving this change that you're making or embedding it in the vector database.”article, stated

Outcomes

Lindy 2.0 replaced the big-prompt approach with guided visual workflows.
The platform became easier to set up and more reliable according to the founder.
The model layer stopped being the main bottleneck; integrations, capabilities, UX, and product engineering became the limiting factors.

The article gives product-evolution timing and qualitative scale rather than exact user counts: six months rebuilding Lindy 2.0, two months in users' hands, 4K-token starting context windows, and a system prompt larger than 4K tokens.

“The agent is way more reliable, way easier to set up, and we're already seeing a ton of new use cases pop up.”article, stated

“It is actually shocking the extent to which the model is no longer the limit.”article, stated