Building a Memory System for No-Code Agent Development

langchain · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: No-code agent memory

LangChain needed LangSmith Agent Builder agents to improve across repeated task sessions without requiring citizen developers to manually edit complex agent configuration or repeat preferences every time.

Constraints:

• Agent Builder targets technically lite users who need no-code configuration
• agents repeat the same workflow many times, making cross-session learning valuable
• memory edits can create prompt-injection risk and invalid configuration files
• models add specific memories more easily than they compact or generalize them
• MCP servers can expose too many tools and overflow context

Architecture

LangSmith Agent Builder represents agent memory as files exposed through a virtual filesystem. Procedural memory lives in AGENTS.md and tools.json, semantic memory lives in agent skills and other knowledge files, and subagents follow a Claude Code-like convention. The files are stored in Postgres but exposed to the agent as a filesystem because models work well with files. Deep Agents supplies the harness for planning, summarization, and tool-output offloading. Memory edits happen in the hot path, pass schema validation where needed, and require explicit human approval unless users enable a riskier yolo mode.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  langsmith-agent-builder[LangSmith Agent Builder] --> deep-agents-harness[Deep Agents harness]
  deep-agents-harness --> agent-runtime[Agent runtime]
  agent-runtime --> virtual-filesystem-memory[Virtual filesystem memory]
  postgres-storage[Postgres storage] --> virtual-filesystem-memory
  virtual-filesystem-memory --> agents-md[AGENTS.md]
  virtual-filesystem-memory --> tools-json[tools.json]
  virtual-filesystem-memory --> agent-skills[Agent skills]
  virtual-filesystem-memory --> subagent-files[Subagent files]
  virtual-filesystem-memory --> knowledge-files[Knowledge files]
  agent-runtime --> validation-step[Validation step]
  tools-json --> validation-step
  agent-skills --> validation-step
  validation-step --> human-approval-gate[Human approval gate]
  human-approval-gate --> virtual-filesystem-memory

Components

Data flow

A builder creates an agent through LangSmith Agent Builder. The agent runs on the Deep Agents harness and reads its virtual filesystem memory: AGENTS.md for core instructions, tools.json for selected MCP tools, skills and knowledge files for semantic context, and optional subagent files. During work, the agent proposes edits to those files based on user corrections or task discoveries. Custom file shapes are validated; failures are returned to the LLM rather than committed. Approved edits update Postgres-backed memory and affect future sessions.

Why this architecture

LangChain chose file-shaped memory because LLMs are good at using filesystems and no-code builders can understand Markdown/JSON better than a custom DSL. Storing those files in Postgres gives infrastructure efficiency while preserving an agent-friendly interface. Human approval and validation control the risks of invalid memory and prompt injection.

Models used: Deep Agents · LangSmith Agent Builder agents · LLMs capable of filesystem-style context editing

Scale

• The example LinkedIn recruiter subagent sources about 50 candidates after calibration.
  “linkedin_search_worker: after the main agent is calibrated on a search, it will kick off this agent to source ~50 candidates.”article, stated
• The example agent currently has three other memory files representing job descriptions.
  “* There are also currently 3 other files in the memory, representing JDs for different candidates. As we’ve worked with the agent on these searches, it has updated and maintained those JDs.”article, stated
• LangChain had one person working full time on prompting for memory.
  “* The agent did not know the right format for skills files * … many more We had one person working full time on prompting for memory (which was a large percentage of the team). Validate file types”article, stated
• A proposed future background memory job would likely run once a day.
  “We want to add a process that runs in the background (probably some cron job, running once a day or so) to reflect over all conversations and update memory.”article, stated

Alternatives considered (roads not taken)

Launch without memory

Rejected because: Task-specific agents repeat the same workflow often, so users would otherwise need to repeat preferences across sessions.

Use a real filesystem as the storage layer

Rejected because: The team wanted the model-facing shape of a filesystem but the operational efficiency and pluggability of database storage.

Expose full MCP server configuration through mcp.json

Rejected because: A full MCP server can expose too many tools, so Agent Builder uses tools.json to give only a subset of tools and avoid context overflow.

Commit generated memory files directly

Rejected because: Agents sometimes generated invalid custom files, so invalid files are returned to the LLM instead of committed.

Let memory update automatically by default

Rejected because: Prompt injection risk made explicit human approval the safer default, with yolo mode reserved for lower-risk cases.

Patterns

agent-memory (agent_architecture)

Agent Builder stores procedural and semantic memory as editable files that the agent reads and updates across repeated workflow sessions.

context-compaction (context_management)

Deep Agents abstracts summarization and tool-result offloading, and the memory lessons explicitly identify compaction as a needed behavior.

structured-output-contract (validation)

Custom memory files have expected schemas or frontmatter, and Agent Builder validates them before committing edits.

human-in-the-loop-gate (control)

Every memory edit requires explicit human approval by default to reduce prompt-injection risk.

tool-use-orchestration (agent_architecture)

tools.json scopes MCP tool access, and Deep Agents provides the runtime for tool use without a custom no-code DSL.

prompt-versioning (prompt_operations)

AGENTS.md accumulates user corrections into durable instruction files, turning iterative natural-language feedback into updated agent specification.

Tradeoffs

Prioritize memory in the first Agent Builder experience

Chose Cross-session file-based memory over Launching initially without memory.

Rationale: Task-specific agents repeat the same work, so preferences learned in one session are much more likely to apply to the next.

Expose database-backed memory as files

Chose Postgres-backed virtual filesystem over A real filesystem or specialized memory API.

Rationale: LLMs work well with filesystem metaphors, while database storage is easier and more efficient operationally.

Leave episodic memory out of the initial launch

Chose Procedural and semantic memory first over Adding episodic memory before launch.

Rationale: The team judged episodic memory less important for these task-specific agents and deferred it to future work.

Require human approval for memory edits

Chose Human-in-the-loop memory updates over Automatic memory writes by default.

Rationale: The safer default reduces prompt-injection risk, while yolo mode lets lower-risk users opt out.

Gotchas (what bit them)

Prompting memory behavior was the hardest part.

Root cause: The agent needed nuanced rules for when to remember, where to write memory, and how to format specialized files.

Resolution: Invest dedicated prompting effort and refine prompts for remembering, not remembering, AGENTS.md usage, and skills-file formatting.

Agents generated invalid memory files.

Root cause: Files like tools.json and skills require schemas or frontmatter that the agent sometimes forgot.

Resolution: Validate custom file shapes and return validation errors to the LLM before committing.

Agents over-add specific memories instead of compacting them.

Root cause: The agent was better at appending concrete cases than generalizing them into broader rules.

Resolution: Prompt users or future background processes to reflect, compact, and generalize memory.

Full MCP server exposure can overflow context.

Root cause: An MCP server may expose more tools than the agent needs for a workflow.

Resolution: Use a custom tools.json to expose only selected tools.

Automatic memory editing expands prompt-injection risk.

Root cause: Memory updates can persist malicious or unwanted instructions if accepted blindly.

Resolution: Require explicit human approval for all memory edits by default.

Evaluation

The article reports qualitative development evaluation through prompt debugging, schema validation, user corrections, and observed failure modes rather than a numeric benchmark suite.

Metrics: whether the agent remembers when it should · whether the agent avoids remembering when it should not · whether AGENTS.md, skills, and tools.json are edited in the right formats · validation pass/fail for custom memory files · human approval of proposed memory edits

Outcomes

• Agent Builder can accumulate task preferences through natural-language corrections rather than manual configuration edits.
• The memory system supports no-code agent building using familiar Markdown and JSON files.
• Agents become more portable because their configuration lives in conventional file formats.
• The initial system supports procedural and semantic memory, while episodic, background, slash-command, semantic-search, and multi-level memory are planned future work.

The source provides product and example scale rather than deployment counts: a LinkedIn recruiter subagent sources about 50 candidates, an example memory folder has three job-description files, and one team member worked full time on memory prompting.