Building a Public AI Agent Workspace for Organizational Learning

shopify · E-commerce · 2026 · source Agentic: yes · promotional bias: low · summary-only (no fetched source) · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Organization-wide coding-agent adoption and learning

Shopify needed its River coding agent to improve developer productivity while also spreading prompt patterns, debugging techniques, and agent skills across the organization instead of hiding every AI interaction in private one-off sessions.

Constraints:

• Private AI assistant interactions do not create shared organizational learning
• The agent needed to operate in existing Slack workflows
• The system needed to support coding tasks, tests, traces, data warehouse queries, and pull requests
• Knowledge diffusion was an explicit design goal
• River needed enough transparency for employees to observe and learn from each other
• The fetched social source is only a preview card, so extraction must rely on the ZenML summary

Architecture

Shopify built River as an AI coding agent that works only in public Slack channels. Employees invoke River in channels rather than DMs; River can read code, run tests, write code, open pull requests, query data, examine traces, and execute development tasks. Public threads become a teaching-workshop environment where others observe prompts, add context, react, assist with reviews, and learn from successful and failed agent interactions.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  shopify-employee[Shopify Employee] --> public-slack-channel[Public Slack Channel]
  observers[Observers] --> public-slack-channel
  public-slack-channel --> river-agent[River Agent]
  river-agent --> codebase[Codebase]
  river-agent --> test-runner[Test Runner]
  river-agent --> pull-request-system[Pull Request System]
  codebase --> pull-request-system
  river-agent --> data-warehouse[Data Warehouse]
  river-agent --> production-traces[Production Traces]
  skills-and-instructions[Skills and Instructions] --> river-agent
  river-agent --> observers

Components

Data flow

An employee mentions River in a public Slack channel. River performs development work against Shopify systems, such as code reading, test execution, pull-request creation, data warehouse queries, and trace inspection. Other employees can watch the thread, add context, review work, and reuse prompting/debugging patterns. Team-level refinements to River's skills and instructions improve future behavior.

Why this architecture

The core architectural choice is social rather than merely technical: River is public-only so every useful or failed interaction becomes inspectable training material for the organization. Running inside Slack reduces workflow switching, while broad development-tool integrations make River a real teammate rather than a Q&A bot.

Models used: AI coding agent

Scale

• In 30 days, 5,938 Shopify employees engaged with River.
  “The scale of deployment is substantial: in a single 30-day period, 5,938 Shopify employees engaged with River across 4,450 different Slack channels.”summary, stated
• River was used across 4,450 Slack channels in that period.
  “The scale of deployment is substantial: in a single 30-day period, 5,938 Shopify employees engaged with River across 4,450 different Slack channels. River opened 1,870 pull requests in just one week in Shopify's main monorepo, representing approximately one in eight merged pull requests.”summary, stated
• River opened 1,870 pull requests in one week in Shopify's main monorepo.
  “River opened 1,870 pull requests in just one week in Shopify's main monorepo, representing approximately one in eight merged pull requests. This indicates that River is not a peripheral tool but rather a core contributor integrated into the company's primary development workflow.”summary, stated
• River authored about one in eight merged pull requests.
  “River opened 1,870 pull requests in just one week in Shopify's main monorepo, representing approximately one in eight merged pull requests. This indicates that River is not a peripheral tool but rather a core contributor integrated into the company's primary development workflow.”summary, stated
• River's merge rate improved from 36% to 77% over two months.
  “The agent's merge rate improved from 36% to 77% over two months through collective learning and iterative refinement of River's skills and instructions by teams across the company.”summary, stated
• Tobi Lütke's River working channel had over 100 observers.
  “The CEO himself works with River in a channel called #tobi-working-with-river, with over 100 people who observe, react to threads, add context, pick up work, assist with reviews, and importantly, learn by watching.”summary, stated

Alternatives considered (roads not taken)

Private AI assistant interactions

Rejected because: Private sessions do not create organizational knowledge diffusion or observable learning.

Direct-message operation

Rejected because: River explicitly declines DMs and pushes users into public channels to preserve the teaching-workshop model.

Patterns

tool-use-orchestration (coding-agent)

River uses code, test, PR, warehouse, and trace tools to participate in development work.

human-in-the-loop-gate (collaborative-development)

Public threads let employees observe, react, add context, pick up work, and assist with reviews around River output.

agent-memory (organizational-learning)

River uses skills, zones, and memory-like knowledge structures to maintain and improve behavior across teams.

observability-tracing (debugging)

River can examine production traces as part of its development and debugging work.

Tradeoffs

Make River public-only

Chose public Slack channels over private assistant windows and DMs.

Rationale: Public operation creates learning, context sharing, review assistance, and prompt-pattern diffusion across the organization.

Use Slack as the workspace

Chose mention-based channel interactions over a new standalone agent interface.

Rationale: Slack was already the collaboration substrate, lowering adoption friction and making observation natural.

Gotchas (what bit them)

Private agent work hides learning opportunities.

Root cause: Only the direct user sees the prompt, agent behavior, debugging moves, and result.

Resolution: Shopify constrained River to public Slack channels.

Transparency creates a different security and collaboration posture.

Root cause: Public-only agent use optimizes for knowledge diffusion rather than individual privacy.

Resolution: River declines direct messages and nudges users into public collaboration spaces.

Evaluation

The summary reports production adoption and contribution metrics: employee/channel reach, PR volume, merged-PR share, and merge-rate improvement. The public-channel design also creates qualitative observation loops where employees learn and refine prompts/skills.

Metrics: employees engaged · Slack channels used · pull requests opened per week · share of merged pull requests · merge rate · observer count · skill/instruction refinement

Outcomes

• River became a public Slack-based coding agent used by thousands of Shopify employees.
• It opened 1,870 pull requests in one week in the main monorepo.
• It authored about one in eight merged pull requests.
• Its merge rate improved from 36% to 77% over two months.
• Public operation produced organizational learning through shared prompts, debugging, context, and reviews.

The summary reports 5,938 employees, 4,450 Slack channels, 1,870 PRs in one week, about one in eight merged PRs, over 100 observers in Tobi's working channel, and merge-rate improvement from 36% to 77%.