Building a Conversational AI Agent for Slack Integration

linear · Tech · 2025 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Slack-native project management agent

Linear wanted a Slack agent that could create issues and answer workspace questions without forcing users to switch tools, while keeping the LLM accurate, fast, and scoped inside Slack's interaction limits.

Constraints:

Slack gives fewer interaction surfaces than Linear's own product
too much Slack and Linear context can confuse the agent
too little thread context can make the agent miss important details
issue creation requires reliable business logic and formatting
Slack users expect fast responses and do not want unnecessary back-and-forth

Architecture

Linear built a Slack agent that starts with localized thread context and minimal Linear state, then uses an early intent split to route issue creation separately from other workspace questions. The issue path gets a narrower prompt and tools, while deterministic business logic and output formatting stay in application code instead of the LLM.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  slack-thread[Slack thread] --> context-window[Context window]
  slack-thread --> invocation-state[Invocation state]
  context-window --> intent-splitter[Intent splitter]
  intent-splitter --> issue-creation-path[Issue creation path]
  intent-splitter --> workspace-query-path[Workspace query path]
  invocation-state --> issue-creation-path
  invocation-state --> workspace-query-path
  issue-creation-path --> linear-tools[Linear tools]
  workspace-query-path --> linear-tools
  issue-creation-path --> business-logic-formatter[Business logic formatter]
  business-logic-formatter --> slack-thread

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Slack thread`	User-facing surface where the agent is invoked and receives recent conversation context.	`Context window`, `Invocation state`
`Invocation state`	Minimal Linear state from the Slack channel, such as issue, project, or initiative IDs.	`Slack thread`, `Workspace query path`, `Issue creation path`
`Context window`	Roughly ten recent Slack messages used as localized context.	`Slack thread`, `Intent splitter`
`Intent splitter`	Small early model that routes issue creation separately from other agent work.	`Context window`, `Issue creation path`, `Workspace query path`
`Issue creation path`	Scoped prompt and tools for creating Linear issues from Slack context.	`Intent splitter`, `Invocation state`, `Linear tools`, `Business logic formatter`
`Workspace query path`	General path for answering questions and aggregating Linear workspace data.	`Intent splitter`, `Invocation state`, `Linear tools`
`Linear tools`	APIs and search tools for issues, projects, initiatives, and workspace data.	`Issue creation path`, `Workspace query path`
`Business logic formatter`	Programmatic layer for deterministic defaults and formatting that should not be inferred by the LLM.	`Issue creation path`, `Slack thread`

Data flow

A Slack invocation passes the recent thread messages and synced Linear context into the agent. A small initial model classifies whether the user is creating an issue or asking for something else. The selected path uses scoped tools and prompts, creates or queries Linear data, and returns a response inside Slack.

3 supporting quotes

“its core, we wanted to meet our users where they're doing a lot of their work communication which is in Slack.”transcript, stated

“we decided to provide 10 or so messages in the beginning so that they could the agent starts off like thinking that it's the 11th message in a conversation.”transcript, stated

“when we took a really quick small model in the beginning and just said, are you creating an issue or are you doing like really anything else?”transcript, stated

Why this architecture

Linear narrowed context and split intents because a single broad prompt with many tools led the LLM to over-search and choose irrelevant tools. Programmatic handling kept deterministic business rules and formatting out of the LLM.

3 supporting quotes

“there's a really fine balance between providing the agent with enough information and context so that it can make an informed decision but not overloading and confusing it.”transcript, implied

“We were able to segment out like I would say half of the system prompt that was tailored around telling you exactly how to fill in different fields of an issue”transcript, stated

“Very specific business logic that doesn't change. that's something that we would want to um handle on our own.”transcript, stated

Models used: small intent model · LLM

Scale

The localized context window starts with about ten Slack messages.
“we decided to provide 10 or so messages in the beginning so that they could the agent starts off like thinking that it's the 11th message in a conversation.”transcript, stated
The prior conversation-history tool failed roughly one out of ten times.
“it would do it maybe 90% of the time, but then one out of every 10 times it would just read the message, make a decision”transcript, stated
The intent split removed about half of the system prompt from the issue-creation path.
“We were able to segment out like I would say half of the system prompt that was tailored around telling you exactly how to fill in different fields of an issue”transcript, stated

Alternatives considered (roads not taken)

Single broad agent prompt with all tools

Rejected because: The agent pulled arbitrary tools and spun its wheels when issue creation needed a narrower path.

“people would ask it to create an issue and then it would start pulling arbitrary tools to like try and get data to fill in fields”transcript, stated

Always search full Slack history

Rejected because: Most requests were localized, and providing recent messages solved the need without another tool call in most cases.

“But for most of the issues that we saw, this uh really fixed it for us. Um but then the the other part of that is”transcript, stated

Ask for confirmation before issue creation

Rejected because: Internal users preferred a first draft they could edit instead of back-and-forth with the agent.

“the internal users that we had were like, I really don't want to talk back and forth with the agent. I wanted to give it a first try”transcript, stated

Patterns

`context-compaction` (agent-design)

Linear sent a small localized Slack window and minimal invocation state instead of dumping all Slack and Linear history into the model.

“we decided to provide 10 or so messages in the beginning so that they could the agent starts off like thinking that it's the 11th message in a conversation.”transcript, stated

`model-routing` (model-ops)

A small upfront model routes the request into the issue-creation path or the broader workspace path.

“when we took a really quick small model in the beginning and just said, are you creating an issue or are you doing like really anything else?”transcript, stated

`tool-use-orchestration` (agent-design)

The agent uses scoped Linear tools for issue creation and workspace data instead of exposing every tool in one path.

“It still has access to the tools that it needs to find data, but its searches were much more um directed and specific”transcript, stated

`structured-output-contract` (integration)

Linear has the LLM emit simple structured ideas while application code handles deterministic formatting.

“maybe if it just spits out an array of ideas and then we can format it however we want. It saved us a lot of headache”transcript, stated

Tradeoffs

Thread context strategy

Chose seed the prompt with recent messages over require the agent to search Slack history first.

Rationale: Most requests were localized, so recent context solved the common case without a tool call.

“even in really long conversations, a lot of the user requests were quite scoped and localized.”transcript, stated

Issue creation flow

Chose early small-model intent split over one prompt that handles issue creation and all other work.

Rationale: The split improved accuracy and speed by removing unrelated tools and prompt content from the issue path.

“higher accuracy on issue creation itself and then faster responses because there were less erroneous attempts at unrelated tools.”transcript, stated

LLM responsibility boundary

Chose LLM for summaries and certain inferred fields over LLM for stable business rules and formatting.

Rationale: Programmatic rules are more reliable for deterministic defaults and presentation.

“titles, descriptions, those are great things for the LLM to do. Very specific business logic that doesn't change. that's something that we would want to um handle on our own.”transcript, stated

Gotchas (what bit them)

Too much tool and conversation context made the agent choose the wrong path.

Root cause: The broad prompt included Slack data, Linear data, search tools, and issue-creation guidance together.

Resolution: Provide localized context and split issue creation into its own path.

“that amount of data really sent it down the wrong path and got it confused sometimes.”transcript, stated

The agent sometimes ignored the Slack history tool when it needed context.

Root cause: The model had to decide to call a tool before seeing localized context.

Resolution: Seed the prompt with about ten recent messages so it starts with thread context.

“one out of every 10 times it would just read the message, make a decision, and not look for any other details.”transcript, stated

LLM-selected business defaults can alert the wrong team.

Root cause: Some issue defaults are deterministic business rules, not language-understanding tasks.

Resolution: Let the LLM summarize titles/descriptions and keep stable defaults in program logic.

“even if it gets it 99% of the time, but then it picks some arbitrary team and then it sets off a bunch of alerts for a team”transcript, stated

Evaluation

Linear evaluated the Slack agent through internal use, observed failure cases around context and tool use, and compared issue-creation accuracy, response speed, and user willingness to accept a first draft instead of a confirmation loop.

Metrics: issue creation accuracy · response speed · erroneous unrelated tool attempts · internal user acceptance · roughly 90% Slack-history tool use before the context change

“the benefits of this were that we had higher accuracy on issue creation itself and then faster responses because there were less erroneous attempts at unrelated tools.”transcript, implied

Outcomes

The Slack agent creates issues and answers workspace questions without requiring users to leave Slack.
A localized context window fixed the common context-retrieval issue.
The intent split made issue creation more accurate and faster.
Internal users preferred editable first attempts over confirmation loops.

The system uses about ten recent messages, a small intent model, and minimal Linear state such as issue, project, or initiative IDs.

“we often use it to create issues and answer questions about your workspace.”transcript, stated

“this uh really fixed it for us.”transcript, stated