Building Secure and Private Enterprise Search with LLMs

slack · Tech · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Secure enterprise search across internal and external knowledge systems

Slack needed to extend AI-powered search from Slack content to external sources such as Google Drive and GitHub without storing external data, bypassing permissions, or weakening Slack AI's existing enterprise security and privacy principles.

Constraints:

Architecture

Slack Enterprise Search extends Slack AI with a federated, real-time retrieval architecture. A user query can include Slack content plus explicitly enabled external systems such as Google Drive and GitHub. Admins opt in each source, users grant OAuth access, and Slack requests only read scopes needed for search. At query time, Slack calls partner public search APIs instead of indexing external data into Slack databases, so results and permissions remain fresh. Permissioned snippets flow into Slack AI's existing RAG path, using the same LLMs in the same AWS escrow VPC. Search Answer summaries are shown to the requesting user and immediately discarded rather than stored.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  requesting-user[Requesting user] --> enterprise-search-interface[Enterprise search interface]
  requesting-user --> user-oauth-consent[User OAuth consent]
  admin-source-control[Admin source control] --> external-source-connector[External source connector]
  admin-source-control --> user-oauth-consent
  user-oauth-consent --> external-source-connector
  enterprise-search-interface --> external-source-connector
  external-source-connector --> partner-public-search-apis[Partner public search APIs]
  enterprise-search-interface --> rag-context-assembler[RAG context assembler]
  slack-content-retrieval[Slack content retrieval] --> rag-context-assembler
  external-source-connector --> rag-context-assembler
  rag-context-assembler --> escrow-vpc-llm[Escrow VPC LLM]
  escrow-vpc-llm --> search-answer-renderer[Search Answer renderer]
  search-answer-renderer --> requesting-user
  enterprise-search-interface --> slack-client-cache[Slack client cache]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Requesting user Slack user submitting a search query or asking for AI Answers. Enterprise search interface, User OAuth consent
Admin source control Organization-level opt-in and revocation control for external sources. External source connector, User OAuth consent
User OAuth consent User-level grant that authorizes Slack to read external content the user can access. Requesting user, External source connector
Enterprise search interface Search and AI Answers product surface that combines Slack and external knowledge. Requesting user, External source connector, RAG context assembler
External source connector Slack app-platform integration that queries partner public search APIs with OAuth-authorized read scopes. Admin source control, User OAuth consent, Partner public search APIs, Enterprise search interface
Partner public search APIs External systems such as Google Drive and GitHub that return up-to-date permissioned results. External source connector
Slack content retrieval Slack-native content retrieval governed by Slack AI ACL rules. RAG context assembler
RAG context assembler Supplies the LLM only runtime content needed for the search answer. Enterprise search interface, Slack content retrieval, External source connector, Escrow VPC LLM
Escrow VPC LLM Closed-source LLM hosted in Slack's AWS escrow VPC under Slack AI trust-boundary guarantees. RAG context assembler, Search Answer renderer
Search Answer renderer Displays generated answers to the requesting user and discards summaries immediately. Escrow VPC LLM, Requesting user
Slack client cache Client-side cache used between reloads for product features such as filtering and previews. Enterprise search interface

Data flow

Admins first enable a source for the organization, then individual users grant OAuth access. When a user searches, Enterprise Search uses partner public search APIs at query time rather than a stored external index. OAuth scopes ensure Slack can only read data the user could access in the external system, while Slack AI ACL rules govern internal Slack content. The permissioned results are assembled into a runtime RAG context and sent to the LLM hosted in Slack's AWS escrow VPC. The generated Search Answer is displayed to the requesting user and immediately discarded.

5 supporting quotes
“Slack admins must opt in each external source for use in their organization’s search results and Search Answers.”article, stated
“Next, Slack users also explicitly grant access before we integrate any external sources in their search. Users may also revoke access to any source at any time. This level of control is possible due to the OAuth-based approach mentioned above. ### Principle”article, stated
“we use public search APIs from our partners to return the most up-to-date, permissioned results for a given user.”article, stated
“Augmented Generation (RAG) instead of training LLMs. Using RAG, we supply an LLM with only the content needed to complete the task. This content is permissioned to the user and only available to the LLM at runtime, meaning the LLM doesn’t retain any of your data, ever. * To provide a private,”article, stated
“we don’t even store Search Answer summaries—we just show them to the requesting user and immediately discard them.”article, stated

Why this architecture

Slack chose federated real-time retrieval because indexing external systems would create stale copies, synchronized permission problems, and extra data-retention risk. OAuth delegation lets external systems enforce what the user can read, while org-level and user-level consent make the integration explicit. Reusing the existing Slack AI RAG and escrow-VPC LLM infrastructure keeps model serving inside Slack's trust boundary and avoids customer-data training.

4 supporting quotes
“can already see”) and that this data is up to date. Using a real-time instead of an index-based approach helps us uphold this principle. Because we’re always fetching data from external sources in response to a user query, we never risk that data getting stale. There’s nothing stored on our side,”article, stated
“Because we’re always fetching data from external sources in response to a user query, we never risk that data getting stale.”article, stated
“By leveraging OAuth, we ensure that enterprise search can never perform an action the user did not authorize the system to perform”article, stated
“We use the same LLMs in the same escrow VPC; we use RAG to avoid training LLMs on user data; and we don’t store Search Answers in the database (whether or not they contain external content). However, enterprise search adds a new twist. We can now provide”article, stated

Models used: Closed-source LLMs hosted in Slack's AWS escrow VPC

Scale

Alternatives considered (roads not taken)

Index external source data into Slack databases

Rejected because: Slack wanted no stored external copies, fresh external data, and source-system permission freshness.

“When developing enterprise search, we decided not to store external source data in our database. Instead, we opted for a federated, real-time approach. Building atop Slack’s app platform, we use”article, stated
“developing enterprise search, we decided not to store external source data in our database. Instead, we opted for a federated, real-time approach. Building atop Slack’s app platform, we use public search APIs”article, stated

Use index-based freshness for external content

Rejected because: A stored index can become stale; real-time fetching makes staleness impossible on Slack's side.

“can already see”) and that this data is up to date. Using a real-time instead of an index-based approach helps us uphold this principle. Because we’re always fetching data from external sources in response to a user query, we never risk that data getting stale. There’s nothing stored on our side,”article, stated
“from external sources in response to a user query, we never risk that data getting stale. There’s nothing stored on our side, so staleness simply isn’t possible. But how do we scope down queries to just data that the querying user can access in the external system? The Slack platform”article, stated

Let admin enablement alone grant external data access

Rejected because: Slack adds individual user OAuth consent so external data remains user-controlled.

“Slack admins must opt in each external source for use in their organization’s search results and Search Answers.”article, stated
“Next, Slack users also explicitly grant access before we integrate any external sources in their search. Users may also revoke access to any source at any time. This level of control is possible due to the OAuth-based approach mentioned above. ### Principle”article, stated

Request broad OAuth permissions

Rejected because: Enterprise Search only needs read scopes for search, and transparency helps admins and users trust the integration.

“we only request the OAuth scopes which are necessary to satisfy search queries—specifically read scopes. Not only do we adhere to the principle of least privilege, we show admins and end users the scopes we plan to request when they enable an external source”article, stated
“we show admins and end users the scopes we plan to request when they enable an external source for use in enterprise search. This means that admins and end users always know which authorizations Slack requires to integrate with an external source. ##”article, stated

Persist generated Search Answer summaries

Rejected because: Slack minimizes data retention by displaying search summaries and immediately discarding them.

“we don’t even store Search Answer summaries—we just show them to the requesting user and immediately discard them.”article, stated

Patterns

privacy-preserving-processing (security)

Slack keeps external data out of its databases, uses RAG without model training, and runs LLMs in the existing escrow VPC trust-boundary architecture.

“* We never store data from external sources in our databases. * External data and permissions are up to date with the external system.”article, stated
“Slack uses AWS to host closed-source large language models (LLMs) in an escrow VPC.”article, stated

rag-retrieval (retrieval)

Enterprise Search supplies only permissioned runtime content to the LLM rather than training on Slack or external customer data.

“We use Retrieval Augmented Generation (RAG) instead of training LLMs.”article, stated
“This content is permissioned to the user and only available to the LLM at runtime”article, stated

tool-use-orchestration (integration)

Slack orchestrates partner public search APIs through its app platform and OAuth, fetching external content at query time.

“Building atop Slack’s app platform, we use public search APIs from our partners”article, stated
“The OAuth protocol allows a user to securely authorize Slack to take agreed-upon actions on their behalf”article, stated

safety-policy-enforcement (security)

The system enforces both Slack ACLs and external OAuth-delegated permissions so the LLM only sees data the user can access.

“Slack AI only operates on the data that the user can already see.”article, stated
“we ensure that enterprise search can never perform an action the user did not authorize the system to perform”article, stated

guardrails-io-filtering (compliance)

Admin/user opt-in, revocation, read-only scopes, and non-persistence constrain what external data enters or leaves the AI workflow.

“Users and admins must explicitly grant Slack access to external sources and may revoke that access at any time.”article, stated
“we only request the OAuth scopes which are necessary to satisfy search queries—specifically read scopes.”article, stated

Tradeoffs

Federated real-time search

Chose Query partner public APIs at search time over Store external data in Slack databases.

Rationale: Real-time federation keeps data and permissions fresh and avoids external-data retention inside Slack.

“we decided not to store external source data in our database. Instead, we opted for a federated, real-time approach.”article, stated
“Because we’re always fetching data from external sources in response to a user query, we never risk that data getting stale. There’s nothing stored on our side, so staleness simply isn’t possible.”article, stated

Two-level consent

Chose Admin source enablement plus user OAuth grants over Single organization-wide enablement.

Rationale: Layered consent gives organizations and individuals control over which external data Slack can search.

“Slack admins must opt in each external source for use in their organization’s search results and Search Answers.”article, stated
“Slack users also explicitly grant access before we integrate any external sources in their search.”article, stated

Use RAG over training

Chose Runtime permissioned context over Training LLMs on user data.

Rationale: RAG lets Slack answer with external and Slack context while avoiding user-data retention in model weights.

“We use Retrieval Augmented Generation (RAG) instead of training LLMs.”article, stated
“complete the task. This content is permissioned to the user and only available to the LLM at runtime, meaning the LLM doesn’t retain any of your data, ever. * To provide a private, permissions-aware AI product, Slack uses the requesting user’s Access Control List (ACL) to ensure that the LLM”article, stated

Least-privilege OAuth

Chose Read scopes only over Broader permissions for richer future integrations.

Rationale: Search can be satisfied with read-only access, and narrower scopes build trust with security-conscious admins and users.

“we only request the OAuth scopes which are necessary to satisfy search queries—specifically read scopes.”article, stated
“admins and end users always know which authorizations Slack requires to integrate with an external source.”article, stated

Discard Search Answer summaries

Chose Show answer then immediately discard it over Store generated summaries for history or analytics.

Rationale: Non-persistence simplifies privacy and compliance for generated content that may contain external-source data.

“we don’t even store Search Answer summaries—we just show them to the requesting user and immediately discard them.”article, stated

Gotchas (what bit them)

Central indexing of external sources creates stale data and duplicated governance risk.

Root cause: External content and permissions can change outside Slack after a copy is indexed.

Resolution: Use real-time federated partner API calls instead of storing external source data.

“External data and permissions are up to date with the external system.”article, stated
“There’s nothing stored on our side, so staleness simply isn’t possible.”article, stated

External-search LLM context can leak data if permissions are not scoped before retrieval.

Root cause: The LLM sees whatever retrieval provides, so retrieval must be permission-aware.

Resolution: Use OAuth to ensure Slack only receives external data the user could access in the source system.

“it’s essential that we only fetch data which the user can access in the external system”article, stated
“the actions we perform are a subset of those the user could themselves perform.”article, stated

External-source access can surprise users if consent is implicit.

Root cause: Org-level integration alone does not prove individual users know which data sources are included.

Resolution: Require both admin opt-in and individual user grant, with revocation at any time.

“### Users must explicitly grant access to external sources We believe that your external data should be yours to control. As such, Slack admins must opt in each external source for use in their organization’s search results and Search Answers.”article, stated
“Slack users also explicitly grant access before we integrate any external sources in their search.”article, stated

Broad OAuth scopes would erode least-privilege guarantees.

Root cause: Search does not need write or broad administrative access, even if broader access could support richer features.

Resolution: Request only read scopes necessary for search and show scopes to admins and users.

“a system should never request more privileges than it requires.”article, stated
“we show admins and end users the scopes we plan to request when they enable an external source”article, stated

Client-side caching is a pragmatic exception to a pure no-storage story.

Root cause: Filtering and previews need performance, so the Slack client may cache between reloads.

Resolution: Keep the cache client-side and bounded to product features rather than storing external data in Slack databases.

“Note that the Slack client may cache data between reloads to performantly serve product features like filtering and previews.”article, stated

Evaluation

The source does not describe a formal evaluation harness for enterprise search quality. It defines operational correctness through permission freshness, OAuth authorization, least-privilege scopes, no external-data storage, and reuse of Slack AI's secure RAG/escrow-VPC architecture.

Metrics: External data not stored in Slack databases · External permissions up to date with source system · Admin and user revocation available at any time · OAuth scopes limited to read scopes · Search Answer summaries discarded immediately

4 supporting quotes
“* We never store data from external sources in our databases. * External data and permissions are up to date with the external system.”article, stated
“External data and permissions are up to date with the external system.”article, stated
“Users and admins must explicitly grant Slack access to external sources and may revoke that access at any time.”article, stated
“we only request the OAuth scopes which are necessary to satisfy search queries—specifically read scopes.”article, stated

Outcomes

The source is architectural and does not report user adoption, latency, reliability, or quality metrics; it does identify Google Drive and GitHub as initial sources and more connected apps as planned.

5 supporting quotes
“Users can now surface up-to-date, relevant content that is permissioned to them directly in Slack’s search.”article, stated
“We’re starting with Google Drive and GitHub, and you’ll see many more of your connected apps as the year goes on.”article, stated
“When developing enterprise search, we decided not to store external source data in our database.”article, stated
“we don’t even store Search Answer summaries—we just show them to the requesting user and immediately discard them.”article, stated
“They can also revoke this access (for both search results and Search Answers) at any time.”article, stated