Enterprise LLM Implementation Panel: Lessons from Box, Glean, Tyace, Security AI and Citibank

various · Tech · 2023 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Enterprise generative AI adoption across content, search, security, and banking

The panel describes enterprises trying to move from generative-AI proofs of concept into production while dealing with unstructured content, enterprise permissions, data quality, security/compliance, user prompting gaps, model choice, regulation, and organization-wide change.

Constraints:

• Roughly 90% of the audience had POCs, but only about a fifth had production applications.
• Enterprise data access and permissions differ from public web search.
• Unstructured content needs governance, metadata, lineage, quality, and entitlement controls.
• Non-expert employees struggle with blank prompt canvases.
• Regulated organizations need to show boards and executive teams that AI is safe.

Architecture

Across Box, Glean, Typeface, Security AI, and Citibank, the panel converges on a governed enterprise AI stack: permissions-aware RAG/search over enterprise systems, secure content controls, model/provider selection, prompt scaffolding for end users, knowledge graphs for unstructured data governance, and broad organizational rollout from lab projects to factory-floor use.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  enterprise-content-systems["Enterprise content systems"] --> permissions-aware-rag-index["Permissions-aware RAG index"]
  enterprise-content-systems --> knowledge-graph-governance["Knowledge graph governance"]
  knowledge-graph-governance --> security-and-compliance-controls["Security and compliance controls"]
  security-and-compliance-controls --> generative-model-layer["Generative model layer"]
  permissions-aware-rag-index --> generative-model-layer["Generative model layer"]
  permissions-aware-rag-index --> enterprise-application-experiences["Enterprise application experiences"]
  generative-model-layer --> enterprise-application-experiences["Enterprise application experiences"]
  enterprise-application-experiences --> adoption-and-training-program["Adoption and training program"]

Components

Data flow

Enterprise content and systems are indexed with permissions and metadata, retrieved through search/RAG, combined with model/provider controls and application experiences such as chatbots, content generation, and copilots, then rolled into production with governance, security, and training programs.

Why this architecture

The panelists argue that enterprise AI succeeds only when model capabilities are coupled with enterprise context, permissions, data quality, security, and organizational adoption rather than isolated POCs.

Models used: Google Cloud AI · Vertex AI · Vertex Search · Copilot · Enterprise RAG platforms · Generative AI models

Scale

• About 90% of the audience had generative-AI POCs, while about a fifth had production applications.
  “poc's with generative AI okay uh that is maybe 90% of the audience”transcript, stated
• Box describes over 100,000 enterprise customers.
  “we provide um Enterprise grade uh ability for to have content Management Services”transcript, implied
• A global software company with over 10,000 employees reported saving almost 40 engineering years in the first year of deploying Glean.
  “a global software company with over 10,000 employees they have they deployed glean across all of their engineering teams”transcript, stated
• A telecommunications customer had over 50,000 agents using Glean for customer care.
  “they have over 50,000 agents who now use glean to um to answer questions that their customers are asking”transcript, stated
• The Glean telecom case reported 47% faster customer-case resolution.
  “this company is actually now resolving customer cases 47% faster than before”transcript, stated
• Typeface cited estimated 2 to 5% sales lift from personalized content.
  “their estimates are they're going to get 2 to 5% sales bump with this kind of a model”transcript, stated
• Citibank describes several tens of thousands of people using Copilot.
  “several tens of thousands of people who use co-pilot um in city today”transcript, stated

Alternatives considered (roads not taken)

Use generic RAG without permission-aware retrieval.

Rejected because: Enterprise knowledge is not globally visible, so retrieval must respect user and group access.

Leave end users with a blank prompt canvas.

Rejected because: Typeface found average users did not know what prompts to type, so it built magic prompts.

Run AI as isolated lab projects.

Rejected because: Citibank argued for moving AI from lab projects into broad factory-floor use.

Treat unstructured data as ready to use without metadata governance.

Rejected because: Security AI says production requires understanding quality, lineage, entitlement, and controls.

Patterns

rag-retrieval (retrieval)

Glean connects enterprise systems into a permissions-aware search/RAG layer for workplace answers.

privacy-preserving-processing (governance)

Panelists emphasize permissions, isolation, privacy, compliance, and not exposing data users cannot access.

guardrails-io-filtering (safety)

Security AI describes inline controls to remove sensitive information and enforce governance before model use.

model-routing (model-ops)

Security AI and the panel describe selecting appropriate models by efficacy, price point, and use-case fit.

Tradeoffs

Make RAG permission-aware.

Chose Identity-aware retrieval limited to information the user can access. over A broad enterprise index available equally to all users..

Rationale: Enterprise information is often visible only to small sets of people.

Scaffold content generation for non-expert users.

Chose Magic prompts and context ingestion. over Expecting every employee to craft high-quality prompts from scratch..

Rationale: The average user's maturity may lag far behind the builder's.

Roll AI out as a broad transformation.

Chose Factory-floor deployment, internal training, and organization redesign. over A few lab projects owned by AI experts..

Rationale: Citibank viewed AI as a new tech stack affecting many projects and departments.

Gotchas (what bit them)

Enterprise AI stalls when security is not foundational.

Root cause: Sensitive content like PII, M&A deals, and financial results cannot be exposed to unauthorized users.

Resolution: Make AI respect enterprise-grade security, permissions, model isolation, and data access rules.

RAG answers fail when retrieved data is stale or low quality.

Root cause: The model can only answer well if the right fresh, subject-matter-expert information is supplied.

Resolution: Filter out-of-date, low-quality, low-engagement, or audience-mismatched information before retrieval.

Prompt sophistication of builders exceeds that of average users.

Root cause: End users often do not know what prompts to type into content-generation tools.

Resolution: Provide prompt scaffolding such as magic prompts from keywords.

Regulation can block production even after POCs work.

Root cause: AI regulatory frameworks and board-level safety expectations arrived quickly.

Resolution: Automate readiness and compliance evidence as part of productionization.

Evaluation

The panel evaluates adoption and value through POC-to-production conversion, permission-safe retrieval, freshness/quality of retrieved data, customer productivity metrics, sales lift estimates, and organizational rollout scale.

Metrics: Audience POC share about 90% · Production application share about one fifth · Almost 40 engineering years saved · 47% faster customer case resolution · 2 to 5% estimated sales lift · 90 geographies for localized content

Outcomes

• Glean customer examples report almost 40 engineering years saved and 47% faster customer-case resolution.
• Typeface customer examples include estimated 2 to 5% sales lift and localization across 90 geographies.
• Citibank reports a very large Copilot deployment and a shift from lab projects to broad organizational use.
• The panel frames production readiness around permissions, governance, security, data quality, and organizational change.

This is a multi-company panel, so outcomes are distributed across Box, Glean, Typeface, Security AI, and Citibank rather than one system.