RAG-powered Decision Intelligence Platform for Manufacturing Knowledge Management

circuitry.ai · Tech · 2023 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Manufacturing decision intelligence and multi-tenant product knowledge management

Circuitry.ai needed to build RAG advisors for manufacturers of complex products while managing proprietary customer data, tenant-specific filters, evaluation criteria, frequent knowledge-base updates, and heterogeneous product information.

Constraints:

• Customer data was proprietary and required segregation and governance across manufacturing clients.
• RAG retrieval needed metadata filters so results could be tailored to individual clients and product contexts.
• Knowledge bases changed as new products and articles were added, so updates could not disrupt RAG chains.
• A small technical team needed to support custom knowledge bases and RAG chatbots without spending all its time on data infrastructure.

Architecture

Circuitry.ai used the Databricks platform as its production data and AI substrate for custom manufacturing RAG advisors. Customer documents are uploaded through a UI, raw data is ingested, processed into embeddings, and served through endpoints backed by Delta Lake, Unity Catalog, MLflow, Model Serving, and a serverless vector database. Llama and DBRX models power generated responses, while feedback loops, metadata filtering, and Databricks-assisted governance support multi-tenant customer deployments.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Customer documents and references"] --> B["Upload interface"]
  B --> C["Databricks notebooks"]
  C --> D["Delta Lake"]
  D --> E["Unity Catalog"]
  C --> F["Embedding pipeline"]
  F --> G["Serverless vector database"]
  E --> G
  C --> H["Model Serving"]
  I["MLflow"] --> H
  H --> J["RAG application stack"]
  G --> J
  E --> J
  K["Llama and DBRX models"] --> J
  J --> L["Feedback mechanism"]

Components

Data flow

Customers upload documents or references through Circuitry.ai's UI. The workflow triggers raw-data ingestion, processing and embedding generation, and serving-endpoint deployment through Databricks notebooks and upstream dependencies. The application stack runs a RAG pipeline over custom knowledge bases, uses Llama and DBRX models for responses, applies metadata filtering for client-specific results, and captures user ratings for continuous improvement.

Why this architecture

The Databricks architecture addressed Circuitry.ai's production RAG bottlenecks by combining reliable incremental data updates, tenant governance, model management, governed vector search, feedback, and metadata filtering in one platform. That mattered because their product depends on customer-specific proprietary manufacturing knowledge and continually changing product data.

Models used: Llama · DBRX · Databricks Model Serving

Scale

• Circuitry.ai was founded in 2023 to serve manufacturers of high-value complex products.
  “Founded in 2023 by industry leaders, Circuitry.ai recognized an opportunity to solve their clients’ most significant obstacles using GenAI tools.”article, stated
• Customers reported 60-70% reduction in time spent searching for information.
  “Customers of the decision intelligence software experienced a 60–70% reduction in the time spent searching for information.”article, stated
• The system targets complex product categories such as heavy equipment and automotive.
  “Their intricate products (think heavy equipment and automotive) come with complex bills of materials”article, stated
• The reported response-time improvement is answers in seconds rather than minutes.
  “happy customers receive answers in seconds rather than minutes.”article, stated

Alternatives considered (roads not taken)

Continue manually managing customer data and RAG updates

Rejected because: The small team was delayed by metadata filters, quality checks, knowledge-base updates, and data segregation needs.

Use a RAG chain that requires disruptive full knowledge-base rebuilds

Rejected because: Circuitry.ai needed incremental knowledge updates without breaking internal RAG chains.

Use unfiltered retrieval across customer product data

Rejected because: Client-specific results and proprietary-data segregation were core requirements.

Commit to a single model family without switching and testing

Rejected because: Circuitry.ai valued Databricks support for multiple models and extensive testing.

Patterns

rag-retrieval (retrieval)

Circuitry.ai built customer-specific knowledge bases and RAG chatbots over uploaded manufacturing documents.

data-quality-loop (feedback)

Users rate generated responses, and that feedback is used to improve the Decision AIdvisor experience and prompts.

eval-driven-development (quality)

The team established internal chatbot checks before deployment and valued model switching/testing.

privacy-preserving-processing (governance)

Unity Catalog and data segregation protect proprietary customer data across manufacturing tenants.

model-routing (model-operations)

The platform supports switching between Llama and DBRX models and extensive testing.

Tradeoffs

Use Databricks as the RAG data and governance substrate.

Chose Delta Lake, Unity Catalog, MLflow, Model Serving, and governed vector search over A bespoke data-management stack maintained by a small team.

Rationale: The platform reduced delays around proprietary data, updates, evaluation, and deployment.

Support incremental updates rather than disruptive RAG rebuilds.

Chose Delta Lake-backed incremental data updates over Knowledge-base update processes that disrupt internal RAG chains.

Rationale: The business needs current product knowledge without taking the chatbot pipeline apart for each update.

Use customer-specific metadata filtering.

Chose Metadata and Model Serving with governed vector retrieval over Generic retrieval over all indexed manufacturing content.

Rationale: Individual clients have different products and applications, so retrieval has to narrow answers to the right customer context.

Expose AI to nontechnical users through customized prompts and simplified querying.

Chose Decision AIdvisor prompt customization and simplified querying over A technical-only interface for querying complex product data.

Rationale: The product is meant for sales, service, channel partners, and customers, so AI needed to be accessible outside the technical team.

Gotchas (what bit them)

Metadata filters were hard to apply on top of retrievers.

Root cause: The ecosystem documentation was insufficient for the client-specific filtering Circuitry.ai needed.

Resolution: Use Databricks assistance, serverless vector database, and metadata filtering to tailor results to individual clients.

Knowledge-base updates could disrupt RAG chains.

Root cause: New products and knowledge articles continually change the retrieval corpus.

Resolution: Use Delta Lake for incremental updates so the knowledge base stays current.

Multi-tenant proprietary data created a trust and governance requirement.

Root cause: The models used customer proprietary knowledge resources, so cross-tenant leakage would damage trust.

Resolution: Use Unity Catalog for data governance, segregation, and security of AI/data assets.

Complex manufacturing data sources did not share a uniform structure.

Root cause: Manufacturing product information arrived from multiple data sources with different structures and formats.

Resolution: Build a stable data integration framework on Databricks before the RAG layer.

Evaluation

Circuitry.ai established internal chatbot quality checks, used Databricks support for model switching and extensive testing, and incorporated user ratings on generated responses as a continuous improvement signal.

Metrics: Internal chatbot evaluation criteria · Model-switching and testing · User ratings on generated responses · Customer feedback on speed and relevance · 60-70% reduction in search time

Outcomes

• Customers reported a 60-70% reduction in time spent searching for information.
• New employees gained faster access to relevant product knowledge.
• Customers praised response speed and relevance in proof-of-concept trials.
• Circuitry.ai planned extensions into guided sales, service recommendations, predictive analytics, richer media, and operational automation.

The case reports 60-70% search-time reduction, answers in seconds rather than minutes, and applicability across manufacturers of heavy equipment and automotive products; it does not provide exact deployment count or query volume.