Knowledge Augmented Generation (KAG) System for Competitive Intelligence and Strategic Advisory

patho_ai · Tech · year n/a · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Competitive intelligence and strategic advisory

Patho AI needed to build enterprise advisory systems that answer research and strategy questions, not just retrieve passages. Plain vector RAG was inadequate for competitive-analysis questions that require expert decision structure, multi-hop relationships, quantitative evidence, and an explainable path from market data to strategic advice.

Constraints:

• client systems must perform research and advisory work in a specific area of interest
• competitive-analysis questions can require complex numerical reasoning and multi-hop queries
• inaccurate advice can cost contract trust with enterprise clients
• the system must preserve expert decision-making taxonomies, not only document chunks
• the prototype needed flexible orchestration that could map an abstract wisdom graph into executable agents
• knowledge-graph extraction still needed expert pruning to remove bad relationships

Architecture

Patho AI describes a knowledge-augmented generation system centered on a wisdom graph: market data, past campaigns, industrial insight, and current situation signals are mapped into a graph structure that represents how experts make strategic decisions. A Node-RED prototype maps the wisdom engine to a supervisory wisdom agent, which can drive OpenAI, Anthropic, or on-prem models while coordinating specialist agents such as an insight agent. Those agents update a centralized graph, and graph database/Cypher-style queries plus function calls answer competitive-intelligence questions that ordinary vector RAG would reduce to loosely related passages. Human experts prune LLM-extracted graph relationships so the taxonomy stays useful.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Strategic_advisory_chatbot["Strategic advisory chatbot"] --> Wisdom_agent["Wisdom agent"]
  Wisdom_agent --> Strategy_generator["Strategy generator"]
  Wisdom_agent --> Insight_agent["Insight agent"]
  Wisdom_agent --> Language_model_options["Language model options"]
  Market_data_store["Market data store"] --> Unified_knowledge_graph["Unified knowledge graph"]
  Past_campaign_store["Past campaign store"] --> Unified_knowledge_graph
  Industrial_insight_store["Industrial insight store"] --> Unified_knowledge_graph
  Situation_context["Situation context"] --> Unified_knowledge_graph
  Insight_agent --> Unified_knowledge_graph
  Strategy_generator --> Unified_knowledge_graph
  Unified_knowledge_graph --> Graph_query_and_function_call_layer["Graph query and function-call layer"]
  Graph_query_and_function_call_layer --> Language_model_advisor_output["Language model advisor output"]
  Language_model_options --> Language_model_advisor_output
  Language_model_advisor_output --> Strategic_advisory_chatbot
  Expert_pruning_loop["Expert pruning loop"] --> Unified_knowledge_graph

Components

Data flow

Client knowledge sources are organized into a wisdom-graph taxonomy: market data maps to knowledge, past campaigns to experience, industrial-insight databases to insight, and current product/competitor performance to situation. A user's strategic question goes to the wisdom agent, which supervises specialist agents, gathers sentiment or market signals, updates the centralized graph, runs graph queries or function calls over structured data, and returns advisory output grounded in that graph. For graph construction, LLM extraction can create candidate nodes and relationships, but experts prune the graph before it becomes trusted decision support.

Why this architecture

The architecture chooses knowledge-graph-centered KAG because enterprise strategy questions depend on relationships, numeric facts, and expert decision structure. A graph lets the system traverse market entities, run structured calculations, preserve the client's taxonomy, and show evidence for decisions, while Node-RED gives a quick way to prototype supervisory and specialist agents before moving to lighter frameworks if needed.

Models used: OpenAI models · Anthropic models · on-prem models · LLM graph transformer · Neo4j-style graph database · Node-RED agent workflow

Scale

• Patho AI had been operating for two years when the talk was recorded.
  “PTO AI started two years ago with a invitation from National Science Foundation from the SBIR grant funding investigating LLM.”transcript, stated
• The company was building expert AI systems for several enterprise clients.
  “We are currently building expert AI system for several clients.”transcript, stated
• The speaker reports a 91% accuracy result for structured extraction in their benchmark.
  “first is accuracy I had achieved 91% because it's really good in extract structure second is flexibility 85%”transcript, stated
• The benchmark also reported 85% flexibility.
  “first is accuracy I had achieved 91% because it's really good in extract structure second is flexibility 85%”transcript, stated

Alternatives considered (roads not taken)

Traditional vector RAG over retrieved passages

Rejected because: The source says vector stores are weak at complex numerical calculations and can return passages rather than the evidence-based quantitative answer marketing decisions need.

Simple ChromaDB RAG with an LLM agent

Rejected because: The speaker reserves simple ChromaDB RAG for simple product-information queries, not strategic competitive questions.

Fully automated LLM graph transformation

Rejected because: The speaker recommends a hybrid extraction model where experts prune the graph taxonomy after LLM extraction.

Staying permanently on Node-RED

Rejected because: Node-RED is framed as a flexible prototyping layer, with possible later migration to lighter frameworks such as LangChain.

Patterns

rag-retrieval (retrieval)

The system extends RAG into knowledge-augmented generation by retrieving and reasoning over a structured knowledge graph instead of relying only on vector similarity.

multi-agent-orchestration (architecture)

A supervisory wisdom agent coordinates specialist agents that work on graph states such as insight collection and decision-making.

tool-use-orchestration (architecture)

The system routes complex strategic questions to graph queries and function calls rather than asking the model to answer from unstructured retrieved text alone.

human-in-the-loop-gate (governance)

Human domain experts prune the LLM-generated graph taxonomy and relationships before relying on the graph for advisory decisions.

eval-driven-development (evaluation)

The speaker reports benchmark dimensions such as accuracy, flexibility, reproducibility, traceability, and scalability to justify graph-backed KAG.

Tradeoffs

Use knowledge-graph-backed KAG for competitive advisory.

Chose structured knowledge graph plus LLM advisor over traditional vector RAG.

Rationale: The graph supports relationships, multi-hop queries, and numeric calculations needed for strategy questions.

Prototype orchestration in Node-RED.

Chose Node-RED workflow automation over immediately building a lightweight custom LangChain-style implementation.

Rationale: Node-RED is flexible for proof-of-concept workflows even if a later production client may want a lighter stack.

Use a hybrid graph-extraction process.

Chose LLM extraction followed by expert pruning over fully automated graph transformation or fully manual construction.

Rationale: The LLM accelerates graph creation, while expert pruning keeps the taxonomy aligned with domain expertise.

Use graph database queries and function calls for quantitative strategy questions.

Chose graph DB plus Cypher-style query and function-call loops over passage retrieval alone.

Rationale: Competitive analysis often needs exact figures and calculated evidence rather than a passage that mentions the topic.

Gotchas (what bit them)

Vector RAG can retrieve semantically similar passages while failing the quantitative question the business actually asked.

Root cause: Vector stores do semantic similarity well but struggle with structured numerical calculations and relationship traversal.

Resolution: Use graph data, graph queries, and function calls for evidence-based numeric competitive analysis.

Bad strategic advice can immediately damage an enterprise consulting relationship.

Root cause: Marketing and strategy users rely on the advisory system for decision-making, so inaccurate data is not a harmless chatbot error.

Resolution: Favor structured graph evidence and accuracy-oriented design over loose retrieval.

An LLM-extracted graph can preserve too many bad or irrelevant relationships.

Root cause: Fully automated graph transformation does not necessarily match the domain expert's taxonomy.

Resolution: Use a hybrid graph-building process where experts prune the extracted graph.

The final answer quality depends more on the taxonomy and graph structure than on model choice alone.

Root cause: A model can only make the right advisory decision if the decision-relevant domain structure is represented.

Resolution: Invest in the taxonomy and graph structure that feed the advisor output.

Evaluation

The transcript reports a benchmark used to justify graph-backed KAG, with dimensions including accuracy, flexibility, reproducibility, traceability, and scalability. It also presents qualitative criteria for choosing KAG over simple RAG: relationship modeling, improved accuracy, scalability/flexibility, rich queries, and diverse data integration.

Metrics: 91% structured-extraction accuracy · 85% flexibility · reproducibility · traceability · scalability · ability to answer multi-hop competitive-analysis questions · ability to produce quantitative evidence for decisions

Outcomes

• Patho AI built wisdom-graph-powered advisory chatbots for competitive analysis.
• The reported benchmark reached 91% structured-extraction accuracy and 85% flexibility.
• The architecture can answer strategic questions such as how to beat a competitor using market share and competitor weakness context.
• The speaker frames graph-backed KAG as better suited than vector RAG for quantitative, multi-hop competitive intelligence.

The source is a conference transcript about several enterprise-client systems rather than a public production post with detailed deployment telemetry. It reports several clients, a competitive-analysis client replacing marketing-department work, and benchmark numbers of 91% accuracy and 85% flexibility.