Building AI Memory Layers with File-Based Vector Storage and Knowledge Graphs

cognee · Tech · 2025 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Agent memory infrastructure

Cognee needed durable agent memory with graph and vector retrieval while avoiding shared-state collisions across developers, users, tests, demos, and production workspaces.

Constraints:

Agent memory must combine vector retrieval, graph reasoning, provenance, and time context.
Developers and tests need isolated memory stores to prevent cross-run contamination.
The local development model should promote to production without redesign.
The stack should avoid operational overhead from separate vector services per sandbox.

Architecture

Cognee combines an Extract-Cognify-Load memory pipeline with LanceDB for file-based vector storage, Kuzu for graph storage, and Cogwit as the hosted production service. Each workspace gets its own isolated LanceDB store, while Cognee builds graph-aware and vector-backed memory from files, APIs, and databases.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  sources[Sources] --> extract-stage[Extract Stage]
  extract-stage --> cognify-stage[Cognify Stage]
  cognify-stage --> load-stage[Load Stage]
  load-stage --> kuzu-graph-store[Kuzu Graph Store]
  load-stage --> lancedb-vector-store[LanceDB Vector Store]
  kuzu-graph-store --> cognee-query-engine[Cognee Query Engine]
  lancedb-vector-store --> cognee-query-engine
  cognee-query-engine --> memify-pipeline[Memify Pipeline]
  cognee-query-engine --> agent-or-app[Agent or App]
  cogwit-hosted-service[Cogwit Hosted Service] --> cognee-query-engine
  cogwit-hosted-service --> agent-or-app

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Sources`	Files, APIs, and databases ingested into the memory layer.	`Extract Stage`
`Extract Stage`	Pulls source data into the Cognee pipeline.	`Sources`, `Cognify Stage`
`Cognify Stage`	Chunks, embeds, enriches entities and relationships, and adds temporal tagging.	`Extract Stage`, `Load Stage`
`Load Stage`	Writes graph structures and vector data into retrieval backends.	`Cognify Stage`, `Kuzu Graph Store`, `LanceDB Vector Store`
`LanceDB Vector Store`	File-based vector store with per-workspace isolation and unified payload/embedding storage.	`Load Stage`, `Cognee Query Engine`
`Kuzu Graph Store`	Graph backend for entity and relationship retrieval.	`Load Stage`, `Cognee Query Engine`
`Cognee Query Engine`	Serves graph and vector search for agents, apps, notebooks, and local UI.	`LanceDB Vector Store`, `Kuzu Graph Store`, `Agent or App`, `Memify Pipeline`
`Memify Pipeline`	Post-deployment memory freshness process that cleans stale nodes and improves relevance.	`Cognee Query Engine`
`Cogwit Hosted Service`	Hosted production path managing Kuzu, LanceDB, PostgreSQL, governance, and autoscaling.	`Cognee Query Engine`, `Agent or App`
`Agent or App`	Consumer querying and using the memory layer.	`Cognee Query Engine`, `Cogwit Hosted Service`

Data flow

Sources are extracted, chunked, embedded, enriched into entities and relations, tagged with temporal context, loaded into graph and vector backends, and queried by agents or applications through graph and vector search.

“It ingests files, APIs, and databases, then applies an Extract–Cognify–Load model to chunk content, enrich entities and relationships, add temporal context”article, stated

“Here the Extract, Cognify, and Load stages turn raw inputs into a knowledge graph and embeddings, then serve both graph and vector search”article, stated

Why this architecture

File-based LanceDB matched Cognee's workspace isolation model, while graph plus vector retrieval made memory more structured than a thin RAG layer.

“Since LanceDB is file based, Cognee can spin up a separate store per test, per user, and per workspace.”article, stated

“Do not treat memory as a thin RAG layer . Cognee ’s structured graph, time awareness, and post-processing combine”article, stated

Models used: embedding models

Scale

Each developer, user, test, and workspace can receive its own independent store.
“isolation model is the way it separates memory stores so that every developer, user, or test instance gets its own fully independent workspace. Instead of sharing a single vector database or graph across contexts, Cognee spins up a dedicated set of backends—most notably a [LanceDB”article, stated
The same model moves from laptop development to hosted production.
“This architecture scales from a laptop to production without changing the mental model. Developers can start locally with Cognee’s UI, build and query memory against LanceDB, and validate behavior with isolated sandboxes.”article, stated
Production Cogwit manages Kuzu, LanceDB, and PostgreSQL with governance and autoscaling.
“which manages Kuzu for graph, LanceDB for vectors, and PostgreSQL for metadata, adding governance and autoscaling”article, stated

Alternatives considered (roads not taken)

Shared vector database or graph across development contexts.

Rejected because: Shared state can cause unpredictable behavior, data corruption, and debugging problems.

“when running tests in parallel, shared state can lead to unpredictable behavior, data corruption, and debugging nightmares.”article, stated

Traditional vector databases requiring running servers per sandbox.

Rejected because: That adds service, port, connection, and dependency overhead during development and testing.

“Unlike traditional vector databases that require running servers, LanceDB’s file-based approach means each workspace gets its own directory on disk.”article, stated

Separate document store, vector database, and graph engine glued together.

Rejected because: Cognee and LanceDB reduce orchestration cost by keeping data, vectors, graph, and memory updates aligned.

“Many stacks bolt a vector database to an unrelated document store and a separate graph system. That adds orchestration cost and creates brittle context.”article, stated

Patterns

`agent-memory` (memory)

Cognee builds durable agent memory from graphs, vectors, temporal context, provenance, and post-deployment memory refresh.

“Cognee delivers a durable memory layer for AI agents by unifying a knowledge graph with high-performance vector search.”article, stated

`rag-retrieval` (retrieval)

Applications query both graph structures and vectors for multi-step answers with clearer provenance.

“Applications built on Cognee query both the graph and the vectors to answer multi-step questions with clearer provenance.”article, stated

`hybrid-search` (retrieval)

LanceDB supplies vector and full-text search while Cognee combines that with graph retrieval.

“Hybrid Search: Native support for both vector search and full-text search”article, stated

`data-quality-loop` (memory-maintenance)

Memify refreshes memory after deployment by cleaning stale nodes and reweighting facts instead of full rebuilds.

“Cognee’s Memify pipeline keeps memory fresh after deployment by cleaning stale nodes, strengthening associations, and reweighting important facts”article, stated

Tradeoffs

Vector storage model

Chose File-based LanceDB stores per workspace over A shared server-backed vector database.

Rationale: The file model made per-test and per-user isolation cheap and reduced local development overhead.

“This eliminates the complexity of managing database connections, ports, and service dependencies during development and testing.”article, stated

Memory representation

Chose Knowledge graph plus vector search over Thin RAG over vectors alone.

Rationale: Graph-aware retrieval supports multi-step reasoning, provenance, and time-aware context.

“converts unstructured and structured inputs into an evolving knowledge graph, then couples it with embeddings for retrieval.”article, stated

Production migration

Chose Same structures/APIs from local to Cogwit over Redesigning the memory layer for hosted production.

Rationale: The same model can be promoted when governance and scale matter.

“This seamless transition is possible because the same data structures and APIs work in both local and hosted environments.”article, stated

Gotchas (what bit them)

Shared memory state makes parallel development unreliable.

Root cause: Multiple developers or tests can mutate the same vector/graph state.

Resolution: Provision independent LanceDB-backed workspaces per developer, user, test, or demo.

“Developers run experiments in parallel without shared state, and they avoid the overhead of managing a separate vector database service for every sandbox.”article, stated

Separating payloads from vectors adds synchronization glue.

Root cause: Pipelines must otherwise keep original data and embeddings consistent across stores.

Resolution: Use LanceDB's columnar storage to keep original data and embeddings together.

“LanceDB stores original data and embeddings together through the Lance columnar format. Pipelines become simpler”article, stated

Index settings become product tradeoffs rather than invisible infrastructure defaults.

Root cause: Recall, latency, and footprint vary by workload.

Resolution: Start with defaults, then profile and adjust indexes and quantization.

“Treat index selection as a product decision. Start with defaults to validate behavior, then profile and adjust the index and quantization settings to meet your target cost and latency.”article, stated

Evaluation

Cognee validates memory quality through isolated local sandboxes, graph/vector query inspection, index profiling, and graph-aware evaluation including HotPotQA-style multi-hop reasoning.

Metrics: Retrieval accuracy · Multi-hop correctness · CI friction · Demo reliability · Recall, latency, and footprint

“At the product level, combining Cognee’s knowledge graph with LanceDB’s vector search has improved retrieval accuracy”article, stated

“strong results on graph-aware evaluation, including high correctness on multi-hop tasks such as HotPotQA.”article, stated

Outcomes

Cognee gained per-workspace vector isolation for tests, users, demos, and development.
The graph-plus-vector memory layer improved retrieval quality on multi-hop reasoning tasks.
The same local setup can move to Cogwit production without redesign.

The source describes one isolated store per test/user/workspace and promotion from laptop workflows to hosted production with governance and autoscaling.

“File-based isolation allows each test, workspace, or demo to run on its own vector store, enabling faster CI, cleaner multi-tenant setups”article, stated

“giving teams a direct path from prototype to production without redesign.”article, stated