Lessons Learned from Information Retrieval in Vector Database Development

weaviate · Tech · 2026 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: information retrieval and vector databases

Weaviate's article captures practical information-retrieval lessons for teams building vector-search and RAG infrastructure, but the fetched source only exposes the opening member-only preview rather than the full 37-item article.

Constraints:

• The full article content is not available in the fetched source, so detailed claims must remain limited to the visible preview and summary.
• Vector search must balance search quality, latency, and scale because exact nearest-neighbor search does not scale well.
• Production vector databases need to manage embeddings, original objects, and metadata rather than embeddings alone.

Architecture

The visible source frames production retrieval as a layered search system: begin with lexical BM25 as a strong baseline, add approximate vector search for scale, and retain original objects and metadata alongside embeddings so downstream RAG systems can retrieve both similarity matches and usable context. Because only the preview is fetched, deeper operational lessons from the full article are not asserted as source-backed details.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  BM25_baseline["BM25 baseline"] --> Retrieval_application["Retrieval application"]
  Retrieval_application --> Vector_database["Vector database"]
  Vector_database --> ANN_index["ANN index"]
  Vector_database --> Object_and_metadata_store["Object and metadata store"]
  Object_and_metadata_store --> Retrieval_application

Components

Data flow

Documents are embedded and stored in a vector database together with their original text and metadata. Queries can be served by BM25, vector search, or a hybrid retrieval approach depending on the problem. Approximate nearest-neighbor indexes such as HNSW, IVF, or ScaNN trade a small amount of exactness for speed at scale, after which retrieved objects and metadata can feed downstream LLM context.

Why this architecture

The source argues against treating vector search as an automatic first step. BM25 is a strong baseline, exact vector search does not scale, and storing objects plus metadata turns the vector database into usable retrieval infrastructure rather than a raw embedding bucket.

Models used: BM25 · embedding models · Approximate Nearest Neighbor algorithms · HNSW · IVF · ScaNN · Weaviate

Scale

• The article reflects two years of retrieval work at a vector database company.
  “Reflections on what I’ve learned about information retrieval in the last two years working at Weaviate”article, stated
• The visible article is positioned as a 37-item lessons-learned piece.
  “37 Things I Learned About Information Retrieval in Two Years at a Vector Database Company”article, stated
• The article was published as a 10-minute read.
  “10 min read”article, stated

Alternatives considered (roads not taken)

Start immediately with vector search

Rejected because: The source explicitly recommends BM25 as a simple, strong baseline before adding vector search complexity.

Use exact brute-force KNN over every vector

Rejected because: The source says exact brute-force vector search does not scale, so vector databases use ANN algorithms.

Store only embeddings

Rejected because: The source says vector databases also store original objects and metadata, implying embeddings alone are insufficient for practical retrieval.

Patterns

hybrid-search (retrieval)

The visible lessons recommend treating BM25 as the baseline and vector search as the next layer, which maps to a hybrid retrieval mindset.

rag-retrieval (retrieval)

The summary connects vector databases to production RAG infrastructure where retrieved context is supplied to LLM applications.

latency-optimization (operations)

ANN indexing is used to make vector search fast enough at scale, explicitly trading a small amount of accuracy for speed.

Tradeoffs

Retrieval baseline

Chose BM25 first over Starting with vector search by default.

Rationale: The visible lesson explicitly recommends a simple lexical baseline before moving to a more complex vector setup.

Vector-search scaling

Chose Approximate nearest-neighbor indexing over Exact brute-force KNN.

Rationale: Exact distance computation against every vector is conceptually simple but not scalable.

Gotchas (what bit them)

Vector search is approximate, not exact.

Root cause: Exact KNN over every vector does not scale for production workloads.

Resolution: Use ANN algorithms while understanding the accuracy/speed tradeoff.

The fetched source does not expose the full 37 lessons.

Root cause: The Medium story is member-only and the source cache only contains the visible preview.

Resolution: Treat this extraction as source-limited and avoid asserting the unseen lessons as evidence-backed details.

Evaluation

The visible source does not describe a formal evaluation harness. Its evaluation guidance is implicit in using BM25 as a baseline and recognizing the speed/accuracy tradeoff of approximate vector retrieval.

Metrics: BM25 baseline comparison · accuracy loss from approximate nearest-neighbor search · search speed at scale

Outcomes

• The visible source provides concrete opening lessons on BM25, approximate vector search, and object/metadata storage.
• The broader 37-lesson article is not available in the fetched source, so broader operational outcomes remain unverified here.

Source-backed scale is limited to the author's two years at Weaviate and the article's 37-lessons framing; no production deployment metrics are visible in the fetched text.