Automated Medical Literature Review System Using Domain-Specific LLMs

john_snow_labs · Healthcare · year n/a · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Medical research literature review

John Snow Labs wanted to reduce the time and coordination burden of medical literature reviews while keeping answers grounded in current, cited medical research rather than generic LLM knowledge.

Constraints:

• literature reviews can take weeks to months for broad medical topics
• research teams must handle large volumes of papers and rapidly changing publications
• medical answers require citations, explainability, and safeguards
• inclusion/exclusion criteria and extraction fields vary by review
• raw extractions may still need normalization before conclusions can be drawn

Architecture

John Snow Labs' medical chatbot combines proprietary medical-domain LLMs with a RAG architecture over daily updated open-access medical knowledge bases such as PubMed, arXiv, bioRxiv, and MDPI, plus optional custom knowledge bases. For literature reviews, users choose target knowledge bases, enter search keywords, specify data points and inclusion/exclusion criteria in natural language, and apply filters such as publication date, impact factor, or article type. The chatbot retrieves candidate studies, processes each paper, extracts structured rows with reasoning and evidence, color-codes inclusion status, lets users stop and revise prompts, and exports CSV/Excel results. Professional and enterprise offerings add safeguards, conversation history, API access, and on-prem deployment options.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  medical-research-knowledge-base[Medical research knowledge base] --> medical-rag-chatbot[Medical RAG chatbot]
  custom-knowledge-bases[Custom knowledge bases] --> medical-rag-chatbot
  medical-research-knowledge-base --> literature-review-setup-ui[Literature review setup UI]
  literature-review-setup-ui --> literature-review-runner[Literature review runner]
  medical-rag-chatbot --> literature-review-runner
  literature-review-runner --> evidence-and-reasoning-table[Evidence and reasoning table]
  evidence-and-reasoning-table --> export-and-history-tools[Export and history tools]
  enterprise-deployment-layer[Enterprise deployment layer] --> medical-rag-chatbot

Components

Data flow

A user configures a literature review by selecting knowledge bases, entering keywords, defining extraction fields, setting inclusion/exclusion criteria, and applying publication filters. The chatbot retrieves relevant studies, runs the literature-review analysis, populates a color-coded results table with extracted values, inclusion decisions, reasoning, and source evidence, and exports the completed table for further normalization and analysis. Prior review configurations can be cloned for follow-up runs.

Why this architecture

The architecture is RAG-based because medical answers need current citations and the existing chat context limit is too small for full literature reviews. Letting users specify fields and criteria in plain English makes the workflow configurable for different review protocols. The table, evidence, and export design acknowledges that LLM automation can cover the first four or five review steps, while normalization, analysis, and manuscript writing still need downstream human work. Enterprise on-prem and custom knowledge-base support address medical privacy, proprietary data, and organizational deployment constraints.

Models used: John Snow Labs proprietary medical LLMs · domain-tuned medical LLMs · RAG chatbot

Scale

• The open-access medical knowledge bases are updated daily.
  “it offers references to the sources that were used to compile those answers and uh sources that are um updated on a daily basis”transcript, stated
• The document QA feature supports up to 10 uploaded documents in one session.
  “allows you to upload up to 10 documents in one session and start chatting about them now if you want to De identify or”transcript, stated
• The chatbot includes around nine medical agents.
  “with the medical agents uh we currently have included around um nine medical agents um adapted to different use cases now how does it all”transcript, stated
• Traditional broad literature reviews can take weeks to months or more.
  “significant time and effort from a few weeks to several months or even more uh for broader or complex topics it usually”transcript, stated
• The demonstrated tissue-regeneration search started with 1,600 relevant articles and filtered to 271.
  “basis and I write the way see that I have 100 1,600 articles which are relevant then I go on and uh explain what data I want to extract like the”transcript, stated

  “then I pick um the the date range to use the minimum impact factor and I see that I I end up with a list of 271”transcript, stated
• The 271-document demo analysis completed in 7 minutes and 10 seconds.
  “necessary okay so the entire analysis for those 271 documents to 7 minutes and 10 seconds now imagine the speed”transcript, stated

Alternatives considered (roads not taken)

Use the normal chatbot answer context for literature review

Rejected because: The regular chat response context is limited to about 20 references, while literature review needs access to every relevant publication.

Manual literature review workflow

Rejected because: Manual reviews require weeks or months, experienced teams, and substantial effort managing information overload.

Automatically write the final literature review paper

Rejected because: The product currently automates the earlier review steps, while normalization and writing still require additional work.

Patterns

rag-retrieval (retrieval)

The medical chatbot grounds responses and literature-review extraction in retrieved medical publications and custom knowledge bases.

structured-output-contract (output-control)

The literature-review runner produces structured tables with extracted data points, inclusion/exclusion decisions, evidence, and metadata.

guardrails-io-filtering (safety)

The product describes safeguards to prevent hallucinations and ensure explainable responses in medical workflows.

privacy-preserving-processing (governance)

Enterprise deployment can run on an organization's own servers for improved security, privacy, and control.

Tradeoffs

Use plain-English setup fields

Chose natural-language data points and criteria over coded extraction templates.

Rationale: Researchers can define review-specific fields and filters without developer intervention.

Keep evidence and reasoning visible per extraction

Chose inspectable table cells with reasoning/evidence over opaque extracted values.

Rationale: Researchers need to verify why a value or inclusion decision was made before trusting it in medical research.

Offer enterprise on-prem deployment

Chose organization-controlled servers over browser-only SaaS for all users.

Rationale: Medical and proprietary-document workflows may require stronger privacy, security, control, and custom knowledge-base management.

Gotchas (what bit them)

The extracted review table is not the same thing as a finished literature review.

Root cause: Measurements, time periods, and units can differ across papers, so raw extractions still need normalization before synthesis.

Resolution: Export structured results, then normalize and analyze them downstream before drawing conclusions.

Overly restrictive criteria can eliminate too many papers.

Root cause: Plain-English filters and data-point prompts may be too narrow or poorly specified on the first run.

Resolution: Let users stop the analysis, edit criteria or prompts, and rerun from scratch or clone prior reviews.

A normal chat response cannot cover a full review corpus.

Root cause: The medical chat interface limits response context to a maximum of about 20 references.

Resolution: Use the dedicated literature-review feature to enumerate and process all relevant publications.

Evaluation

The demo evaluates usefulness operationally through citation-backed extraction, color-coded inclusion/exclusion decisions, per-cell reasoning/evidence, exportability, and speed on a 271-document example; model benchmark claims are vendor-reported.

Metrics: state-of-the-art accuracy on Open Medical LLM Leaderboard benchmarks, vendor-reported · 271 documents processed in 7 minutes and 10 seconds · color-coded include/exclude/unknown status · evidence for data points and criteria · CSV/Excel export completeness

Outcomes

• The demo processed 271 candidate documents in about 7 minutes and 10 seconds.
• Users can export paper metadata, summaries, links, extracted data points, evidence, and inclusion/exclusion values.
• The product can automate the first four or five literature-review steps, while normalization and writing remain outside the demonstrated automation.

The platform uses daily updated medical knowledge bases, supports custom knowledge bases, and in enterprise settings supports unlimited knowledge bases, users, and groups.