MultiCare: A Large-Scale Medical Case Report Dataset for AI Model Training

national_university_of_the_south · Healthcare · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Medical AI dataset engineering

The MultiCare project needed to provide large-scale, open, multimodal medical training data for language, vision, and multimodal AI models without PHI-heavy medical records.

Constraints:

Medical AI needs high-quality multimodal training data.
Clinical case reports over-represent rare and complex presentations.
Published images can be compound figures requiring splitting and caption alignment.
Medical records often contain PHI and noisy irrelevant text.

Architecture

MultiCare extracts PubMed Central open-access case reports, segments patient-level text, extracts demographics, preprocesses compound images with OpenCV edge detection, splits and maps captions, and normalizes image labels with contextual NLP parsers and custom dictionaries.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  pubmed-central-extraction["PubMed Central Extraction"]
  patient-level-segmentation["Patient-Level Segmentation"]
  image-preprocessing["Image Preprocessing"]
  caption-processing["Caption Processing"]
  label-extraction["Label Extraction"]
  dataset-packaging["Dataset Packaging"]
  pubmed-central-extraction --> patient-level-segmentation
  patient-level-segmentation --> image-preprocessing
  patient-level-segmentation --> label-extraction
  image-preprocessing --> caption-processing
  caption-processing --> label-extraction
  label-extraction --> dataset-packaging

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`PubMed Central Extraction`	Collects open-access case reports and metadata using BioC and Python libraries.	`Patient-Level Segmentation`
`Patient-Level Segmentation`	Separates article sections corresponding to individual patients and extracts demographics.	`Image Preprocessing`, `Label Extraction`
`Image Preprocessing`	Splits compound medical figures and removes borders using edge detection.	`Caption Processing`
`Caption Processing`	Splits compound captions and maps relevant text to each image component.	`Label Extraction`
`Label Extraction`	Normalizes modality, anatomy, and finding labels from caption text.	`Dataset Packaging`
`Dataset Packaging`	Publishes data and subset tooling for LLM, CV, and multimodal model training.	—

Data flow

The pipeline identifies open-access case reports through BioC, extracts article metadata and text, segments patients, extracts age/gender, splits compound images, assigns caption fragments to sub-images, normalizes labels, and publishes the dataset with tools for custom subsets.

“The process began with identifying open-access case report articles from PubMed Central using the BioC API.”summary, stated

“Image preprocessing addressed a common challenge in medical publishing: compound images where multiple images are combined into a single file.”summary, stated

Why this architecture

Clinical case reports provide reviewed medical narratives with aligned images, making them useful training infrastructure even though their distribution differs from ordinary medical records.

“The MultiCare project addresses the challenge of training medical AI models by creating a comprehensive, open-access dataset of over 75,000 clinical case reports spanning 1990 to 2023.”summary, stated

“The text quality in published case reports is generally higher than in medical records because the content undergoes multiple rounds of review before publication.”summary, stated

Models used: JSSS contextual parsers · OpenCV · NLP label extraction

Scale

The dataset contains more than 75,000 case report articles.
“containing more than 75,000 case report articles spanning from 1990 to recent years.”summary, stated
The dataset includes 135,000 medical images.
“includes 135,000 medical images across virtually every imaging modality used in clinical practice.”summary, stated
The articles were authored by about 380,000 medical professionals and reference about 96,000 patients.
“These articles were authored by approximately 380,000 medical professionals and reference about 96,000 patients.”summary, stated
The dataset totals 8.8GB.
“The dataset totals 8.8 gigabytes”summary, stated
Patient demographics average 41.5 years with 48.5% female and 51.5% male.
“mean patient age of 41.5 years, with a relatively balanced gender split of 48.5% female and 51.5% male patients.”summary, stated

Alternatives considered (roads not taken)

Train directly on medical records

Rejected because: Medical records contain PHI, noise, filler text, and different privacy constraints.

“The text is also more concise and avoids irrelevant filler content (like "patient denies pain") that is common in medical records but adds noise for training purposes.”summary, stated

Use compound figures as-is

Rejected because: Multi-panel images need splitting for image-label alignment.

“automatically split compound images into individual components. This is essential for proper image-label alignment in training multimodal models.”summary, stated

Patterns

`fine-tuning-adaptation` (modeling)

The dataset is designed as training data for language, computer vision, and multimodal medical models.

“creating a comprehensive, open-access dataset of over 75,000 clinical case reports spanning 1990 to 2023.”summary, stated

`context-engineering` (context)

Caption splitting and label normalization preserve the context needed to align text, images, anatomy, modality, and findings.

“the system splits the caption and assigns relevant portions to each sub-image while preserving shared context.”summary, stated

Tradeoffs

Use published case reports

Chose Reviewed PHI-free clinical narratives over Raw medical records.

Rationale: Case reports reduce PHI/privacy problems and improve text quality but introduce distribution shift.

“While this creates a distribution shift from real-world clinical practice, it also provides richer variety for training models that need to recognize unusual conditions.”summary, stated

Use caption-derived labels

Chose NLP parsing of descriptive captions over Manual labeling of every image.

Rationale: Captions are descriptive and rarely include hypothetical or negated mentions.

“captions are descriptive by nature and rarely contain hypothetical or negated mentions”summary, stated

Gotchas (what bit them)

Case reports are not distributed like everyday medical records.

Root cause: They over-represent rare and complex presentations.

Resolution: Use the dataset with awareness of distribution shift and target use case.

“Clinical cases tend to over-represent rare and complex presentations compared to the distribution found in typical medical records.”summary, stated

Compound medical figures break simple image-label pairing.

Root cause: Published articles often combine multiple images and captions into one figure.

Resolution: Split compound images and compound captions before label assignment.

“For compound images with compound captions (e.g., "brain CT scan showing mass in the frontal lobe (A) and hemorrhage in the right temporal lobe (B)"), the system splits the caption”summary, stated

Evaluation

The case describes dataset composition, demographics, labels, preprocessing, and publication artifacts rather than model performance evaluation.

Metrics: case report count · image count · dataset size · patient demographics

“Each image comes with multiple layers of labels covering image type, imaging technique (such as Doppler or contrast status), anatomical information”summary, stated

Outcomes

MultiCare published an 8.8GB medical case-report dataset.
The dataset includes over 75,000 reports and 135,000 medical images.
Tools support custom subsets for LLM, computer vision, and multimodal training.

75,000+ reports, 135,000 images, 380,000 authors, 96,000 patients, and 8.8GB of data.

“with a GitHub repository providing tools for creating custom subsets based on specific use cases for training language models, computer vision models, or multimodal models.”summary, stated