LLM-Powered Information Extraction from Pediatric Cardiac MRI Reports

uk_national_health_service_(nhs) · Healthcare · year n/a · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Clinical information extraction from pediatric cardiac MRI reports

Great Ormond Street Hospital needed to extract structured clinical identifiers and measurements from heterogeneous unstructured cardiac MRI PDF reports so clinicians and researchers could analyze cohorts at scale.

Constraints:

• The source corpus was nearly 15,000 cardiac MRI reports produced over 10 years by different consultants.
• Reports were heterogeneous PDFs with narrative text and tabular measurements.
• NHS information-governance rules separated internet-connected development from patient-data processing.
• The team lacked GPU capacity and therefore used smaller models and CPU-friendly approaches.

Architecture

The team built a governed GRID environment with separate development and staging Linux servers, then developed two extraction pipelines: smaller LLMs used as question-answering models for patient identifiers, and a few-shot SetFit table-row classifier for headers, values, and non-relevant rows. Prompt context improved entity extraction, and the staging server ran pipelines on patient data without internet access.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  Dev_server["GRID development server"] --> Pipeline_components["Pipeline components"]
  Pipeline_components --> Staging_server["GRID staging server"]
  Staging_server --> Report_corpus["Patient report corpus"]
  Report_corpus --> Identifier_models["Identifier QA models"]
  Report_corpus --> Table_classifier["Table row classifier"]
  Identifier_models --> Structured_data["Structured clinical data"]
  Table_classifier --> Structured_data

Components

Data flow

Pipelines and components are built on the internet-connected development server, then run against patient data on the disconnected staging server. LLM question-answering extracts identifiers from report text chunks so reports can link to structured data; table rows are labeled with a small annotated set, embedded by sentence transformers, and classified with an SVM head.

Why this architecture

The hospital needed a controlled environment for patient data and methods that worked with heterogeneous reports without GPU-heavy infrastructure. LLM QA absorbed textual heterogeneity for identifiers, while SetFit addressed table-row classification where a small labeled set and recurring table patterns existed.

Models used: Flan-T5 Large · Flan-T5 Small · BERT · RoBERTa · DistilRoBERTa · SetFit · sentence transformers · SVM classifier

Scale

• The report corpus contained about 15,000 cardiac MRI reports.
  “one of them is cardiac M reports it's almost like 15,000 reports which were produced over a period of 10 years by different Consultants”transcript, stated
• GRID reduced processing time from days on a laptop to a few hours.
  “grid was very crucial for this work because it pulled down the processing time from days on a work laptop to few hours”transcript, stated
• The SetFit experiment used 280 training examples and 80 validation examples.
  “our uh annotated data was we split them into 280 examples for training and 80 examples for validation”transcript, stated

Alternatives considered (roads not taken)

Manual review of unstructured reports

Rejected because: Manual extraction was time-consuming, required domain expertise, and could not scale to large volumes.

Traditional NLP techniques for heterogeneous report text

Rejected because: The reports varied over time, so accounting for all changes and maintaining reliability would be difficult.

Rule-based table extraction

Rejected because: Rules did not scale across the heterogeneous 10-year corpus and captured incorrect tabular information.

Patterns

privacy-preserving-processing (safety)

Patient-data processing occurs on an internet-disconnected staging server while development happens separately.

context-engineering (prompting)

Adding report context to extraction prompts improved model focus on the right patient identifier.

fine-tuning-adaptation (model-adaptation)

SetFit fine-tuned sentence-transformer representations on a small labeled table-row dataset.

eval-driven-development (evaluation)

The team compared model variants, prompt context, F1 validation, and attribution scores before choosing extraction/classification approaches.

Tradeoffs

Use smaller CPU-friendly LLMs

Chose small language models on available CPU infrastructure over GPU-heavy large-model processing.

Rationale: The team did not have GPU access, so extraction had to fit local compute constraints.

Use SetFit for table-row classification

Chose few-shot sentence-transformer classifier with an SVM head over rules over heterogeneous PDF table extraction.

Rationale: The data had recurring table patterns but limited labeled examples, making few-shot classification more appropriate than brittle rules.

Gotchas (what bit them)

Patient data could not be processed on ordinary internet-connected development machines.

Root cause: NHS information-governance constraints required careful separation of development and patient-data execution.

Resolution: Build GRID with separate development and staging servers.

Rule-based table extraction captured incorrect tabular information.

Root cause: PDF tables varied over 10 years and produced heterogeneous/incorrect table captures.

Resolution: Train a few-shot SetFit classifier on annotated table rows.

Some models hallucinated or performed poorly on identifiers.

Root cause: Small models and large-number identifiers were unevenly suited to the extraction task.

Resolution: Evaluate multiple small models and prompt context; avoid reporting models that hallucinated badly.

Evaluation

The team compared small LLMs for identifier extraction, tested prompt context effects, inspected attribution behavior, and validated SetFit table classification with F1 score on a held-out validation split.

Metrics: 15,000 reports over 10 years · 280 training examples and 80 validation examples · F1 validation for table-row classification · processing reduced from days to a few hours

Outcomes

• GRID made large-scale patient-report processing practical under governance constraints.
• Prompt context improved patient-identifier extraction for small LLMs.
• SetFit table-row classification outperformed the rule-based table approach in the described experiments.

The work targets nearly 15,000 cardiac MRI reports accumulated over 10 years at a pediatric NHS hospital.