Building Healthcare-Specific LLM Pipelines for Oncology Patient Timelines

roche_diagnostics_/_john_snow_labs · Healthcare · year n/a · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Oncology clinical data abstraction

Roche Diagnostics and John Snow Labs explored healthcare-specific LLM pipelines for turning fragmented oncology EHR notes into patient chemotherapy timelines and guideline-relevant structured data. The core challenge is that healthcare data is unstructured, multilingual, semantically ambiguous, and expensive to annotate, while clinical use requires accurate temporal relations, privacy awareness, and responsible human oversight.

Constraints:

• Most healthcare data is unstructured and not quickly consumable at the point of care.
• Clinical notes vary across report formats, languages, abbreviations, synonyms, and ambiguous healthcare concepts.
• Chemotherapy timeline extraction requires mapping medication events to time expressions and normalizing temporal relations.
• Healthcare-specific LLMs must handle privacy, legal, bias, hallucination, and diagnostic-safety risks.
• The work sits in a clinical decision-support context where multidisciplinary review and responsible use matter.

Architecture

The pipeline sits around Roche's navify oncology clinical hub. Fragmented EHR notes feed an AI-assisted data abstraction layer that uses NLP, machine learning, and healthcare-specific LLM prompts to detect medical concepts, extract chemotherapy events and time expressions, classify temporal relations, normalize time expressions, and refine patient-level timelines. The structured timeline can then support navify's longitudinal patient view and smart navigation to relevant NCCN guideline sections. For relation extraction, the team tested structured prompt variants and healthcare-specific models such as JSL Med Llama 3 8B, using zero-shot prompting where labeled data is limited.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  oncology-care-team[Oncology care team] --> navify-oncology-clinical-hub[navify oncology clinical hub]
  navify-oncology-clinical-hub --> nccn-guideline-navigator[NCCN guideline navigator]
  navify-oncology-clinical-hub --> structured-patient-timeline[Structured patient timeline]
  unstructured-ehr-notes[Unstructured EHR notes] --> ai-assisted-abstraction-pipeline[AI-assisted abstraction pipeline]
  ai-assisted-abstraction-pipeline --> medical-concept-detector[Medical concept detector]
  ai-assisted-abstraction-pipeline --> chemotherapy-event-extractor[Chemotherapy event extractor]
  ai-assisted-abstraction-pipeline --> time-expression-extractor[Time expression extractor]
  ai-assisted-abstraction-pipeline --> prompted-relation-extractor[Prompted relation extractor]
  chemotherapy-event-extractor --> prompted-relation-extractor
  time-expression-extractor --> prompted-relation-extractor
  prompted-relation-extractor --> healthcare-specific-llm[Healthcare-specific LLM]
  prompted-relation-extractor --> temporal-normalization-stage[Temporal normalization stage]
  temporal-normalization-stage --> structured-patient-timeline
  multidisciplinary-governance[Multidisciplinary governance] --> ai-assisted-abstraction-pipeline

Components

Data flow

Unstructured oncology notes from EHR sources enter the abstraction pipeline. NLP and machine-learning components auto-detect medical concepts and translate free text into structured data. The timeline task decomposes into chemotherapy event extraction, time-expression extraction, temporal-relation classification, time-expression normalization, and patient-level timeline refinement. Prompt templates guide the LLM to label relations between entity pairs, and the resulting patient timeline can be consumed by navify Oncology Clinical Hub to show a longitudinal journey and open patient-relevant NCCN guideline sections.

Why this architecture

The architecture is staged because a patient timeline is not a single generation problem: chemotherapy events, time expressions, temporal links, normalization, and patient-level refinement each need explicit handling. Healthcare-specific models are favored because the source claims they are more accurate for clinical text than generic models. Prompt templates allow zero-shot relation extraction where class-level training data is scarce, while navify integration connects structured timelines to clinician-facing longitudinal views and guideline navigation.

Models used: JSL Med Llama 3 8B version 1.0 · Llama 2 · Mistral 7B · healthcare-specific large language models

Scale

• Roche is described as a 126-year-old company.
  “Ro is a 126 years old company with headquarters in Basel Switzerland”transcript, stated
• Roche reports 27 billion diagnostic tests performed.
  “we have 27 billion diagnostic tests perform improving patient care”transcript, stated
• Roche reports over 16 million patients treated with Roche medicines last year.
  “we have over 16 million patients treated with ro medicines last year”transcript, stated
• The mock oncology timeline spans 2014 through 2020.
  “in 2014 the patient was diagnosed with breast cancer”transcript, stated

  “in 2020 the patient had second relapse marked by new bone metastasis”transcript, stated
• The SemEval task covered breast, ovarian, and skin cancer EHRs.
  “from ehrs of patients with breast ovian and skin cancers”transcript, stated
• The presentation highlights JSL Med Llama 3 8B version 1.0 as a healthcare-specific model example.
  “JSL Med Lama 3 uh 8B version 1.0”transcript, stated

Alternatives considered (roads not taken)

Manual data entry and annotation.

Rejected because: Manual annotation is labor-intensive and subjective, and the goal is to reduce manual data-entry burden.

Use a generic non-healthcare LLM.

Rejected because: The presenter states healthcare-specific LLMs are more accurate for healthcare-domain unstructured data.

Prompt relations from a separate list of drugs.

Rejected because: The second prompt variant had trouble labeling relationships from distinct lists, so prompt one was used for submission and evaluation.

Treat generative AI as only a query UI.

Rejected because: The presenter frames most current gen-AI solutions that way, but this pipeline needs structured timeline abstraction and guideline support.

Patterns

structured-output-contract (extraction)

The team decomposes clinical text into events, time expressions, temporal relations, normalized times, and patient-level timelines.

prompt-chaining (llmops)

Structured prompt templates guide relation labeling between clinical entity pairs as one stage in the broader timeline pipeline.

fine-tuning-adaptation (model-selection)

The project uses healthcare-specific LLMs such as JSL Med Llama 3 rather than relying only on general-domain LLMs for clinical relation extraction.

eval-driven-development (quality)

The team compares prompt variants and model choices using a shared task with gold annotations, development/test splits, and relation-extraction evaluation.

privacy-preserving-processing (healthcare)

The case explicitly treats privacy and legal risk from personal healthcare data as a deployment constraint for LLM systems.

safety-policy-enforcement (clinical-safety)

The presenter calls out hallucination, false treatment suggestions, bias, and the need for multidisciplinary responsible use in healthcare LLMs.

Tradeoffs

Use healthcare-specific LLMs.

Chose domain-specific models such as JSL Med Llama 3 over general non-healthcare LLMs.

Rationale: The presenter states healthcare-specific models are more accurate for healthcare-domain unstructured data.

Use zero-shot prompt templates for relation extraction.

Chose prompt-templated zero-shot extraction over requiring explicit training data for every class or category.

Rationale: Zero-shot templates produced reasonably high precision while avoiding explicit class-level training data requirements.

Use a pair-based relation prompt.

Chose Prompt 1 relation labeling from entity pairs over Prompt 2 relation labeling from separate drug lists.

Rationale: The separate-list prompt had relationship-labeling issues, so the pair prompt was used for final submission and evaluation.

Provide guideline smart navigation based on patient data.

Chose patient-data-driven NCCN section navigation over manual guideline lookup by clinicians.

Rationale: Automatically opening relevant guideline sections saves clinician time and supports evidence-based care decisions.

Gotchas (what bit them)

Unstructured healthcare data is not point-of-care ready.

Root cause: Clinical data appears in diverse formats, languages, abbreviations, synonyms, and ambiguous concepts.

Resolution: Use OCR/NLP and healthcare-specific LLM relation extraction to translate free text into structured, enriched data.

Chemotherapy timeline extraction is multi-stage rather than a simple text generation task.

Root cause: Medication events must be mapped to terminologies and assigned temporal extent from note dates and modifiers.

Resolution: Decompose extraction into event extraction, time expression extraction, temporal relations, normalization, and timeline refinement.

A drug-list prompt variant failed to label relations well.

Root cause: Separating drugs into a distinct list made relationship labeling harder.

Resolution: Use the pair-based relation-labeling prompt for submission and evaluation.

Healthcare LLMs carry high-stakes risks beyond ordinary extraction errors.

Root cause: LLMs can hallucinate, expose privacy/legal issues, and amplify biased training data.

Resolution: Treat healthcare LLM deployment as a multidisciplinary governance problem involving clinical, legal, regulatory, technical, and patient expertise.

LLMs are expensive and operationally demanding at healthcare scale.

Root cause: Computational resources, preprocessing, specialized talent, and infrastructure costs add up.

Resolution: The presentation acknowledges this as a production consideration rather than claiming it is fully solved.

Evaluation

The presentation discusses evaluation through the 2024 SemEval chemotherapy treatment timeline extraction task. It used gold annotations for chemotherapy events, time expressions, temporal relations, and patient-level timelines; split gold data into train, development, and test sets; compared prompt variants; and reported that zero-shot prompting achieved reasonably high precision. Quantitative performance details are not provided in the transcript.

Metrics: chemotherapy event extraction · time expression extraction · temporal relation classification · time expression normalization · patient-level timeline refinement · precision · prompt variant performance · healthcare-specific versus non-healthcare model suitability

Outcomes

• The team built an AI-assisted abstraction approach for translating unstructured oncology text into structured enriched data.
• The timeline task was decomposed into chemotherapy event extraction, time expression extraction, temporal relation classification, time normalization, and patient timeline refinement.
• Prompt 1, relation labeling from entity pairs, was used for submission and evaluation after Prompt 2 struggled with distinct drug lists.
• Zero-shot prompt templates reportedly achieved reasonably high precision.
• The presenter identifies JSL Med Llama 3 8B version 1.0 as production-ready and adaptable for healthcare-specific unstructured data.
• The presenter frames healthcare LLM use as an ongoing journey requiring multidisciplinary responsibility.

The talk cites Roche's 126-year history, 27 billion diagnostic tests, over 16 million patients treated with Roche medicines last year, a 2014-2020 mock oncology timeline, the 2024 chemotherapy timeline extraction shared task, breast/ovarian/skin cancer EHRs, and JSL Med Llama 3 8B version 1.0.