Multi-Agent AI Development Assistant for Clinical Trial Data Analysis

astrazeneca · Healthcare · 2024 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Pharmaceutical R&D data analysis and clinical-trial operations

AstraZeneca needed clinical and R&D users to query drug-development data products in natural language, generate trustworthy analyses, and expand from a clinical-trial assistant into cross-domain R&D workflows without worsening hallucinations, performance, data-governance risk, or user-adoption fatigue.

Constraints:

• The environment is pharmaceutical R&D, where output correctness, patient impact, data governance, and regulated decision-making matter.
• Clinical, regulatory, safety, quality, and other R&D domains use different terminology and data products.
• The assistant had to build on existing trusted data platforms rather than create another disconnected AI initiative.
• The team needed transparent reasoning and source traceability so users could verify generated SQL and outputs.
• User adoption required change management because employees were already experiencing technology and AI fatigue.

Architecture

AstraZeneca built Development Assistant on the existing Drug Development Data Platform (3DP), initially as a single natural-language-to-SQL agent for clinical-trial data. The system exposes reasoning steps, SQL, source tables, and visual results so users can verify answers. As the team expanded beyond clinical trials, the single-agent approach showed hallucination, latency, and complex-problem limits, so AstraZeneca moved to an Amazon Bedrock multi-agent setup with a supervisor agent, terminology agent, and domain sub-agents for clinical, regulatory, quality, and other R&D areas. Controlled vocabulary and metadata augmentation improve query generation, while SMEs and product managers validate outputs in sprints and workflows.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  UI["Development Assistant UI"] --> Platform["3DP data platform"]
  UI --> Single["Single-agent query path"]
  Single --> Platform
  Platform --> Vocab["Controlled vocabulary augmentation"]
  Platform --> Metadata["Metadata augmentation"]
  UI --> Supervisor["Amazon Bedrock supervisor agent"]
  Supervisor --> Term["Terminology agent"]
  Term --> Domains["Domain sub-agents"]
  Domains --> Platform
  Domains --> Metadata
  Vocab --> Term
  UI --> SME["SME validation loop"]
  Domains --> SME
  SME --> Change["Change management program"]

Components

Data flow

A user asks a natural-language clinical or R&D question in Development Assistant. The assistant sits on 3DP data products rather than a new data silo, augments the request with controlled vocabulary and improved metadata, and uses the agent path to generate SQL, retrieve source data, and return visualized or summarized results. In the expanded architecture, Amazon Bedrock receives the prompt, the supervisor routes it to terminology and domain sub-agents, and clinical or business SMEs validate whether the insight matches how their persona would reason over the data.

Why this architecture

AstraZeneca chose to build on 3DP because the platform already had users, security, privacy work, extensible data products, and trust. The first single-agent system proved enough for a quick clinical text-to-SQL use case, but broader R&D expansion pushed too much data and terminology ambiguity into one agent. The Bedrock supervisor/sub-agent architecture let the team split domain context, route terminology-sensitive prompts, and keep the assistant extensible while SMEs and product managers validated outputs in production workflows.

Models used: Amazon Bedrock multi-agent architecture · Supervisor agent · Terminology agent · Clinical domain agent · Regulatory domain agent · Quality domain agent · Natural-language-to-SQL LLM agent

Scale

• AstraZeneca's corporate ambition was to deliver 20 new medicines by 2030.
  “by 2030, we will deliver 20 new medicines, be an 80 billion company, and be carbon negative”transcript, stated
• The existing 3DP platform already had thousands of users before Development Assistant was layered on top.
  “our existing platform had already thousands of users so we knew that there was a gain trust in this platform”transcript, stated
• The initial clinical-trial example was built in about a week.
  “what are the top five countries with the most clinical trial uh, sites? And I want to visualize this in a pie chart. Now, for the sake of this, it's going to be a very quick demo”transcript, stated

  “and we were able to do this in a week. Okay.”transcript, stated
• The system went from concept to production MVP in six months.
  “we had from concept to P to production. We hit production and MVP last year within six months.”transcript, stated
• The change-management program includes a four-tier AI accreditation program.
  “we have certifications that are coming out and we have folks, we're rewarding folks with certificates as long as they complete the curriculum. Um, there's four tiers of this.”transcript, stated

Alternatives considered (roads not taken)

Build a new AI data platform from scratch

Rejected because: The team used 3DP because it already had scalable data products, trust, security, privacy work, and extensible infrastructure.

Keep the assistant as a single agent while expanding across clinical, regulatory, quality, and other R&D domains

Rejected because: The single-agent path could not handle the cross-domain data volume, caused more hallucinations and weaker responses, created bottlenecks, and struggled with complex problems.

Rely only on raw data products and generic LLM prompting

Rejected because: Domain acronyms and weak column metadata caused quality problems, so the team added controlled-vocabulary and metadata augmentation.

Treat the project as only a technical rollout

Rejected because: The team paired technical deployment with HR, legal, business, AI accelerator, showcase, follow-up, and accreditation programs to manage adoption and correct use.

Patterns

multi-agent-orchestration (architecture)

Development Assistant moved from a single-agent clinical tool to a Bedrock supervisor and sub-agent architecture spanning terminology, clinical, regulatory, quality, and other R&D domains.

context-engineering (context)

The team improved the model's working context with controlled vocabulary and column metadata so domain terms and data products were interpreted correctly.

eval-driven-development (quality)

AstraZeneca exposed reasoning and SQL for verification, used domain experts to validate insights, and benchmarked changes every sprint against prior versions.

human-in-the-loop-gate (governance)

Clinical and business SMEs actively validate whether the assistant's insights match real domain-persona reasoning before adoption expands.

safety-policy-enforcement (governance)

The assistant is built on a platform where security and privacy had already been factored in, and validation rigor is emphasized because clinical answers must be correct.

Tradeoffs

Where to host the assistant's data foundation.

Chose Build on 3DP and its existing data products. over A new standalone AI data foundation..

Rationale: 3DP already had trust, users, security/privacy work, FAIR-oriented data products, and extensible infrastructure, which accelerated production.

How to expand beyond clinical-trial questions.

Chose Bedrock supervisor agent with specialized sub-agents. over Continuing with one general agent for all R&D domains..

Rationale: The source says one agent could not handle the data volume and domain complexity without hallucination, lag, and incorrect complex-problem handling.

How much internal reasoning to expose to users.

Chose Expose reasoning steps, SQL, and source traceability. over Only showing final answers and charts..

Rationale: In the regulated clinical setting, transparent intermediate steps allow quality checks and source verification.

How to roll out AI assistance to domain users.

Chose Pair production deployment with change-management, showcases, accreditation, and workflow pilots. over Relying on the tool's availability alone to drive adoption..

Rationale: The team saw change fatigue and adoption challenges as systemic issues that needed structured communication, training, and follow-up.

Gotchas (what bit them)

A single agent did not scale cleanly from clinical trials to broader R&D domains.

Root cause: The combined data volume, terminology ambiguity, and cross-domain complexity increased hallucinations, weakened responses, and introduced bottlenecks.

Resolution: AstraZeneca moved to a Bedrock supervisor/sub-agent architecture with domain and terminology specialization.

Life-sciences acronyms and terminology created retrieval and query-generation errors.

Root cause: The same terms can mean different things across domains, and domain vocabularies must be mapped to the right data products.

Resolution: The team augmented the LLM with controlled vocabulary and added a terminology step in the multi-agent flow.

Weak metadata made it harder for the LLM to generate the right query.

Root cause: Column labels and data descriptions were not consistently explicit enough for natural-language-to-SQL generation.

Resolution: The team improved metadata descriptions as a second augmentation strategy alongside vocabulary enrichment.

Production value depended on user trust, not just working model calls.

Root cause: Change fatigue was real and domain users needed confidence that answers were correct and useful in their workflow.

Resolution: AstraZeneca used SME validation, product-manager workflow pilots, showcases, follow-ups, and an AI accreditation program.

Evaluation

AstraZeneca evaluates Development Assistant through exposed reasoning and SQL, source verification, clinical/business SME validation, sprint-based testing, and benchmarking against prior versions. The presentation does not report formal accuracy or hallucination-rate metrics.

Metrics: Concept to production MVP in six months · Initial demo use case built in about one week · Top-five clinical-trial-site query with pie-chart visualization · Sprint-by-sprint testing and version benchmarking · SME validation of whether insights match clinical personas · Four-tier AI accreditation program

Outcomes

• Development Assistant reached production MVP in six months.
• The assistant expanded from an initial clinical-trial use case toward regulatory, quality, and other R&D domains.
• Controlled vocabulary and metadata augmentation improved robustness for domain terminology and text-to-SQL generation.
• The team replaced a lagging single-agent path with a more flexible supervisor/sub-agent architecture.
• Product managers and SMEs began using and validating the tool in real workflows.

The transcript gives scale through AstraZeneca's 20-new-medicines-by-2030 ambition, an existing 3DP platform with thousands of users, a one-week initial demo, six months to production MVP, expansion to additional R&D domains, and a four-tier AI accreditation program.