Human-AI Synergy in Pharmaceutical Research and Document Processing

merantix · Healthcare · 2023 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: pharmaceutical safety assessment, regulated document review, and enterprise knowledge work

Merantix and Red Tech Cortex describe production AI systems for high-stakes domains where full automation is risky: pharmaceutical teams need to detect rare behavior in rodent videos, legal and compliance teams need semi-automated document review with no tolerance for errors, and enterprises need searchable internal knowledge without locking themselves to one AI provider.

Constraints:

• drug-safety analysts must inspect large volumes of sequential video for rare toxicity-related behavior
• the system must learn from scarce human examples rather than rely on abundant labeled data
• legal and compliance document review cannot allow errors, so humans must retain authority where confidence is insufficient
• the same interaction pattern needs to work across video, time-series, tabular, scientific, and document modalities
• enterprise knowledge workers need access to Google Drive, OneDrive, Slack, databases, and legal sources through a reusable assistant interface
• enterprise deployments need EU-based processing, access-level inference, confidentiality handling, and provider flexibility

Architecture

The presentation describes two related architectures. Merantix's pharmaceutical workflow starts with a human expert describing the rare behavior or document signal they want, then uses an interactive learning loop that requests feedback on uncertain snippets, learns from those examples, and gradually shifts tasks from manual annotation to AI-assisted review to autonomous annotation where the behavior is well learned. The same pattern is applied to high-accuracy legal and compliance document workflows by leaving humans responsible for cases the model cannot safely take over. Red Tech Cortex presents an enterprise knowledge platform where companies create reusable persona agents, connect internal repositories such as Google Drive, OneDrive, Slack, and databases, index content continuously, enrich files with metadata such as department/access/confidentiality, retrieve answers through RAG-style search, and route across multiple AI providers while keeping processing inside the EU.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  Human_domain_expert["Human domain expert"] --> Interactive_learning_system["Interactive learning system"]
  Video_and_document_inputs["Video and document inputs"] --> Interactive_learning_system
  Interactive_learning_system --> Feedback_snippets["Feedback snippets"]
  Feedback_snippets --> Human_domain_expert
  Foundation_model_priors["Foundation model priors"] --> Interactive_learning_system
  Interactive_learning_system --> Autonomous_annotation_engine["Autonomous annotation engine"]
  Interactive_learning_system --> Final_review_queue["Final review queue"]
  Autonomous_annotation_engine --> Final_review_queue
  Final_review_queue --> Human_domain_expert
  Enterprise_connectors["Enterprise connectors"] --> Ongoing_indexing_layer["Ongoing indexing layer"]
  Ongoing_indexing_layer --> Metadata_extractor["Metadata extractor"]
  Metadata_extractor --> RAG_search_layer["RAG search layer"]
  Ongoing_indexing_layer --> RAG_search_layer
  RAG_search_layer --> Persona_agents["Persona agents"]
  RAG_search_layer --> Provider_aggregation_layer["Provider aggregation layer"]
  Provider_aggregation_layer --> Persona_agents

Components

Data flow

For the pharmaceutical video case, a human expert defines the behavior to find, the system searches video or time-series snippets, asks for feedback on examples it is uncertain about, and uses that interaction history to fine-tune recognition until the task can move from manual labeling to autonomous annotation for learned behaviors. For enterprise knowledge work, Red Tech Cortex connects company repositories, indexes them on an ongoing basis, extracts metadata, retrieves relevant content for a user question or reusable persona agent, and sends prompts to an AI provider selected without reconnecting the underlying knowledge sources.

Why this architecture

The Merantix systems use progressive human-AI collaboration because rare-event pharmaceutical annotation and regulated document review cannot be safely solved by a single full-automation jump. Human feedback lets the model learn from scarce examples, while human fallback preserves accuracy where errors are unacceptable. Red Tech Cortex uses connected knowledge repositories, ongoing indexing, metadata enrichment, and provider aggregation because enterprise knowledge work depends on existing internal sources, access semantics, data sovereignty, and the ability to change AI backends without reconnecting all enterprise integrations.

Models used: interactive behavior-recognition models · foundation models for tabular data · foundation models for time series · AI-for-science foundation models · retrieval-augmented generation models · multiple AI providers behind Red Tech Cortex

Scale

• Merantix says it has delivered more than 150 industry AI projects.
  “we're doing this um already for a couple of years in over 150 projects. uh we do um strategy consulting, corporate uh research but also tailor made uh solutions”transcript, stated
• The Boehringer Ingelheim case involves video analysis at very large scale.
  “they have to look at videos of rodents um at a very large uh scale to um figure out should you give this substances to humans or better not.”transcript, stated
• Red Tech Cortex claims about 2 million users.
  “we have currently around 2 million users it's counting and going up all thanks to organic growth.”transcript, stated
• Red Tech Cortex claims 30 new-hire-equivalent productivity gain for every 100 onboarded users.
  “for every 100 on board users they're seeing a productivity gain of 30 new user in hires uh which is really really important to companies”transcript, stated
• Red Tech Cortex says it began building the knowledge-search concept in 2021.
  “similar to the assistance that we have today chat and others cloud AI um but when we start doing like a start back in 2021 it was like a really big concept”transcript, stated

Alternatives considered (roads not taken)

Full automation from the start

Rejected because: The speaker explicitly frames human-AI synergy as a safer alternative for tasks where rare events, scarce labels, and regulated errors make full automation inappropriate.

Pure manual annotation or document review

Rejected because: The workflow starts manually but then has the system surface snippets and learn from feedback so humans do not need to perform the entire sequential task forever.

Single-provider enterprise AI platform

Rejected because: Red Tech Cortex argues that companies need to switch or test different providers without rebuilding their knowledge connections.

Plain file search over enterprise drives

Rejected because: The platform positions connected indexing and semantic question answering as replacing the old behavior of searching Google Drive manually.

Patterns

human-in-the-loop-gate (oversight)

The system deliberately keeps humans in the loop while the AI is learning and for regulated cases it cannot handle safely, rather than replacing expert review outright.

eval-driven-development (quality)

Human feedback on selected snippets acts as the supervision signal for improving detection of rare behaviors and deciding when the system can move toward autonomous annotation.

fine-tuning-adaptation (model-adaptation)

The speaker describes using foundation-model priors so the system can adapt more quickly to a current intent with fewer annotations.

rag-retrieval (retrieval)

Red Tech Cortex connects enterprise repositories, continuously indexes them, and lets users ask questions over the indexed knowledge base through reusable agents.

model-routing (model-operations)

The enterprise platform abstracts knowledge connections from the AI provider so customers can test or switch providers for specific prompts.

privacy-preserving-processing (governance)

Red Tech Cortex emphasizes EU-based processing and metadata about access level and confidentiality for enterprise files.

Tradeoffs

Progressive automation instead of immediate full automation

Chose manual-to-assisted-to-autonomous workflow over fully automated behavior detection from the start.

Rationale: The task is rare-event, high-stakes, and initially label-scarce, so the system needs human feedback until it has learned the target behavior well enough.

Human fallback for regulated document work

Chose semi-automated review where humans handle uncertain cases over forcing document review through an autonomous model.

Rationale: Legal and compliance workflows are explicitly described as not allowing errors, so the architecture trades maximum automation for controlled reliability.

Provider aggregation instead of provider lock-in

Chose shared knowledge connections with swappable AI providers over one AI provider tied to the enterprise data integrations.

Rationale: Customers want to test which provider works best for specific prompts without rebuilding their knowledge connections and integrations.

Metadata-enriched indexing instead of plain document search

Chose ongoing indexing plus inferred metadata for department, access, and confidentiality over manual search through drives and folders.

Rationale: The platform addresses the fact that information is locked inside many enterprise files and not easy to find or govern using ordinary drive search.

Gotchas (what bit them)

Rare-event annotation remains hard even when AI is available.

Root cause: Drug-safety review requires analysts to find rare behaviors in large sequential video streams, so labels are scarce and human attention is expensive.

Resolution: Use an interactive system that asks for expert feedback on selected snippets and gradually automates the learned behavior.

High-stakes document workflows cannot be treated as ordinary automation tasks.

Root cause: Legal and compliance review has low tolerance for errors and often lacks training data.

Resolution: Apply the same semi-automated human-feedback mechanism to documents so humans handle cases the model cannot safely take over.

Foundation models outside text need modality-specific caution.

Root cause: The speaker calls time series challenging because it is probably not a single modality, so transfer from one experiment to another is not trivial.

Resolution: Use learned priors from related experiments to reduce annotation requirements, but treat each new intent as an adaptation problem.

Enterprise knowledge agents inherit access-control and confidentiality problems from file repositories.

Root cause: Corporate files contain hidden information, department context, and different access or confidentiality levels.

Resolution: Generate metadata from files and infer access level and confidentiality classification as part of the search/indexing platform.

Evaluation

The presentation does not describe a formal evaluation harness. It uses staged human feedback, progressive autonomy, and claims about scale/productivity as evidence of system behavior, while the summary explicitly warns that Red Tech Cortex productivity metrics are marketing claims requiring independent verification.

Metrics: transition from manual annotation to feedback snippets to autonomous annotation · Merantix claims over 150 AI projects · Red Tech Cortex claims about 2 million users · Red Tech Cortex claims productivity gain of 30 new hires per 100 onboarded users

Outcomes

• Merantix reports a Boehringer Ingelheim system that can learn rodent behavior detection from human descriptions and feedback.
• The same progressive automation mechanism is applied to legal and compliance document review where errors are not allowed.
• Red Tech Cortex reports an enterprise knowledge platform with around 2 million users and ongoing indexing across connected repositories.
• The presentation frames human-AI co-creation, rather than tool-only usage, as the important production UX shift.

The case is broad and partially promotional: Merantix claims over 150 AI projects and Red Tech Cortex claims 2 million users and 30 new-hire-equivalent productivity per 100 onboarded users, but the detailed Boehringer and Red Tech architectures are described at presentation level rather than with independent benchmarks.