Developing a Multilingual Ayurvedic Medical LLM: Challenges and Learnings

trigent_software · Healthcare · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Multilingual medical-domain LLM adaptation

Trigent tried to build a multilingual Ayurvedic medical LLM that could preserve ancient medical concepts, regional language nuance, cultural context, and modern diagnostic terminology, but the available data and modeling approach were not sufficient for the ambition.

Constraints:

• Ayurveda terminology, concepts, and diagnostic frames are culturally and linguistically specific.
• The target included Kannada, Tamil, Malayalam, Sanskrit-to-English translation, and English medical terminology.
• Medical advice requires contextual accuracy and culturally relevant responses.
• Regional language data and cultural nuance were hard to capture at sufficient quality.
• The project was framed as a learning journey rather than a success story.

Architecture

The team assembled Ayurvedic books, PubMed data, chatbot data, and synthetic patient interactions, initially aiming for a multilingual RAG/fine-tuning setup. After the multilingual plan became too difficult, they pivoted to English, fine-tuned GPT-2 Medium with about 347 million parameters on roughly 150,000 validated rows/draws of data, and treated the prototype as a learning step toward future data collection and academic collaboration.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  ayurvedic-source-corpus["Ayurvedic source corpus"] --> data-preparation-pipeline["Data preparation pipeline"]
  chatbot-and-pubmed-datasets["Chatbot and PubMed datasets"] --> data-preparation-pipeline["Data preparation pipeline"]
  synthetic-patient-interactions["Synthetic patient interactions"] --> data-preparation-pipeline["Data preparation pipeline"]
  data-preparation-pipeline --> gpt-2-medium-fine-tuning["GPT-2 Medium fine-tuning"]
  gpt-2-medium-fine-tuning --> ayurvedic-consultation-prototype["Ayurvedic consultation prototype"]
  ayurvedic-consultation-prototype --> future-regional-data-program["Future regional data program"]
  future-regional-data-program --> ayurvedic-source-corpus["Ayurvedic source corpus"]

Components

Data flow

Domain data came from chatbot rows, PubMed rows, public-domain Ayurvedic books, and synthetic patient interactions. The team intended to use multilingual and RAG-style data to inject terminology and context, then pivoted to English-only to make progress. The final training path fine-tuned GPT-2 Medium over a validated dataset and used the results to identify missing regional and cultural data.

Why this architecture

The team started with multilingual domain adaptation because Ayurveda depends on regional language, cultural nuance, and specialized medical concepts. They pivoted to English-only and a smaller GPT-2 Medium fine-tune because multilingual contextual accuracy was too hard with the available data.

Models used: GPT-2 Medium · IRGPT / Ayur GPT prototype · Synthetic data generation pipeline · RAG logic planned but not completed

Scale

• Ayurveda is described as about 5,000 years old.
  “about 5,000 years old and it is the oldest world's oldest holistic healing systems that are there”transcript, stated
• The chatbot dataset had 113,000 rows.
  “the chatbot data set had 113,000 rows”transcript, stated
• The team generated about 35,000 rows of synthetic patient interactions.
  “we also had generated synthetic data of about 35,000 rows um essentially patient interactions”transcript, stated
• The final data pool was about 150,000 rows/draws.
  “altogether we had 150,000 draws of data that was good enough for us to go ahead and high trade and see through”transcript, stated
• The fine-tuned model had 347 million parameters.
  “we had 347 million parameters”transcript, stated

Alternatives considered (roads not taken)

Build the multilingual version immediately

Rejected because: The team could not generate sufficiently contextual and culturally accurate regional-language responses.

Continue with full RAG/domain-text injection first

Rejected because: Although the team collected roughly 2GB of books for RAG logic, the roadblock came before the broader multilingual setup was solved.

Present the work as a success story

Rejected because: The presenter explicitly frames it as a lessons/failure story to drive continued iteration.

Patterns

fine-tuning-adaptation (model-ops)

The team fine-tuned GPT-2 Medium for an Ayurvedic medical-domain prototype.

synthetic-data-generation (data)

Synthetic patient interactions were generated to cover disease and symptom phrasings in colloquial target languages.

rag-retrieval (retrieval)

The team planned RAG-style use of public-domain Ayurveda books to inject terminology and context, though this remained part of the learning path rather than a successful deployment.

data-quality-loop (data)

The main lesson was that more targeted regional and cultural data is needed before the medical multilingual model can be reliable.

Tradeoffs

Pivot from multilingual to English

Chose English-only prototype over Immediate Kannada/Tamil/Malayalam multilingual support.

Rationale: The multilingual version could not yet produce culturally and contextually accurate responses, so narrowing scope enabled learning progress.

Fine-tune GPT-2 Medium

Chose A 347-million-parameter GPT-2 Medium prototype over Waiting for a larger or perfect medical LLM stack.

Rationale: The team prioritized getting a learning prototype working over claiming production success.

Continue after an unsuccessful first version

Chose More targeted data and academic collaboration over Stopping after the failed multilingual attempt.

Rationale: The team viewed the failure as a step toward a better context-aware Ayurveda model.

Gotchas (what bit them)

Multilingual medical domain adaptation failed on cultural/contextual accuracy.

Root cause: The data did not capture enough regional language nuance, cultural context, and Ayurvedic terminology.

Resolution: Pivot to English-only for the prototype and plan targeted regional data collection with academic partners.

Volume alone did not solve domain quality.

Root cause: The team had many rows and books, but culturally correct, contextual response generation remained hard.

Resolution: Treat the corpus as a starting point and return to targeted data design.

Medical-cultural terminology created translation and diagnosis risk.

Root cause: Ayurvedic concepts, modern medical terminology, symptoms, doshas, and local dialect all had to align.

Resolution: Simplify scope, preserve context-aware response goals, and iterate with better cultural data.

Evaluation

The source describes validation of a roughly 150,000-row dataset and qualitative learning from prototype limitations, but no production benchmark or clinical evaluation is provided.

Metrics: Dataset volume/validation · Contextual response accuracy · Cultural/regional nuance coverage · Ability to generate diagnosis/treatment/lifestyle responses · Prototype learning rather than production success

Outcomes

• Produced an English-only GPT-2 Medium Ayurvedic medical LLM prototype.
• Learned that multilingual cultural response quality required more targeted regional data.
• Deferred broader multilingual ambitions and planned continued work with academic partners.
• Documented the project as a failure/learning story rather than a production success.

The prototype used about 150,000 data rows/draws, 35,000 synthetic patient interaction rows, roughly 2GB of public-domain Ayurvedic books, and a 347-million-parameter GPT-2 Medium model.