Project Overview
Built a Korean medical specialized LLM by constructing a 449,500-sample domain dataset and applying SFT plus model merge on top of google/gemma2-9b.
Problem Definition & Goals
- Problem: General-purpose LLMs often hallucinate on medical questions and show weak understanding of Korean clinical terminology, which is risky in trust-critical healthcare settings.
- Goal 1: Build a reliable Korean medical dataset from licensing exams, disease dictionaries, and medical law documents.
- Goal 2: Maximize medical expertise while preserving general reasoning ability via SFT and model merge.
- Goal 3: Deliver a practical Korean medical domain LLM pipeline for specialized QA tasks.
Key Features & Contributions
- Hybrid Dataset Engineering: Built 449,500 training samples from KorMedMCQA, Asan disease dictionary crawls, translated MedExpQA/UltraMed, and legal synthetic QA generated with Solar API.
- Data Freshness: Added newly labeled 2025 Korean medical licensing exam questions.
- Data-Centric Strategy: Compensated for limited expert annotation by combining trusted institutional sources with high-quality translated medical corpora.
- Instruction Tuning Optimization: Structured data as QA/chat instructions instead of plain text to improve instruction-following quality.
Technical Challenges & Solutions
- Catastrophic Forgetting: Heavy medical-only tuning degraded general knowledge performance.
- Mixing Ratio Search: Derived a stable Korean:English ratio of 60:40 through iterative experiments and prior-paper analysis.
- Domain-General Balance: Applied approximately General:Medical = 2.5:1 to maintain usability outside narrow medical prompts.
- Merge Strategy: Merged SFT model with a general base to retain conversational capability while keeping domain gains.
Results & Learnings
- Result: Completed an end-to-end Korean medical LLM training pipeline with KorMedMCQA and custom datasets.
- Result: Optimized the trade-off between domain adaptation performance and general capability through data mixing and model merge.
- Insight: Performance was more sensitive to balanced data composition than to simply increasing domain data volume.
- Insight: High-quality translation and synthetic data are key levers for improving minor-language medical LLMs.