medical-rag - GitLite

KO | EN

gitlite — search

#ruby #video #monitoring #containers #redis #artificial-intelligence #framework #windows #anthropic #shell

medical-rag

(★ 12)

Repository for the implementation of a medical Retrieval-Augmented Generation (RAG) system.

파일 탐색기

ZIP 다운로드

- evidence_filtering.py
- retrieval.py
- retrieval_using_rationale_query.py
- - pubmed.py
  - statpearls.py
  - textbooks.py
  - wikipedia.py
- config.py
- medrag.py
- template.py
- utils.py
.gitignore
README.md
requirements.txt

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`conda activate`

설명 보기 ▼

Activate a conda environment.
See also: `conda deactivate`.

conda activate myenv

Activate an existing environment named `myenv`:

conda activate {{path/to/myenv}}

Activate an existing environment located at custom path:

conda activate --stack myenv

Stack `myenv` environment on top of a previous environment making libraries/commands/variables from both accessible:

🔍

`conda create`

설명 보기 ▼

Create new conda environments.

conda create {{[-y|--yes]}} {{[-n|--name]}} py39 python=3.9 "numpy>=1.11" scipy

Create a new environment named `py39`, install Python 3.9, NumPy v1.11 or above in it, and the latest stable version of SciPy. Say yes to all confirmations:

conda create {{[-n|--name]}} myenv --file {{file1.yml}} --file {{file2.yml}}

Create a new environment named `myenv` and install packages listed in files:

conda create {{[-p|--prefix]}} {{path/to/myenv}}

Create a new environment at a custom path (i.e. prefix):

🔍

`pip install`

설명 보기 ▼

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repository implements a lightweight and modular **Retrieval-Augmented Generation (RAG)** pipeline for **medical and clinical tasks**.  
It is built on top of the [MedRAG toolkit](https://github.com/Teddy-XiongGZ/MedRAG) and adds a small set of practical extensions:

- 🔥 Support for recent dense retrievers (e.g., [*Qwen3-embedding*](https://huggingface.co/collections/Qwen/qwen3-embedding))
- 🔥 Evidence filtering
- 🔥 Query reformulation

The goal is to provide a simple and configurable reference pipeline for experimenting with RAG in the medical domain, while keeping the codebase easy to understand and extend.

---

## 🔧 Installation

### 1. Create environment
```bash
conda create -n medical-rag python==3.10
conda activate medical-rag
```

### 2. Install PyTorch (CUDA version may vary)
```bash
pip install torch==2.9.0 torchvision==0.24.0 torchaudio==2.9.0 --index-url https://download.pytorch.org/whl/cu128
```

### 3. Install remaining dependencies
```bash
pip install -r requirements.txt
```

## ⚡ Quick Start

This repository provides core components for building RAG systems in the medical domain.  
Each component has a minimal runnable example under the **`examples/`** directory.

### Retrieval
Retrieves top-k candidate passages from a medical corpus using sparse/dense embeddings  
(e.g., *BM25*, *MedCPT*, *Qwen3-embedding*, etc.).

- Input: user query  
- Output: list of retrieved passages (+ scores)
- Example: `examples/retrieval.py`

### Evidence Filtering
Removes irrelevant or low-quality passages from the retrieved set. 
This step is particularly helpful when top-k retrieval includes substantial noise, which is a common issue in clinical and biomedical corpora.

Our implementation uses a fine-tuned evidence-filtering model (based on Llama-3.1-8B-Instruct), trained to judge whether a candidate passage contains supporting evidence for a given medical query.
The model outputs *“Yes”* (contains evidence) or *“No”* (does not contain evidence).
Model weights are publicly available on the [Hugging Face repository](https://huggingface.co/Yale-BIDS-Chen/Llama-3.1-8B-Evidence-Filtering).

- Input: query-passage pair  
- Output: Yes or No  
- Example: `examples/evidence_filtering.py`

### Retrieval with Query Reformulation (Rationale Querying)
Rewrites the original query into a rationale-style query, which corresponds to the model's intermediate reasoning or justification for the answer.
Instead of retrieving evidence using the possibly short or ambiguous initial query, the system uses a richer, more informative rationale-based query, often leading to more accurate and semantically aligned retrieval.

- Input: original user query  
- Output: rationale-style reformulated query  
- Example: `examples/retrieval_using_rationale_query.py`

For the motivation behind rationale-based retrieval and further methodological details, please refer to [this reference](https://aclanthology.org/2025.naacl-long.635/).

## 🙌 Citation

If this repository is useful for your work, please cite:

### Primary Project

```bash
@article{kim2025rethinking,
  title={Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights},
  author={Kim, Hyunjae and Sohn, Jiwoong and Gilson, Aidan and Cochran-Caggiano, Nicholas and Applebaum, Serina and Jin, Heeju and Park, Seihee and Park, Yujin and Park, Jiyeong and Choi, Seoyoung and others},
  journal={arXiv preprint arXiv:2511.06738},
  year={2025}
}
```

## 📬 Contact

For questions or suggestions, feel free to open an issue or email us (`hyunjae.kim@yale.edu` or `qingyu.chen@yale.edu`).