TokenCleaning - GitLite

KO | EN

gitlite — search

#ai #llm #golang #android #docker #macos #nodejs #java #windows #cli #vue #awesome-list #security #open-source

TokenCleaning

(★ 54)

[ICML 2025] Official implementation of paper "Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning"

파일 탐색기

최종 버전 다운로드 (.zip)

- full_baseline.sh
- random_baseline.sh
- rho_baseline.sh
- calculate_loss.sh
- finetune.sh
- - run_eval.py
- - run_eval.py
- - data.py
  - evaluation.py
  - execution.py
  - run_eval.py
- - examplars.py
  - run_eval.py
- - instructions.py
  - instructions_registry.py
  - instructions_util.py
  - run_eval.py
- - examplars.py
  - minerva_utils.py
  - run_eval.py
  - utilities.py
- - evaluation.py
  - execution.py
  - mbpp.py
  - run_eval.py
- - categories.py
  - run_eval.py
  - run_eval_backup.py
- - run_eval.py
- - configs.py
  - metrics.py
  - presets.py
  - run_eval.py
  - utilities.py
- - run_eval.py
- - classify_refusal.py
  - run_eval.py
- dispatch_openai_requests.py
- predict.py
- templates.py
- utils.py
- token_cleaning_overivew.jpg
- token_selection_2.0.pdf
- fsdp_config.yaml
- calculate_token_loss.py
- finetune.py
- finetune_trainer.py
- generate_subset.py
- generate_token_label.py
- merge_lora.py
.gitignore
fixed_model_cleaning.sh
get_ref_model.sh
prepare_eval_data.sh
read_results.ipynb
README.md
requirements.txt
run_eval.sh
self_evolving_cleaning.sh

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/UCSC-REAL/TokenCleaning

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd TokenCleaning

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

The tabular results can be printed via the `read_results.ipynb` jupyter notebook.

이 명령어를 터미널에 그대로 입력해 실행하세요.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning

<a href='https://github.com/JlPang863/LLM_token_selection'><img src='https://img.shields.io/badge/Project-Page-Green'></a>
<a href='https://arxiv.org/abs/2502.01968'><img src='https://img.shields.io/badge/Paper-PDF-orange'></a>

[Jinlong Pang](https://jlpang863.github.io/), [Na Di](https://www.linkedin.com/in/na-di/), [Zhaowei Zhu](https://users.soe.ucsc.edu/~zhaoweizhu/), [Jiaheng Wei](https://sites.google.com/ucsc.edu/jiahengwei), [Hao Cheng](https://haochenglouis.github.io/), [Chen Qian](https://users.soe.ucsc.edu/~qian/), [Yang Liu](http://www.yliuu.com/).

University of California, Santa Cruz

-----
### Brief Introduction
This project investigates token quality from a noisy-label perspective and propose a generic token cleaning pipeline for SFT tasks. Our method filters out uninformative tokens while preserving those carrying key task-specific information. Specifically, we first evaluate token quality by examining the influence of model updates on each token, then apply a threshold-based separation. The token influence can be measured in a single pass with a fixed reference model or iteratively with self-evolving reference models.

![The Overview of Token Cleaning Pipelines](https://raw.githubusercontent.com/UCSC-REAL/TokenCleaning/main/figures/token_cleaning_overivew.jpg)

- **Fixed-Model Cleaning** This pipeline applies a one-shot cleaning process to the entire dataset.

- **Self-Evolving Cleanning**  This pipeline follows an iterative approach.

## 🎉🎉 News 
- [x] [2025.05.01] 🚀🚀 Accepted by ICML 2025.
- [x] [2025.04.01] 🚀🚀 Code Release

## Environment Setup
To run training, evaluation, or inference for finetuned models, you need to install the required packages by running the following command (after installing pytorch):
```
pip install -r requirements.txt
```

## Dataset Preparation

The data pool (50k samples) is constructed based on a new powerful data curation pipeline [**DS2**](https://github.com/UCSC-REAL/DS2), which involves selecting data samples using quality rating scores generated by LLMs. For convenience, the 50k used samples can be accessed from Huggingface via the [link](https://huggingface.co/datasets/jlpang888/DS2_50k).

Our selected evaluation and training data are listed below.

| **Category**         | **Dataset**                                  |
|----------------------|----------------------------------------------|
| **Evaluation Data**   | MMLU, TruthfulQA, TydiQA, HellaSwag, BoolQ, ARC-C, LoqiQA|
| **Training Data**     | Flan v2, OASST1, WizardLM, Dolly, Stanford Alpaca |

## 🚀🚀 Get Started

Note that our cleaning pipelines consists of **Fixed-Model Cleaning** and **Self-Evolving Cleaning**. One can run the code by

```bash
# Fixed-model cleaning
bash get_ref_model.sh
bash fixed_model_cleaning.sh

# Self-evolving cleaning
bash self_evolving_cleaning.sh
```
The implementations of our baselines can be found in the `baselines` directory, including full, random and rho baselines.

## Model Evaluation
The task performances are evaluated on the [lm-eval-hareness](https://github.com/EleutherAI/lm-evaluation-harness) repository.
For convenience, one can do evaluation by

```bash 
bash run_eval.sh
```
Note that `lm-eval-harness` repo does not contain `TydiQA` task. Here, we follows the original [Tydiqa](https://github.com/google-research-datasets/tydiqa) code repo to conduct evaluation. The TydiQA dataset can be downloaded via `prepare_eval_data.sh`.

## Results Presentations

The tabular results can be printed via the `read_results.ipynb` jupyter notebook.

## Citation
If you used this repository, please cite our work:
```
@article{pang2025token,
  title={Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning},
  author={Pang, Jinlong and Di, Na and Zhu, Zhaowei and Wei, Jiaheng and Cheng, Hao and Qian, Chen and Liu, Yang},
  journal={arXiv preprint arXiv:2502.01968},
  year={2025}
}

```