multiagent-ft - GitLite

KO | EN

gitlite — search

#typescript #go #android #machine-learning #deep-learning #docker #nodejs #macos #java #windows #awesome #kubernetes #ios #mcp

multiagent-ft

(★ 237)

No description available.

파일 탐색기

최종 버전 다운로드 (.zip)

- - finetune.py
  - ft.sh
  - llama2_flash_attn_monkey_patch.py
  - lm_ft.py
  - lm_ft.txt
  - README.md
- eval_math.py
- ft_critic.py
- ft_gen.py
- ft_generator.py
- grader.py
- math_normalize.py
- original_gen.py
README.md
requirements.txt

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/vsubramaniam851/multiagent-ft

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd multiagent-ft

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains
### [Project Page](https://llm-multiagent-ft.github.io/) | [Paper](https://arxiv.org/abs/2501.05707)

[Vighnesh Subramaniam](https://vsubramaniam851.github.io/),
[Yilun Du](https://yilundu.github.io/),
[Joshua B Tenenbaum](https://scholar.google.com/citations?user=rRJ9wTJMUB8C&hl=en),
[Antonio Torralba](https://groups.csail.mit.edu/vision/torralbalab/),
[Shuang Li](https://people.csail.mit.edu/lishuang/),
[Igor Mordatch](https://scholar.google.com/citations?user=Vzr1RukAAAAJ&hl=en)

This is the implementation of our paper "Multiagent Finetuning of Language Models". We design this implementation for the [MATH](https://arxiv.org/pdf/2103.03874) dataset for simplicity. Using other datasets requires little modification of the code.

## Installation and Setup
We include a `requirements.txt` to provide the basic requirements for set-up. To do finetuning with open-source language models, we also include a more details set up in the path `multiagent-ft/lm_ft` with a more detailed list of required packages.

Set your OpenAI API Key using `export OPENAI_API_KEY=your_api_key_here`.

Create a conda/pip environment and install [Pytorch](https://pytorch.org/).
Then run
```
pip install -r requirements.txt
```

### Data
To download the MATH dataset, follow the steps [here](https://github.com/hendrycks/math/).

### Hardware
We ran all experiments either with GPT-3.5 with the OpenAI API or with open-source language models on four 40GB A100s or 8 H100s.

## Creating Finetuning Data
To create finetuning data, change directory to the `multiagent-ft` directory and run
```
python original_gen.py --agents [AGENTS] --rounds [ROUNDS] --model [MODEL] --save_str [SAVE_STR] --summarize
```
where `agents` refers to the number of multiagent debate agents, `rounds` refers to the number of rounds of debate, `model` refers to the model to use debate on, and `save_str` refers to the save log. `summarize` is an argument that makes the model summarize responses from other agents. We also include parameters for `top_p` and `temperature` that can be used with the open-source models.

## Generation Model Finetuning
To create data for finetuning generation agents, run 
```
python ft_generator.py --file_path [FILE_PATH] --save_path [SAVE_PATH] [--gpt] --iteration [ITERATION]
```

This takes in a path for a JSON file and creates the JSON/JSONL files for finetuning. If you include the `--gpt` flag, it will use the OpenAI API for finetuning assuming the goal is to finetune GPT-3.5. You can track which iteration of finetuning you're applying using the `--iteration` flag. You also can pass in the GPT model IDs to finetuning using the `model_ids` flag.

## Critic Model Finetuning
Similar to generator finetuning. Run
```
python ft_critic.py --file_path [FILE_PATH] --save_path [SAVE_PATH] [--gpt] --iteration [ITERATION]
```

## Open-Source Language Model Finetuning
To finetune open-source models, refer to the `lm_ft` directory.

## Running Finetuning Models
To run finetune models, pass either the model ID strings for OpenAI models or the model paths for the finetuned models as arguments when running
```
python ft_gen.py --generators [GENERATORS] --critics [CRITICS] --model [MODEL] --save_str [SAVE_STR]
```

To account for hardware/memory limitations, this will put all open-source models from HuggingFace on the same GPU.

## Evaluation
To evaluate the performance, set the correct path for the JSON file and run
```
python eval_math.py
```