verl-internvl

KO | EN

gitlite — search

#react #typescript #go #rust #deep-learning #linux #docker #macos #nodejs #vue #pytorch #ios #openai #open-source

(★ 53)

No description available.

File Explorer

Download Latest Version (.zip)

- - check-pr-title.yml
  - checkpoint_converter.yml
  - cpu_unit_tests.yml
  - doc.yml
  - e2e_ascend.yml
  - e2e_dapo.yml
  - e2e_eval_aime24.yml
  - e2e_genrm_remote.yml
  - e2e_ppo_trainer.yml
  - e2e_ppo_trainer_megatron.yml
  - e2e_sft.yml
  - e2e_spin.yml
  - e2e_sppo.yml
  - gpu_unit_tests.yml
  - model.yml
  - pre-commit-full.yml
  - pre-commit.yml
  - sanity.yml
  - scorecard.yml
  - secrets_scan.yml
  - sgl.yml
  - type-coverage-check.yml
  - vllm.yml
- CODEOWNERS
- dependabot.yml
- PULL_REQUEST_TEMPLATE.md
- settings.json
- Apptainerfile.rocm
- Dockerfile.awsefa
- Dockerfile.ngc.vllm
- Dockerfile.ngc.vllm0.8
- Dockerfile.ngc.vllm0.8.sagemaker
- Dockerfile.rocm
- Dockerfile.sglang
- Dockerfile.vemlp.vllm.te
- Dockerfile.vllm.sglang.megatron
- Dockerfile.vllm.sglang.megatron.deepseek
- - logo.png
- - checkpoint.rst
  - dpo_extension.rst
  - fsdp_extension.rst
  - megatron_extension.rst
  - placement.rst
  - ppo_lora.rst
  - rope.rst
- - baseline.md
  - dapo.md
  - entropy.md
  - grpo.md
  - opo.md
  - ppo.md
  - spin.md
  - sppo.md
- - amd_build_dockerfile_page.rst
  - amd_vllm_page.rst
- - data.rst
  - single_controller.rst
  - trainer.rst
  - utils.rst
- - ascend_quick_start.rst
- - config.rst
  - gsm8k_example.rst
  - multi_modal_example.rst
  - ppo_code_architecture.rst
  - sandbox_fusion_example.rst
- - faq.rst
- - device_tuning.rst
  - dpsk.md
  - nsight_profiling.md
  - perf_tuning.rst
- - prepare_data.rst
  - reward_function.rst
- - interaction_system.rst
  - multiturn.rst
  - sandbox_fusion.rst
  - search_tool_example.rst
- - install.rst
  - more_resources.rst
  - multinode.rst
  - quickstart.rst
  - ray_debug_tutorial.rst
- - fsdp_workers.rst
  - megatron_workers.rst
  - ray_trainer.rst
  - sglang_worker.rst
- conf.py
- hybrid_flow.rst
- index.rst
- Makefile
- README.md
- README_vllm0.7.md
- README_vllm0.8.md
- requirements-docs.txt
- single_controller.rst
- - aime2024_multiturn_w_tool.py
  - dapo_multiturn_w_tool.py
  - full_hh_rlhf.py
  - geo3k.py
  - geo3k_multiturn_w_tool.py
  - gsm8k.py
  - gsm8k_multiturn_w_interaction.py
  - gsm8k_multiturn_w_tool.py
  - hellaswag.py
  - math_dataset.py
  - multiturn.py
  - preprocess_search_r1_dataset.py
- - run_deepseek7b_mutli_node.sh
  - run_deepseek_v2_lite_math.sh
- - README.md
  - run_deepseek671b_math_megatron.sh
  - run_deepseek7b_llm.sh
  - run_deepseek7b_llm_math.sh
  - run_deepseek7b_llm_math_megatron.sh
  - run_deepseek7b_llm_seq_balance.sh
  - run_minicpmo2_6.sh
  - run_moonlight16b_math_megatron.sh
  - run_qwen2-7b.sh
  - run_qwen2-7b_math.sh
  - run_qwen2-7b_math_megatron.sh
  - run_qwen2-7b_seq_balance.sh
  - run_qwen2-7b_seq_balance_math_megatron.sh
  - run_qwen2-7b_sgl_megatron.sh
  - run_qwen2_5-3b_gsm8k_grpo_lora.sh
  - run_qwen2_5-7b_math_megatron_diff_tp.sh
  - run_qwen2_5_32b_grpo_npu.sh
  - run_qwen2_5_7b_grpo_npu.sh
  - run_qwen2_5_vl-7b-megatron.sh
  - run_qwen2_5_vl-7b.sh
  - run_qwen2_5_vl-7b_lora.sh
  - run_qwen2_5_vl-7b_seq_balance.sh
  - run_qwen2_5_vl_32b_npu.sh
  - run_qwen2_5_vl_3b_npu.sh
  - run_qwen2_5_vl_7b_npu.sh
  - run_qwen3-236b_megatron.sh
  - run_qwen3-8b.sh
  - run_qwen3moe-30b_megatron.sh
- - README.md
  - run_deepseek7b_llm.sh
  - run_deepseek7b_llm_modelscope.sh
  - run_deepseek7b_llm_pfppo.sh
  - run_deepseek7b_llm_sandbox_fusion.sh
  - run_deepseek7b_llm_sp2.sh
  - run_deepseek_full_hh_rlhf.sh
  - run_deepseek_math_gsm8k_megatron.sh
  - run_deepseek_math_gsm8k_megatron_nsys.sh
  - run_gemma.sh
  - run_moonlight16b_a3b_gsm8k_megatron.sh
  - run_qwen1.5_moe_a2.7b-gsm8k_megatron.sh
  - run_qwen2-7b_math_gsm8k_megatron.sh
  - run_qwen2-7b_rm.sh
  - run_qwen2-7b_rm_seq_balance.sh
  - run_qwen2-7b_rm_seq_balance_fused_kernels.sh
  - run_qwen2-7b_rm_seq_balance_nsys.sh
  - run_qwen2-7b_seq_balance.sh
  - run_qwen2-7b_sglang_seq_balance.sh
  - run_qwen2.5-32b.sh
- - tutorial.ipynb
- - run_qwen2-7b_math_rf.sh
  - run_qwen2-7b_math_rf_baseline.sh
- - run_qwen2.5-3b_seq_balance.sh
  - run_qwen2.5-7b_seq_balance.sh
- - run_qwen2-7b.sh
- - README.md
  - run_qwen2.5-0.5b_gsm8k_multiturn_w_interaction.sh
  - run_qwen2.5-3b_gsm8k_multiturn.sh
  - run_qwen2.5-3b_gsm8k_multiturn_4xgpu.sh
  - run_qwen2.5-3b_megatron_gsm8k_multiturn.sh
  - run_qwen3-4b_gsm8k_multiturn.sh
- - ray_on_slurm.slurm
- - main_ppo_split.py
  - README.md
  - run_deepseek7b_llm.sh
  - split_monkey_patch.py
- - create_dataset.py
  - README.md
  - reward_function.py
  - train_grpo.sh
  - train_sft.sh
- - dapo_ray_trainer.py
  - main_dapo.py
  - prepare_dapo_data.sh
  - README.md
  - run_dapo_early_qwen2.5_32b.sh
  - run_dapo_qwen2.5_32b.sh
  - run_dapo_wo_ds_qwen2.5_32b.sh
  - runtime_env.yaml
  - test_dapo_7b.sh
  - test_dapo_7b_math.sh
  - test_dapo_7b_math_lora.sh
  - test_dapo_7b_math_megatron.sh
  - test_dapo_dspk_671b_megatron.sh
  - test_dapo_qwen3_30b_math.sh
- - 32b_clip_cov.sh
  - 32b_kl_cov.sh
  - 32b_kl_cov_mininbsz.sh
  - 7b_clip_cov.sh
  - 7b_kl_cov.sh
  - entropy_ray_trainer.py
  - main_entropy.py
  - README.md
  - reward.py
- - README.md
  - reward_function.py
  - run_genrm_remote.sh
- - rl_dataset.py
- - __init__.py
  - main_prime.py
  - prime_core_algos.py
  - prime_dp_rm.py
  - prime_fsdp_workers.py
  - prime_ray_trainer.py
  - run_prime_qwen.sh
  - run_prime_qwen_code.sh
- - __init__.py
  - data_process.py
  - main_eval.py
  - README.md
  - reward_score.py
  - run_r1_distill_qwen.sh
- - retool_multi_turn_sft_preprocess.py
  - run_qwen2.5_32b_sp8.sh
  - run_qwen2.5_7b_sp4.sh
  - run_qwen3_4b_sp4.sh
  - run_sft.sh
- - core_algos.py
  - dp_actor.py
  - fsdp_workers.py
  - main_spin.py
  - README.md
  - run_spin.sh
  - spin_trainer.py
- - __init__.py
  - dp_actor.py
  - main_sppo.py
  - README.md
  - run_qwen2.5-7b_rm.sh
  - sppo_ray_trainer.py
  - sppo_worker.py
- README.md
- __init__.py
- converter_hf_to_mcore.py
- diagnose.py
- init_random_model.py
- install_vllm_sglang_mcore.sh
- legacy_model_merger.py
- internvl3_5_14b.sh
- internvl3_5_1b.sh
- internvl3_5_241b_a28b.sh
- internvl3_5_2b.sh
- internvl3_5_30b_a3b.sh
- internvl3_5_38b.sh
- internvl3_5_4b.sh
- internvl3_5_8b.sh
- - __init__.py
  - test_gsm8k_interaction.py
  - test_interaction_registry.py
- - test_transformer.py
  - test_transformers_ulysses.py
- - __init__.py
  - test_auto_padding_on_cpu.py
  - test_colocated_workers.py
  - test_colocated_workers_fused.py
  - test_data_transfer.py
  - test_decorator_on_cpu.py
  - test_driverfunc_to_worker.py
  - test_fused_workers_on_cpu.py
  - test_high_level_scheduling_api.py
  - test_ray_collectives.py
  - test_ray_local_envs_on_cpu.py
  - test_ray_utils_on_cpu.py
  - test_rvdz.py
  - test_worker_group_basics.py
  - test_worker_group_torch.py
- - README.md
  - run_all.sh
  - test_fsdp_ckpt.py
  - test_tensor_dict.py
- - __init__.py
  - check_custom_rwd_fn.py
  - check_results.py
  - README.md
  - run_dapo.sh
  - run_genrm_remote.sh
  - run_geo3k_fsdp_sgl_multiturn_w_tool.sh
  - run_grpo_lora_with_merge.sh
  - run_gsm8k_fsdp_sgl_multiturn_sf_tool.sh
  - run_gsm8k_fsdp_sgl_multiturn_w_tool.sh
  - run_ppo_trainer_megatron.sh
  - run_prime.sh
  - run_r1_distill_qwen_aime24_eval.sh
  - run_spin.sh
  - run_sppo.sh
  - run_test.sh
- - run_qwen2_5_05b_dapo.sh
  - run_qwen2_5_05b_grpo.sh
  - run_qwen2_5_05b_sft_peft_sp2.sh
  - run_qwen2_5_vl_3b_npu.sh
- - check_api_docs.py
  - check_device_api_usage.py
  - check_docs_time_info.py
  - check_license.py
  - check_pr_description.py
  - check_pr_title.py
  - test_config_docs.py
  - test_import.py
  - type_coverage_check.py
  - validate_imported_docs.py
  - validate_structure.py
- - README.md
  - test_memory_buffers.py
- - test_base_tool_on_cpu.py
- - __init__.py
- - _test_module.py
  - test_activation_offload.py
  - test_config_on_cpu.py
  - test_flops_counter.py
  - test_fs_on_cpu.py
  - test_import_utils_on_cpu.py
  - test_linear_cross_entropy.py
  - test_linear_cross_entropy_tp.py
  - test_model_on_cpu.py
  - test_nvtx_profile.py
  - test_seqlen_balancing.py
  - test_timeout_decorator_cpu.py
  - test_torch_functional.py
- __init__.py
- kill_github_tests.sh
- README.md
- test_protocol_on_cpu.py
- - __init__.py
- - __init__.py
  - base.py
  - gsm8k_interaction.py
- - __init__.py
  - __main__.py
  - base_model_merger.py
  - fsdp_model_merger.py
  - megatron_model_merger.py
- - __init__.py
  - README.md
  - registry.py
  - weight_loader_registry.py
- - __init__.py
- - __init__.py
- - __init__.py
  - base_tool.py
  - geo3k_tool.py
  - gsm8k_tool.py
  - mcp_base_tool.py
  - mcp_search_tool.py
  - sandbox_fusion_tools.py
  - schemas.py
  - search_tool.py
- - __init__.py
  - fsdp_sft_trainer.py
  - main_eval.py
  - main_generation.py
  - main_ppo.py
  - runtime_env.yaml
- - __init__.py
  - activation_offload.py
  - config.py
  - device.py
  - distributed.py
  - flops_counter.py
  - fs.py
  - fsdp_utils.py
  - hdfs_io.py
  - import_utils.py
  - logging_utils.py
  - megatron_utils.py
  - memory_buffer.py
  - model.py
  - net_utils.py
  - py_functional.py
  - ray_utils.py
  - seqlen_balancing.py
  - tokenizer.py
  - torch_dtypes.py
  - torch_functional.py
  - tracking.py
  - ulysses.py
  - vllm_utils.py
- - version
- - __init__.py
  - fsdp_workers.py
  - megatron_workers.py
- __init__.py
- protocol.py
.gitignore
.pre-commit-config.yaml
.readthedocs.yaml
CONTRIBUTING.md
LICENSE
Notice.txt
pyproject.toml
README.md
README_verl.md
requirements-npu.txt
requirements.txt
requirements_sglang.txt
setup.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

[\[🔥 InternVL3.5 Report\]](https://huggingface.co/papers/2508.18265)
[\[🗨️ Chat Demo\]](https://chat.intern-ai.org.cn/)

![image/jpg](https://huggingface.co/OpenGVLab/InternVL3_5-241B-A28B/resolve/main/images/performance.jpg)

</div>

This repository open-sources the training code of InternVL3.5 during the online RL stage, which is built upon the [PR](https://github.com/volcengine/verl/pull/2327) in [verl](https://github.com/volcengine/verl). Compared to the original PR, we have corrected the dialogue template for InternVL and updated a monkey patch for InternVL to enable sequence-parallel. For training details, please refer to the provided [scripts](shell).

We use [MMPR-Tiny](https://huggingface.co/datasets/OpenGVLab/MMPR-Tiny) as the training dataset and initialize the model with InternVL3.5 trained after MPO. We also provide a [packaged conda environment](https://huggingface.co/Weiyun1025/InternVL3_5-RL-conda-env/blob/main/verl-internvl.tar.gz) for easy reproduction.

For the original README of verl, please refer to [this file](README_verl.md).

## Experimental Results

Based on this codebase, the InternVL3.5 series across all model scales achieve a significant improvement in reasoning performance.

![image/jpg](https://huggingface.co/OpenGVLab/InternVL3_5-241B-A28B/resolve/main/images/ablation_cascade_rl.jpg)

![image/jpg](https://huggingface.co/OpenGVLab/InternVL3_5-241B-A28B/resolve/main/images/ablation_cascade_rl_table.jpg)

## Quick Start

### Training

We open-source our training data (i.e., [MMPR-Tiny](https://huggingface.co/datasets/OpenGVLab/MMPR-Tiny)) on HuggingFace. To reproduce our training results, you need to download this dataset and move it into this folder.
Additionally, considering that verl requires a validation dataset to be loaded, please prepare this data using this [script](examples/data_preprocess/geo3k.py).

```
├── MMPR-Tiny
│   ├── images
│   └── mmpr_tiny.parquet
├── verl_data
│   └── geo3k
│       └── test.parquet
├── verl
└── README.md
```

We also provide a [packaged conda environment](https://huggingface.co/Weiyun1025/InternVL3_5-RL-conda-env/blob/main/verl-internvl.tar.gz) for easy reproduction. After preparing the dataset and conda environment, you can launch the training using the commond as follows:

```shell
sh shell/internvl3_5_8b.sh
```

### Evaluation

We mainly use [VLMEvalkit](https://github.com/open-compass/VLMEvalKit) to evaluate our models. Please refer to their documentation and our model configs for more details. As an example, you can set the config as follows:

```python
"InternVL3_5-8B-Thinking": partial(
    InternVLChat,
    model_path="/path/to/your/moel",
    version="V2.0",
    cot_prompt_version="r1",
    max_new_tokens=32768,
    do_sample=True,
    use_lmdeploy=True,
),
```

## Citation
If you find this project useful in your research, please consider citing:
```BibTeX
@article{wang2025internvl3_5,
  title={InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency},
  author={Wang, Weiyun and Gao, Zhangwei and Gu, Lixin and Pu, Hengjun and Cui, Long and Wei, Xingguang and Liu, Zhaoyang and Jing, Linglin and Ye, Shenglong and Shao, Jie and others},
  journal={arXiv preprint arXiv:2508.18265},
  year={2025}
}
@article{wang2024mpo,
  title={Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization},
  author={Wang, Weiyun and Chen, Zhe and Wang, Wenhai and Cao, Yue and Liu, Yangzhou and Gao, Zhangwei and Zhu, Jinguo and Zhu, Xizhou and Lu, Lewei and Qiao, Yu and Dai, Jifeng},
  journal={arXiv preprint arXiv:2411.10442},
  year={2024}
}
```

verl-internvl

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

verl-internvl

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README