LongVT

(★ 260)

[CVPR 2026] LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

파일 탐색기

최종 버전 다운로드 (.zip)

- capture.gif
- qr_code.jpg
- teaser.png
- lmms_lab_recipe.py
- tool_reward.py
- vl_agent.py
- video_tools.py
- visual_toolbox.py
- - clip_caption.py
  - detect_segment.py
  - imcott_generate.py
  - merge_caption.py
  - merge_event.py
  - qa_filter_text.py
  - qa_filter_vl.py
  - qa_generate.py
  - text_summary.py
- - run_clip_caption.sh
  - run_detect_segment.sh
  - run_imcott_generate.sh
  - run_qa_filter_text.sh
  - run_qa_filter_vl.sh
  - run_qa_generate.sh
  - run_text_summary.sh
- - __init__.py
  - base_server.py
  - mapping.py
  - openai.py
- - merge_json.py
  - split_json_by_team.py
- constant.py
- Makefile
- pyproject.toml
- utils.py
- - run_eval.sh
  - run_single_inference.sh
  - single_inference.py
  - tool_call_qwen2_5_vl.jinja
- - longvt_7b_rft.yaml
  - run_longvt_7b_rft.sh
- - longvt_7b_sft.yaml
  - run_longvt_7b_sft.sh
- - launch.sh
  - longvt_7b_rl_train.sh
  - mcp_client.py
  - mcp_server.py
  - process_time_r1.py
  - qwen2_5_vl_chat_template_w_tool.jinja
- - aime24_nofigures_agg8.yaml
  - aime25_nofigures_agg8.yaml
  - utils.py
- - charades_reasoning.yaml
  - charades_tool.yaml
  - utils.py
- - _default_template_yaml
  - charxiv.yaml
  - charxiv_val_descriptive.yaml
  - charxiv_val_reasoning.yaml
  - constant.py
  - utils.py
- - _default_template_yaml
  - _generate_config.py
  - dynamath.yaml
  - utils.py
- - gpqa_thinking.yaml
  - utils.py
- - logicvista_thinking.yaml
  - utils.py
- - longvideobench_no_visual.yaml
  - longvideobench_random_choice.yaml
  - longvideobench_val_v.yaml
  - longvideobench_val_v_tool.yaml
  - utils.py
- - longvt_no_visual.yaml
  - longvt_non_think.yaml
  - longvt_reasoning.yaml
  - longvt_tool.yaml
  - utils.py
- - lvbench_no_visual.yaml
  - lvbench_random_choice.yaml
  - lvbench_reasoning.yaml
  - lvbench_tool.yaml
  - utils.py
- - mathverse_evals.py
  - mathverse_testmini.yaml
  - utils.py
- - _default_template_yaml
  - eval_utils.py
  - mathvision_reason_test.yaml
  - mathvision_reason_testmini.yaml
  - utils.py
- - mathvista_evals.py
  - mathvista_testmini_cot.yaml
  - utils.py
- - _default_template_yaml
  - _generate_config.py
  - mmlu_thinking.yaml
  - mmlu_thinking_abstract_algebra.yaml
  - mmlu_thinking_anatomy.yaml
  - mmlu_thinking_astronomy.yaml
  - mmlu_thinking_business_ethics.yaml
  - mmlu_thinking_clinical_knowledge.yaml
  - mmlu_thinking_college_biology.yaml
  - mmlu_thinking_college_chemistry.yaml
  - mmlu_thinking_college_computer_science.yaml
  - mmlu_thinking_college_mathematics.yaml
  - mmlu_thinking_college_medicine.yaml
  - mmlu_thinking_college_physics.yaml
  - mmlu_thinking_computer_security.yaml
  - mmlu_thinking_conceptual_physics.yaml
  - mmlu_thinking_econometrics.yaml
  - mmlu_thinking_electrical_engineering.yaml
  - mmlu_thinking_elementary_mathematics.yaml
  - mmlu_thinking_formal_logic.yaml
  - mmlu_thinking_global_facts.yaml
  - mmlu_thinking_high_school_biology.yaml
  - mmlu_thinking_high_school_chemistry.yaml
  - mmlu_thinking_high_school_computer_science.yaml
  - mmlu_thinking_high_school_european_history.yaml
  - mmlu_thinking_high_school_geography.yaml
  - mmlu_thinking_high_school_government_and_politics.yaml
  - mmlu_thinking_high_school_macroeconomics.yaml
  - mmlu_thinking_high_school_mathematics.yaml
  - mmlu_thinking_high_school_microeconomics.yaml
  - mmlu_thinking_high_school_physics.yaml
  - mmlu_thinking_high_school_psychology.yaml
  - mmlu_thinking_high_school_statistics.yaml
  - mmlu_thinking_high_school_us_history.yaml
  - mmlu_thinking_high_school_world_history.yaml
  - mmlu_thinking_human_aging.yaml
  - mmlu_thinking_human_sexuality.yaml
  - mmlu_thinking_international_law.yaml
  - mmlu_thinking_jurisprudence.yaml
  - mmlu_thinking_logical_fallacies.yaml
  - mmlu_thinking_machine_learning.yaml
  - mmlu_thinking_management.yaml
  - mmlu_thinking_marketing.yaml
  - mmlu_thinking_medical_genetics.yaml
  - mmlu_thinking_miscellaneous.yaml
  - mmlu_thinking_moral_disputes.yaml
  - mmlu_thinking_moral_scenarios.yaml
  - mmlu_thinking_nutrition.yaml
  - mmlu_thinking_philosophy.yaml
  - mmlu_thinking_prehistory.yaml
  - mmlu_thinking_professional_accounting.yaml
  - mmlu_thinking_professional_law.yaml
  - mmlu_thinking_professional_medicine.yaml
  - mmlu_thinking_professional_psychology.yaml
  - mmlu_thinking_public_relations.yaml
  - mmlu_thinking_security_studies.yaml
  - mmlu_thinking_sociology.yaml
  - mmlu_thinking_us_foreign_policy.yaml
  - mmlu_thinking_virology.yaml
  - mmlu_thinking_world_religions.yaml
  - utils.py
- - _default_template_yaml
  - _mmlu_pro.yaml
  - mmlu_pro_biology.yaml
  - mmlu_pro_business.yaml
  - mmlu_pro_chemistry.yaml
  - mmlu_pro_computer_science.yaml
  - mmlu_pro_economics.yaml
  - mmlu_pro_engineering.yaml
  - mmlu_pro_health.yaml
  - mmlu_pro_history.yaml
  - mmlu_pro_law.yaml
  - mmlu_pro_math.yaml
  - mmlu_pro_other.yaml
  - mmlu_pro_philosophy.yaml
  - mmlu_pro_physics.yaml
  - mmlu_pro_psychology.yaml
  - README.md
  - utils.py
- - _default_template_yaml
  - arial.ttf
  - mmmu_val_reasoning.yaml
  - utils.py
- - _default_template_yaml
  - mmmu_pro_cot.yaml
  - mmmu_pro_standard_cot.yaml
  - mmmu_pro_vision_cot.yaml
  - utils.py
- - mmvu_thinking.yaml
  - utils.py
- - math_500.yaml
  - utils.py
- - _default_template_yaml
  - utils.py
  - videomme.yaml
  - videomme_long.yaml
  - videomme_no_visual.yaml
  - videomme_tool.yaml
  - videomme_w_subtitle.yaml
  - videomme_w_subtitle_long.yaml
  - videomme_w_subtitle_tool.yaml
- - _default_template_yaml
  - adaptation.yaml
  - adaptation_tool.yaml
  - comprehension.yaml
  - comprehension_tool.yaml
  - perception.yaml
  - perception_tool.yaml
  - utils.py
  - video_mmmu.yaml
  - video_mmmu_tool.yaml
- - utils.py
  - wemath_testmini_thinking.yaml
- - create_dataset.py
  - README.md
  - reward_function.py
  - train_grpo.sh
  - train_sft.sh
- - dapo_ray_trainer.py
  - main_dapo.py
  - prepare_dapo_data.sh
  - README.md
  - run_dapo_early_qwen2.5_32b.sh
  - run_dapo_qwen2.5_32b.sh
  - run_dapo_wo_ds_qwen2.5_32b.sh
  - runtime_env.yaml
  - test_dapo_7b.sh
  - test_dapo_7b_math.sh
  - test_dapo_7b_math_lora.sh
  - test_dapo_7b_math_megatron.sh
  - test_dapo_dspk_671b_megatron.sh
  - test_dapo_qwen3_30b_math.sh
  - test_dapo_qwen3_30b_math_single_node.sh
- - 32b_clip_cov.sh
  - 32b_kl_cov.sh
  - 32b_kl_cov_mininbsz.sh
  - 7b_clip_cov.sh
  - 7b_kl_cov.sh
  - entropy_ray_trainer.py
  - main_entropy.py
  - README.md
  - reward.py
- - README.md
  - reward_function.py
  - run_genrm_remote.sh
- - __init__.py
  - chat_model.py
  - react_agent_loop.py
  - test_react_agent_loop.py
- - rl_dataset.py
- - dapo_7b_math_fsdp2_4_12.sh
  - dapo_7b_math_fsdp2_colocate.sh
  - dapo_7b_math_megatron_4_12.sh
  - dapo_7b_math_megatron_colocate.sh
  - fsdp_workers.py
  - grpo_0.6b_gsm8k_fsdp2_2_6.sh
  - grpo_3b_gsm8k_fsdp2_2_6.sh
  - main_ppo.py
  - megatron_workers.py
  - ray_trainer.py
  - README.md
  - vllm_sharding_manager.py
- - __init__.py
  - main_prime.py
  - prime_core_algos.py
  - prime_dp_rm.py
  - prime_fsdp_workers.py
  - prime_ray_trainer.py
  - run_prime_qwen.sh
  - run_prime_qwen_code.sh
- - __init__.py
  - data_process.py
  - main_eval.py
  - README.md
  - reward_score.py
  - run_r1_distill_qwen.sh
- - retool.py
  - retool_multi_turn_sft_preprocess.py
  - retool_sft_preprocess.py
  - run_qwen2-32b_sft.sh
  - run_qwen2.5_32b_sp8.sh
  - run_qwen2.5_7b_sp4.sh
  - run_qwen3_4b_sp4.sh
  - sandbox_fusion_tool_config.yaml
- - core_algos.py
  - dp_actor.py
  - fsdp_workers.py
  - main_spin.py
  - README.md
  - run_spin.sh
  - spin_trainer.py
- - __init__.py
  - dp_actor.py
  - main_sppo.py
  - README.md
  - run_qwen2.5-7b_rm.sh
  - sppo_ray_trainer.py
  - sppo_worker.py
- README.md
- __init__.py
- converter_hf_to_mcore.py
- diagnose.py
- generate_trainer_config.sh
- init_random_model.py
- install_vllm_sglang_mcore.sh
- legacy_model_merger.py
- print_cfg.py
- rollout_viewer.py
- - __init__.py
  - test_gsm8k_interaction.py
  - test_interaction_registry.py
- - test_transformer.py
  - test_transformers_ulysses.py
- - __init__.py
  - test_auto_padding_on_cpu.py
  - test_colocated_workers.py
  - test_colocated_workers_fused.py
  - test_data_transfer.py
  - test_decorator_on_cpu.py
  - test_driverfunc_to_worker.py
  - test_fused_workers_on_cpu.py
  - test_high_level_scheduling_api.py
  - test_ray_collectives.py
  - test_ray_local_envs_on_cpu.py
  - test_ray_utils_on_cpu.py
  - test_rvdz.py
  - test_worker_group_basics.py
  - test_worker_group_torch.py
- - README.md
  - run_all.sh
  - test_fsdp_ckpt.py
  - test_tensor_dict.py
- - __init__.py
  - check_custom_rwd_fn.py
  - check_results.py
  - README.md
  - run_dapo.sh
  - run_genrm_remote.sh
  - run_geo3k_fsdp_sgl_multiturn_w_tool.sh
  - run_grpo_lora_with_merge.sh
  - run_gsm8k_fsdp_sgl_multiturn_sf_tool.sh
  - run_gsm8k_fsdp_sgl_multiturn_w_tool.sh
  - run_one_step_off_policy.sh
  - run_ppo_trainer_megatron.sh
  - run_prime.sh
  - run_r1_distill_qwen_aime24_eval.sh
  - run_spin.sh
  - run_sppo.sh
  - run_test.sh
- - run_qwen2_5_05b_dapo.sh
  - run_qwen2_5_05b_grpo.sh
  - run_qwen2_5_05b_sft_peft_sp2.sh
  - run_qwen2_5_vl_3b_npu.sh
- - check_api_docs.py
  - check_device_api_usage.py
  - check_docs_time_info.py
  - check_docstrings.py
  - check_license.py
  - check_pr_description.py
  - check_pr_title.py
  - test_config_docs.py
  - test_import.py
  - type_coverage_check.py
  - validate_imported_docs.py
  - validate_structure.py
- - README.md
  - test_memory_buffers.py
- - test_base_tool_on_cpu.py
- - __init__.py
- - _test_module.py
  - test_activation_offload.py
  - test_config_on_cpu.py
  - test_flops_counter.py
  - test_fs_on_cpu.py
  - test_import_utils_on_cpu.py
  - test_linear_cross_entropy.py
  - test_linear_cross_entropy_tp.py
  - test_model_on_cpu.py
  - test_nvtx_profile.py
  - test_rollout_trace_on_cpu.py
  - test_seqlen_balancing.py
  - test_temp_env_on_cpu.py
  - test_timeout_decorator_cpu.py
  - test_torch_functional.py
- __init__.py
- kill_github_tests.sh
- README.md
- test_base_config_on_cpu.py
- test_protocol_on_cpu.py
- - __init__.py
- - __init__.py
  - base.py
  - gsm8k_interaction.py
- - __init__.py
  - __main__.py
  - base_model_merger.py
  - fsdp_model_merger.py
  - megatron_model_merger.py
- - __init__.py
  - README.md
  - registry.py
  - weight_loader_registry.py
- - __init__.py
- - __init__.py
- - __init__.py
  - base_tool.py
  - geo3k_tool.py
  - gsm8k_tool.py
  - mcp_base_tool.py
  - mcp_search_tool.py
  - sandbox_fusion_tools.py
  - schemas.py
  - search_tool.py
- - __init__.py
  - constants_ppo.py
  - fsdp_sft_trainer.py
  - main_eval.py
  - main_generation.py
  - main_ppo.py
  - runtime_env.yaml
- - __init__.py
  - activation_offload.py
  - config.py
  - device.py
  - distributed.py
  - flops_counter.py
  - fs.py
  - fsdp_utils.py
  - hdfs_io.py
  - import_utils.py
  - logging_utils.py
  - megatron_utils.py
  - memory_buffer.py
  - model.py
  - net_utils.py
  - py_functional.py
  - ray_utils.py
  - rollout_trace.py
  - seqlen_balancing.py
  - tokenizer.py
  - torch_dtypes.py
  - torch_functional.py
  - tracking.py
  - ulysses.py
  - vllm_utils.py
- - version
- - __init__.py
  - fsdp_workers.py
  - megatron_workers.py
- __init__.py
- base_config.py
- hermes_tool_parser.py
- protocol.py
- py.typed
.gitignore
.pre-commit-config.yaml
LICENSE
pyproject.toml
README.md
requirements-npu.txt
requirements.txt
requirements_reproduce.txt
requirements_sglang.txt
setup.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/EvolvingLMMs-Lab/LongVT

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd LongVT

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. 공식 설치 스크립트

쉬움 추천

사전 준비물

Python 3 pip 명령어를 쓰려면 Python이 필요합니다.

pip install --no-deps -e .

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

pip install huggingface_hub

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

✅ 설치 후 새 터미널을 열고, 프로그램의 버전 확인 명령(예: --version)으로 정상 설치됐는지 확인하세요.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

3. Python

쉬움

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install --no-deps -e .

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

uv pip install -e .

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

pip install -e .

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

uv pip install -e ".[dev]"

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

pip install huggingface_hub

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

4. Make

보통

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Make Linux/macOS는 보통 기본 설치되어 있습니다. Windows는 별도 설치(예: MSYS2, WSL)가 필요합니다.

cd data

이 프로젝트의 관련 파일이 하위 폴더 안에 있어서, 먼저 그 폴더로 이동합니다.

make

생성된 빌드 설정을 바탕으로 실제 컴파일을 진행해 실행 파일을 만듭니다.

✅ 에러 없이 끝나면 성공입니다. 생성된 실행 파일을 직접 실행해보세요.

LongVT

파일 탐색기

# 설치 가이드

2. 공식 설치 스크립트

3. Python

4. Make

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

LongVT

파일 탐색기

# 설치 가이드

2. 공식 설치 스크립트

3. Python

4. Make

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README