llm-hessian
Using PyTorch autograd to compute Hessian of Perplexity for Large Language Models
파일 탐색기
- hessian_diag_q_proj_vhp_samples_1.pt
- hessian_diag_q_proj_vhp_samples_10.pt
- hessian_diag_q_proj_vhp_samples_100.pt
- hessian_diag_q_proj_vhp_samples_1000.pt
- hessian_diag_q_proj_vhp_samples_3000.pt
- hessian_diag_q_proj_vhp_samples_5000.pt
- partial_relative_l2_loss.pt
- relative_l2_diffs.pt
- b_1.pdf
- b_10.pdf
- b_100.pdf
- b_101.pdf
- b_102.pdf
- b_103.pdf
- b_104.pdf
- b_105.pdf
- b_106.pdf
- b_107.pdf
- b_108.pdf
- b_109.pdf
- b_11.pdf
- b_110.pdf
- b_111.pdf
- b_112.pdf
- b_113.pdf
- b_114.pdf
- b_115.pdf
- b_116.pdf
- b_117.pdf
- b_118.pdf
- b_119.pdf
- b_12.pdf
- b_120.pdf
- b_121.pdf
- b_122.pdf
- b_123.pdf
- b_124.pdf
- b_125.pdf
- b_126.pdf
- b_127.pdf
- b_128.pdf
- b_129.pdf
- b_13.pdf
- b_130.pdf
- b_131.pdf
- b_132.pdf
- b_133.pdf
- b_134.pdf
- b_135.pdf
- b_136.pdf
- b_137.pdf
- b_138.pdf
- b_139.pdf
- b_14.pdf
- b_140.pdf
- b_15.pdf
- b_16.pdf
- b_17.pdf
- b_18.pdf
- b_19.pdf
- b_2.pdf
- b_20.pdf
- b_21.pdf
- b_22.pdf
- b_23.pdf
- b_24.pdf
- b_25.pdf
- b_26.pdf
- b_27.pdf
- b_28.pdf
- b_29.pdf
- b_3.pdf
- b_30.pdf
- b_31.pdf
- b_32.pdf
- b_33.pdf
- b_34.pdf
- b_35.pdf
- b_36.pdf
- b_37.pdf
- b_38.pdf
- b_39.pdf
- b_4.pdf
- b_40.pdf
- b_41.pdf
- b_42.pdf
- b_43.pdf
- b_44.pdf
- b_45.pdf
- b_46.pdf
- b_47.pdf
- b_48.pdf
- b_49.pdf
- b_5.pdf
- b_50.pdf
- b_51.pdf
- b_52.pdf
- b_53.pdf
- b_54.pdf
- b_55.pdf
- b_56.pdf
- b_57.pdf
- b_58.pdf
- b_59.pdf
- b_6.pdf
- b_60.pdf
- b_61.pdf
- b_62.pdf
- b_63.pdf
- b_64.pdf
- b_65.pdf
- b_66.pdf
- b_67.pdf
- b_68.pdf
- b_69.pdf
- b_7.pdf
- b_70.pdf
- b_71.pdf
- b_72.pdf
- b_73.pdf
- b_74.pdf
- b_75.pdf
- b_76.pdf
- b_77.pdf
- b_78.pdf
- b_79.pdf
- b_8.pdf
- b_80.pdf
- b_81.pdf
- b_82.pdf
- b_83.pdf
- b_84.pdf
- b_85.pdf
- b_86.pdf
- b_87.pdf
- b_88.pdf
- b_89.pdf
- b_9.pdf
- b_90.pdf
- b_91.pdf
- b_92.pdf
- b_93.pdf
- b_94.pdf
- b_95.pdf
- b_96.pdf
- b_97.pdf
- b_98.pdf
- b_99.pdf
- b_1.pt
- b_10.pt
- b_100.pt
- b_101.pt
- b_102.pt
- b_103.pt
- b_104.pt
- b_105.pt
- b_106.pt
- b_107.pt
- b_108.pt
- b_109.pt
- b_11.pt
- b_110.pt
- b_111.pt
- b_112.pt
- b_113.pt
- b_114.pt
- b_115.pt
- b_116.pt
- b_117.pt
- b_118.pt
- b_119.pt
- b_12.pt
- b_120.pt
- b_121.pt
- b_122.pt
- b_123.pt
- b_124.pt
- b_125.pt
- b_126.pt
- b_127.pt
- b_128.pt
- b_129.pt
- b_13.pt
- b_130.pt
- b_131.pt
- b_132.pt
- b_133.pt
- b_134.pt
- b_135.pt
- b_136.pt
- b_137.pt
- b_138.pt
- b_139.pt
- b_14.pt
- b_140.pt
- b_15.pt
- b_16.pt
- b_17.pt
- b_18.pt
- b_19.pt
- b_2.pt
- b_20.pt
- b_21.pt
- b_22.pt
- b_23.pt
- b_24.pt
- b_25.pt
- b_26.pt
- b_27.pt
- b_28.pt
- b_29.pt
- b_3.pt
- b_30.pt
- b_31.pt
- b_32.pt
- b_33.pt
- b_34.pt
- b_35.pt
- b_36.pt
- b_37.pt
- b_38.pt
- b_39.pt
- b_4.pt
- b_40.pt
- b_41.pt
- b_42.pt
- b_43.pt
- b_44.pt
- b_45.pt
- b_46.pt
- b_47.pt
- b_48.pt
- b_49.pt
- b_5.pt
- b_50.pt
- b_51.pt
- b_52.pt
- b_53.pt
- b_54.pt
- b_55.pt
- b_56.pt
- b_57.pt
- b_58.pt
- b_59.pt
- b_6.pt
- b_60.pt
- b_61.pt
- b_62.pt
- b_63.pt
- b_64.pt
- b_65.pt
- b_66.pt
- b_67.pt
- b_68.pt
- b_69.pt
- b_7.pt
- b_70.pt
- b_71.pt
- b_72.pt
- b_73.pt
- b_74.pt
- b_75.pt
- b_76.pt
- b_77.pt
- b_78.pt
- b_79.pt
- b_8.pt
- b_80.pt
- b_81.pt
- b_82.pt
- b_83.pt
- b_84.pt
- b_85.pt
- b_86.pt
- b_87.pt
- b_88.pt
- b_89.pt
- b_9.pt
- b_90.pt
- b_91.pt
- b_92.pt
- b_93.pt
- b_94.pt
- b_95.pt
- b_96.pt
- b_97.pt
- b_98.pt
- b_99.pt
- hessian_self_attn.k_proj_block0_t512_b60_seed0.pdf
- hessian_self_attn.k_proj_block0_t512_b60_seed0.pt
- hessian_self_attn.o_proj_block0_t512_b60_seed0.pdf
- hessian_self_attn.o_proj_block0_t512_b60_seed0.pt
- hessian_self_attn.q_proj_block0_t512_b60_seed0.pdf
- hessian_self_attn.q_proj_block0_t512_b60_seed0.pt
- hessian_self_attn.v_proj_block0_t512_b60_seed0.pdf
- hessian_self_attn.v_proj_block0_t512_b60_seed0.pt
- hessian_all_layers_all_blocks_t_25.pt
- hessian_all_layers_all_blocks_t_25_small_bs.pt
- hessian_all_layers_first_block_t_300.pt
- hessian_q_proj_all_blocks_t_150.pt
- hessian_q_proj_b60_t768.pt
- hessian_q_proj_b_60_t_768.pt
- hessian_q_proj_t_25.pt
- hessian_q_proj_t_768.pt
- wikitext2_dataset_test_cache_seqlen2048_b140_seed0.pt
- wikitext2_dataset_train_cache_seqlen2048_b140_seed0.pt
- hessian_all_layers_all_blocks_t_25.pdf
- hessian_all_layers_all_blocks_t_25.png
- hessian_all_layers_first_block_t_300.pdf
- hessian_all_layers_first_block_t_300.png
- hessian_different_bs.pdf
- hessian_q_proj_all_blocks_t_150.pdf
- hessian_q_proj_all_blocks_t_150.png
- hessian_q_proj_diag_5000.pdf
- hessian_q_proj_diag_5000.png
- hessian_q_proj_t_25.pdf
- hessian_q_proj_t_25.png
- hessian_q_proj_t_768.pdf
- hessian_q_proj_t_768.png
- losses_vs_bs.pdf
- losses_vs_bs.png
- losses_vs_k.pdf
- losses_vs_k.png
- q_proj_hessian_diag_different_k.pdf
- all_layers_all_blocks_diags.py
- all_layers_from_block_separately.py
- data.py
- finite_diffs.py
- hessian_diag_single_layer.py
- hessian_diag_single_layer_fisher.py
- several_layers_several_blocks.py
- single_layer_several_blocks.py
- single_layer_single_block.py
- utils.py
- .gitignore
- LICENSE
- PYTHON_INSTALL.md
- README.md
- requirements.txt
# CDN으로 사용하기
jsDelivrjsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.
링크
예시
명령어 용어집
이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.
pip install
설명 보기 ▼
pip install
Install Python packages.
pip install {{package1 package2 ...}}
Install one or more packages:
pip install {{package1 package2 ...}} {{[-U|--upgrade]}}
Upgrade all specified packages to the latest version, installing any that are not already present:
pip install {{package}}=={{version}}
Install a specific version of a package:
python
설명 보기 ▼
python
Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.
python
REPL(대화형 셸) 시작:
python {{path/to/file.py}}
특정 Python 파일 실행:
python -i {{path/to/file.py}}
특정 Python 파일 실행 후 REPL 시작:
// repository documentation
Was this content helpful?
(0 ratings)
