HiNeRV - GitLite

KO | EN

gitlite — search

#javascript #python #llm #go #android #machine-learning #deep-learning #docker #macos #java #awesome #cli #vue #open-source

HiNeRV

(★ 76)

No description available.

File Explorer

Download Latest Version (.zip)

- - bunny-hinerv-a-s_1280x720.txt
  - bunny-hinerv-a-xs_1280x720.txt
  - bunny-hinerv-a-xxs_1280x720.txt
  - bunny-hinerv-b-s_1280x720.txt
  - bunny-hinerv-b-xs_1280x720.txt
  - bunny-hinerv-b-xxs_1280x720.txt
  - bunny-hinerv-s_1280x720.txt
  - bunny-hinerv-xs_1280x720.txt
  - bunny-hinerv-xxs_1280x720.txt
  - mcl-hinerv-l_1920x1080.txt
  - mcl-hinerv-m_1920x1080.txt
  - mcl-hinerv-s_1920x1080.txt
  - mcl-hinerv-xs_1920x1080.txt
  - mcl-hinerv-xxs_1920x1080.txt
  - uvg-hinerv-a-l_1920x1080.txt
  - uvg-hinerv-a-m_1920x1080.txt
  - uvg-hinerv-a-s_1920x1080.txt
  - uvg-hinerv-b-l_1920x1080.txt
  - uvg-hinerv-b-m_1920x1080.txt
  - uvg-hinerv-b-s_1920x1080.txt
  - uvg-hinerv-l_1920x1080.txt
  - uvg-hinerv-m_1920x1080.txt
  - uvg-hinerv-s_1920x1080.txt
  - uvg-hinerv-xl_1920x1080.txt
  - uvg-hinerv-xxl_1920x1080.txt
- - hinerv_1280x720.txt
  - hinerv_1920x1080.txt
  - hinerv_1920x1080_1200e.txt
  - hinerv_1920x1080_150e_no-compress.txt
  - hinerv_1920x1080_37e_no-compress.txt
  - hinerv_1920x1080_480x360.txt
  - hinerv_1920x1080_600e.txt
  - hinerv_1920x1080_75e_no-compress.txt
- __init__.py
- codec_utils.py
- prune_utils.py
- quant_utils.py
- utils.py
- __init__.py
- encoding.py
- hinerv.py
- layers.py
- patch_utils.py
- upsample.py
- utils.py
- - uvg_results.xlsx
- - mcljcv_results.xlsx
  - uvg_results.xlsx
datasets.py
hinerv_compress.py
hinerv_main.py
hinerv_tasks.py
LICENSE
losses.py
README.md
requirements.txt
utils.py

# Installation Guide

1. Get the code

git clone https://github.com/hmkx/HiNeRV

Downloads the entire project code from GitHub to your computer.

cd HiNeRV

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# HiNeRV

This is the repository of the paper "HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation"  (NeurIPS 2023).

By Ho Man Kwan, Ge Gao, Fan Zhang, Andrew Gower and David Bull

[Project page](https://hmkx.github.io/hinerv/)

[arXiv](https://arxiv.org/abs/2306.09818)

## Dependencies
```
accelerate==0.23.0
deepspeed==0.11.1
pytorch-msssim==1.0.0
timm==0.9.7
torch==2.1.0
torchac==0.9.3
torchvision==0.16.0
```

## Examples
### Prepare the dataset
This implementation requires first converting the videos into PNGs. For example, you can use [FFMpeg](https://www.ffmpeg.org/):
```
mkdir video
ffmpeg -video_size 1920x1080 -pixel_format yuv420p -i video.yuv video/%4d.png
```

The UVG dataset can be downloaded [here](https://ultravideo.fi/dataset.html).

### Training
To train HiNeRV-S with the UVG dataset (ReadySetGo sequence) and the video compression setting:
```
dataset_dir=~/Datasets/UVG/1920x1080
dataset_name=ReadySetGo
output=~/Models/HiNeRV
train_cfg=$(cat "cfgs/train/hinerv_1920x1080.txt")
model_cfg=$(cat "cfgs/models/uvg-hinerv-s_1920x1080.txt")
accelerate launch --mixed_precision=fp16 --dynamo_backend=inductor hinerv_main.py \
  --dataset ${dataset_dir} --dataset-name ${dataset_name} --output ${output} \
  ${train_cfg} ${model_cfg} --batch-size 144 --eval-batch-size 1 --grad-accum 1 --log-eval false --seed 0
```

The output will be saved into a new folder in the output path, e.g. ~/Models/HiNeRV/HiNeRV-20231030-032238-133f0dfc. This path can be used for resuming training/loading bitstream directly.

Please note that the batch size refers to the number of patches. Make sure to adjust it accordingly if you have changed the patch size (see 'cfgs/train/hinerv_1920x1080.txt' for details). While the original configuration uses 120x120 patches, using larger patches (e.g., 'cfgs/train/hinerv_1920x1080_480x360.txt' uses 480x360 patches) reduces overhead but may slightly impact performance.

To save the model outputs into images, set --log-eval to true.

### Evaluation
To evaluate with the compressed bitstream:
```
dataset_dir=~/Datasets/UVG/1920x1080
dataset_name=ReadySetGo
output=~/Models/HiNeRV
train_cfg=$(cat "cfgs/train/hinerv_1920x1080.txt")
model_cfg=$(cat "cfgs/models/uvg-hinerv-s_1920x1080.txt")
checkpoint_path=~/Models/HiNeRV/HiNeRV-20231030-032238-133f0dfc
accelerate launch --mixed_precision=fp16 --dynamo_backend=inductor hinerv_main.py \
  --dataset ${dataset_dir} --dataset-name ${dataset_name} --output ${output} \
  ${train_cfg} ${model_cfg} --batch-size 144 --eval-batch-size 1 --grad-accum 1 --log-eval false --seed 0 \
  --bitstream ${checkpoint_path} --bitstream-q 6 --eval-only
```

checkpoint_path is the path of the trained model directory, and bitstream-q is the quantization level.

### Other settings
To train HiNeRV-S with the 37 epochs setting (no pruning/quantization):
```
dataset_dir=~/Datasets/UVG/1920x1080
dataset_name=ReadySetGo
output=~/Models/HiNeRV
train_cfg=$(cat "cfgs/train/hinerv_1920x1080_37e_no-compress.txt")
model_cfg=$(cat "cfgs/models/uvg-hinerv-s_1920x1080.txt")
accelerate launch --mixed_precision=fp16 --dynamo_backend=inductor hinerv_main.py \
  --dataset ${dataset_dir} --dataset-name ${dataset_name} --output ${output} \
  ${train_cfg} ${model_cfg} --batch-size 144 --eval-batch-size 1 --grad-accum 1 --log-eval false --seed 0
```

# Results
This implementation has slightly improved average performance compared to the original one. The results for both the original and this version will be provided in the 'results' folder.

## Acknowledgement
Part of the implementation is based on the code from [PyTorch Image Models](https://github.com/huggingface/pytorch-image-models) and [HNeRV](https://github.com/haochen-rye/HNeRV).

## Citation
Please consider citing our work if you find that it is useful.
```
@inproceedings{
  author       = {Ho Man Kwan and Ge Gao and Fan Zhang and Andrew Gower and David Bull},
  title        = {HiNeRV: Video Compression with Hierarchical Encoding-based Neural Representation},
  booktitle    = {NeurIPS},
  year         = {2023}
}
```