diffusion-4k - GitLite

KO | EN

gitlite — search

#ai #database #cli #claude-code #react #codex #devops #animation #ruby #kotlin

diffusion-4k

(★ 366)

[CVPR 2025] Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models

파일 탐색기

ZIP 다운로드

- dataset.jpg
- demo.jpg
- sc-vae.png
- sac+logos+ava1-l14-linearMSE.pth
- autoencoder_kl.py
- config.yaml
- discriminator_vae.py
- README.md
- train_vae.py
- train_vae.sh
- vae.py
.gitignore
autoencoder_kl.py
ds_config.yaml
eval.py
generate_and_eval_fid.py
pipeline_flux_4k_wavelet.py
pipeline_sd3_4k_wavelet.py
README.md
requirements.txt
test_flux.py
test_sd3.py
train_flux.sh
train_flux_4k_wavelet.py
train_sd3.sh
train_sd3_4k_wavelet.py
vae.py

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`pip install`

설명 보기 ▼

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

🔍

`python`

설명 보기 ▼

Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.

python

REPL(대화형 셸) 시작:

python {{path/to/file.py}}

특정 Python 파일 실행:

python -i {{path/to/file.py}}

특정 Python 파일 실행 후 REPL 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">
  <a href="https://arxiv.org/abs/2503.18352"><img src="https://img.shields.io/static/v1?label=Arxiv&message=Diffusion-4K&color=red&logo=arxiv"></a> &ensp;
  <a href="https://huggingface.co/datasets/zhang0jhon/Aesthetic-4K"><img src="https://img.shields.io/static/v1?label=Dataset&message=Aesthetic-4K&color=yellow"></a> &ensp;
</div>

<p align="center" border-raduis="10px">
  <img src="asset/demo.jpg" width="100%" alt="teaser_page2"/>
</p>

## Updates
[2025/06/03] We release the updated checkpoint [flux_wavelet_v2_sc](https://huggingface.co/zhang0jhon/flux_wavelet_v2_sc) using SC-VAE and Aesthetic-Train-V2.

Please refer to our new arxiv paper [Ultra-High-Resolution Image Synthesis: Data, Method and Evaluation](https://arxiv.org/abs/2506.01331) for more details.

The training code of SC-VAE is released at [sc-vae](https://github.com/zhang0jhon/diffusion-4k/tree/main/sc-vae).

The Aesthetic-Train-V2 training set is released at [huggingface/Aesthetic-Train-V2](https://huggingface.co/datasets/zhang0jhon/Aesthetic-Train-V2).

## Introduction

We present Diffusion-4K, a novel framework for direct ultra-high-resolution image synthesis using text-to-image diffusion models.
The core advancements include:

(1) **Aesthetic-4K Benchmark**: addressing the absence of a publicly available 4K image synthesis dataset, we construct Aesthetic-4K, a comprehensive benchmark for ultra-high-resolution image generation.
We curated a high-quality 4K dataset with carefully selected images and captions generated by GPT-4o.
Additionally, we introduce GLCM Score and Compression Ratio metrics to evaluate fine details, combined with holistic measures such as FID, Aesthetics and CLIPScore for a comprehensive assessment of ultra-high-resolution images.

(2) **Wavelet-based Fine-tuning**: we propose a wavelet-based fine-tuning approach for direct training with photorealistic 4K images, applicable to various latent diffusion models, demonstrating its effectiveness in synthesizing highly detailed 4K images.

Consequently, Diffusion-4K achieves impressive performance in high-quality image synthesis and text prompt adherence, especially when powered by modern large-scale diffusion models (e.g., SD3-2B and Flux-12B).

## Aesthetic-4K Dataset

* Aesthetic-4K dataset is released at [huggingface/Aesthetic-4K](https://huggingface.co/datasets/zhang0jhon/Aesthetic-4K).

<p align="center" border-raduis="10px">
  <img src="asset/dataset.jpg" width="100%" alt="teaser_page2"/>
</p>

## Installation
Install the required packages:

```
pip install -r requirements.txt
```

Download the [dataset](https://huggingface.co/datasets/zhang0jhon/Aesthetic-4K) in *Aesthetic-4K* folder, and the pretrained latent diffusion models([SD3-2B](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers) and [Flux-12B](https://huggingface.co/black-forest-labs/FLUX.1-dev)) in *pretrain* folder.

## Training
Modify the configuration in *ds_config.yaml* accordingly (*e.g.* num_processes), and then run the script for training Flux-12B:

```
sh train_flux.sh
```

<!-- Run the script for training SD3-2B:

```
sh train_sd3.sh
``` -->

## Checkpoint

Flux-12B-WLF checkpoint is released at [flux_wavelet](https://huggingface.co/zhang0jhon/flux_wavelet).

## Image Generation
Generate images with Flux-12B:

```
CUDA_VISIBLE_DEVICES=0 python test_flux.py --guidance_scale 5.0 --height 3072 --width 4096 --seed 0 --num_inference_steps 50 --checkpoint_path $CHECKPOINT_PATH --prompt $PROMPT
```

<!-- Generate images with SD3-2B:

```
CUDA_VISIBLE_DEVICES=0 python test_sd3.py --guidance_scale 7.0 --height 3072 --width 4096 --seed 0 --num_inference_steps 28 --checkpoint_path $CHECKPOINT_PATH --prompt $PROMPT
``` -->

## Tips
For 4K image generation, it is recommended to generate at the resolution of (4096, 3072) or (3072, 4096) with guidance scale of 5 or 5.5.

## Evaluation 
Generate images for evaluation and compute FID, CLIPScore and Aesthetics:

```
python generate_and_eval_fid.py --size 2048 --checkpoint_path $CHECKPOINT_PATH --save_path $SAVE_PATH
```

Compute GLCM Score and Compression Ratio with generated images:

```
python eval.py --size 2048 --img_root $SAVE_PATH
```

## Acknowledgement

We are grateful for the following awesome projects and models when implementing Diffusion-4K:

* [Diffusers](https://github.com/huggingface/diffusers).
* [Transformers](https://github.com/huggingface/transformers).
* [SD3](https://huggingface.co/stabilityai/stable-diffusion-3-medium-diffusers).
* [Flux](https://huggingface.co/black-forest-labs/FLUX.1-dev).
* [CLIP+MLP Aesthetic Score Predictor](https://github.com/christophschuhmann/improved-aesthetic-predictor).

## Citation

If you find our paper or dataset is helpful in your research or applications, generously cite with:

```
@inproceedings{zhang2025diffusion4k,
    title={Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models},
    author={Zhang, Jinjin and Huang, Qiuyu and Liu, Junjie and Guo, Xiefan and Huang, Di},
    year={2025},
    booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
}

@misc{zhang2025ultrahighresolutionimagesynthesis,
    title={Ultra-High-Resolution Image Synthesis: Data, Method and Evaluation},
    author={Zhang, Jinjin and Huang, Qiuyu and Liu, Junjie and Guo, Xiefan and Huang, Di},
    year={2025},
    note={arXiv:2506.01331},
}
```