MEFLUT - GitLite

KO | EN

gitlite — search

#javascript #python #hacktoberfest #llm #go #machine-learning #rust #linux #java #windows #kubernetes #vue #awesome-list #mcp

MEFLUT

(★ 130)

【ICCV 2023】This is the official Pytorch implementation of our ICCV2023 paper "MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion".

File Explorer

Download Latest Version (.zip)

- pipeline.png
- - 0.jpg
  - 1.jpg
  - 2.jpg
  - 3.jpg
- - 0.jpg
  - 1.jpg
  - 2.jpg
  - 3.jpg
- - 0.jpg
  - 1.jpg
  - 2.jpg
  - 3.jpg
- test.txt
- data_selection.py
- __init__.py
- batch_transformers.py
- ImageDataset.py
- __init__.py
- box_filter.py
- guided_filter.py
- __init__.py
- mefssim.py
- - 0_weight.txt
  - 1_weight.txt
- - 0_weight.txt
  - 1_weight.txt
  - 2_weight.txt
- - 0_weight.txt
  - 1_weight.txt
  - 2_weight.txt
  - 3_weight.txt
- __init__.py
- model.py
- model_base.py
.gitignore
LICENSE
main.py
README.md
requirements.txt
test.py
train.py

# Installation Guide

1. Get the code

git clone https://github.com/Hedlen/MEFLUT

Downloads the entire project code from GitHub to your computer.

cd MEFLUT

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

✅ If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# [ICCV 2023] MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion [Paper](https://arxiv.org/abs/2309.11847)

<h4 align="center">Ting Jiang<sup>1,*</sup>, Chuan Wang<sup>1</sup>,  Xinpeng Li<sup>1</sup>,  Ru Li<sup>1</sup>,  Haoqiang Fan<sup>1</sup>,  Shuaicheng Liu<sup>2,1,†</sup></center>
<h4 align="center"> 1. Megvii Research, 2. University of Electronic Science and Technology of China</center>
<h6 align="center"> †Corresponding author</center>

## Abstract

In this paper, we introduce a new approach for high-quality multi-exposure image fusion (MEF). We show that the fusion weights of an exposure can be encoded into a 1D lookup table (LUT), which takes pixel intensity value as input and produces fusion weight as output. We learn one 1D LUT for each exposure, then all the pixels from different exposures can query 1D LUT of that exposure independently for high-quality and efficient fusion. Specifically, to learn these 1D LUTs, we involve attention mechanism in various dimensions including frame, channel and spatial ones into the MEF task so as to bring us significant quality improvement over the state-of-the-art (SOTA). In addition, we collect a new MEF dataset consisting of 960 samples, 155 of which are manually tuned by professionals as ground-truth for evaluation. Our network is trained by this dataset in an unsupervised manner. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the SOTA in our and another representative dataset SICE, both qualitatively and quantitatively. Moreover, our 1D LUT approach takes less than 4ms to run a 4K image on a PC GPU. Given its high quality, efficiency and robustness, our method has been shipped into millions of Android mobiles across multiple brands world-wide.

## Pipeline

![pipeline](https://raw.githubusercontent.com/Hedlen/MEFLUT/main/assets/pipeline.png)

## Our Dataset
The  dataset including training and testing dataset can be download from [Baidu Netdisk](Link: https://pan.baidu.com/s/1F3Re_2CWYibsqejN-pA2tw?pwd=2tih 提取码: 2tih ) or [Huggingface](https://huggingface.co/datasets/peterJing/MEFLUT/tree/main). The dataset is organized as follow:

```
Training set
|--train
|  |--0  // Sample ID, internal file ranging from EV-4 to EV+2. 
|  |  |--0.jpg  
|  |  |--1.jpg
|  |  |--2.jpg
|  |  |--3.jpg
|  |--1
|  |--2
|  |--3
|  |--...
|--train.txt //  Training set index.

Test set
|--test
|  |--0 // Sample ID, internal file ranging from EV-4 to EV+2. 
|  |  |--0.jpg  
|  |  |--1.jpg
|  |  |--2.jpg
|  |  |--3.jpg
|  |--1
|  |--2
|  |--3
|  |--...
|--test.txt //  Test set index.
```
I write a data_selection.py code under the data_utils folder to handle the user can select 2, 3 or 4 frames to operate as needed. In the actual training, our experiments are also randomly selected, so there may be bias in the experimental results.

## Usage
### Requirements

This code is developed under

* Python 3.7.10
* Pytorch 1.12.1
* torchvision 0.13.1
* CUDA 10.0 on Ubuntu 18.04

We strongly recommend you using anaconda to ensure you can get the same results as us.

Install the require dependencies:

```bash
conda create -n meflut  python=3.7.10
conda activate meflut
pip install -r requirements.txt
```

## Usage
### Training
```
1. cd MEFLUT
2. python main.py --status 1dluts_train # execute in GPU
```

### Testing
```
1. cd MEFLUT
2. python main.py --status 1dluts_eval # execute in GPU
```
## 🚀 Beyond MEF: LUT-Based AI for Edge Devices

The 1D LUT paradigm introduced in this work is more than a fusion technique — it represents a principled approach to deploying AI on **chips and terminal devices** where quality, latency, and power consumption must be carefully balanced.

**Why LUTs matter for on-device AI:**
- A trained neural network can be distilled into compact LUTs, enabling **inference without running the full model** at deployment time
- LUT lookup is an extremely lightweight operation, making it ideal for **ISP pipelines, mobile SoCs, and dedicated imaging chips**
- This is a highly insightful form of **model compression**: the network's learned knowledge is baked into a table, achieving near-zero runtime cost while preserving quality
- Our method has already been **deployed in production on terminal devices and chips**, demonstrating real-world viability at scale

**Extensibility beyond 1D:**

The 1D LUT idea naturally generalizes to higher dimensions:

| LUT Type | Input | Typical Use Case |
|----------|-------|-----------------|
| 1D LUT | Single channel intensity | Tone mapping, exposure fusion (this work) |
| 2D LUT | Two-channel pairs (e.g., Y+context) | Noise-aware enhancement, local tone control |
| 3D LUT | RGB triplet | Color grading, white balance, style transfer |

This family of LUT-based methods opens up a broad design space for **efficient, hardware-friendly AI image processing**, applicable to:
- Computational photography pipelines
- Real-time video enhancement on edge NPUs
- Low-power wearable and IoT imaging devices
- Any scenario requiring a favorable trade-off between AI quality and compute budget

We hope this work inspires further exploration of LUT-based compression across diverse vision tasks.

## ⚠️ Maintenance Notice

Please note the following:

1. **Weights not provided**: Only the 1D LUTs (under the `luts/` folder) are retained in this repository. Model weights are **not included** — researchers can train the model from scratch using either the training code and the dataset introduced in our paper, or their own custom dataset.

2. **Known dataset issue**: Some training dataset folders (e.g., folders 584 and 486) contain erroneous files named `._0.jpg`, which may cause training failures. If you encounter this issue, a community-maintained fork may help: [https://github.com/qulishen/MEFLUT](https://github.com/qulishen/MEFLUT). Thanks to [@qulishen](https://github.com/qulishen) for the contribution.

## Citation

If you find this work helpful, please cite our paper:

```
@InProceedings{Jiang_2023_ICCV,
    author    = {Jiang, Ting and Wang, Chuan and Li, Xinpeng and Li, Ru and Fan, Haoqiang and Liu, Shuaicheng},
    title     = {MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {10542-10551}
}
```