MODNet - GitLite

KO | EN

gitlite — search

#react #ai #typescript #llm #machine-learning #deep-learning #docker #macos #awesome-list #vue #security #pytorch #openai #open-source

MODNet

(★ 4,351)

A Trimap-Free Portrait Matting Solution in Real Time [AAAI 2022]

File Explorer

Download Latest Version (.zip)

- - commercial_image_matting_model_result.gif
  - commercial_image_matting_website.gif
  - homepage_demo.gif
  - image_matting_demo.gif
  - video_matting_demo.gif
- __init__.py
- export_onnx.py
- inference_onnx.py
- modnet_onnx.py
- README.md
- requirements.txt
- README.md
- - __init__.py
  - modnet.py
- __init__.py
- trainer.py
- __init__.py
- export_torchscript.py
- modnet_torchscript.py
- README.md
.gitignore
LICENSE
README.md

# Installation Guide

1. Get the code

git clone https://github.com/ZHKKKe/MODNet

Downloads the entire project code from GitHub to your computer.

cd MODNet

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r onnx/requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h2 align="center">MODNet: Trimap-Free Portrait Matting in Real Time</h2>

<div align="center"><i>MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition (AAAI 2022)</i></div>

<br />

<img src="doc/gif/homepage_demo.gif" width="100%">

<div align="center">MODNet is a model for <b>real-time</b> portrait matting with <b>only RGB image input</b></div>
<div align="center">MODNet是一个<b>仅需RGB图片输入</b>的<b>实时</b>人像抠图模型</div>

<br />

<p align="center">
  <a href="#online-application-在线应用">Online Application (在线应用)</a> |
  <a href="#research-demo">Research Demo</a> |
  <a href="https://arxiv.org/pdf/2011.11961.pdf">AAAI 2022 Paper</a> |
  <a href="https://youtu.be/PqJ3BRHX3Lc">Supplementary Video</a>
</p>

<p align="center">
  <a href="#community">Community</a> |
  <a href="#code">Code</a> |
  <a href="#ppm-benchmark">PPM Benchmark</a> |
  <a href="#license">License</a> |
  <a href="#acknowledgement">Acknowledgement</a> |
  <a href="#citation">Citation</a> |
  <a href="#contact">Contact</a>
</p>

---

## Online Application (在线应用)

The model used in the online demo (unpublished) is only **7M**! Process **2K** resolution image with a **Fast** speed on common PCs or Mobiles! **Beter** than research demos!  
Please try online portrait image matting on [my personal homepage](https://zhke.io/#/?modnet_demo) for fun!

在线应用中使用的模型（未发布）大小仅为**7M**！可以在普通PC或移动设备上**快速**处理具有**2K**分辨率的图像！效果比研究示例**更好**！  
请通过[我的主页](https://zhke.io/#/?modnet_demo)在线尝试图片抠像！

## Research Demo

All the models behind the following demos are trained on the datasets mentioned in [our paper](https://arxiv.org/pdf/2011.11961.pdf).

### Portrait Image Matting
We provide an [online Colab demo](https://colab.research.google.com/drive/1GANpbKT06aEFiW-Ssx0DQnnEADcXwQG6?usp=sharing) for portrait image matting.  
It allows you to upload portrait images and predict/visualize/download the alpha mattes.

### Portrait Video Matting
We provide two real-time portrait video matting demos based on WebCam. When using the demo, you can move the WebCam around at will.
If you have an Ubuntu system, we recommend you to try the [offline demo](demo/video_matting/webcam) to get a higher *fps*. Otherwise, you can access the [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).  
We also provide an [offline demo](demo/video_matting/custom) that allows you to process custom videos.

## Community

We share some cool applications/extentions of MODNet built by the community.

- **Colab Demo of Bokeh (Blur Background)**  
You can try [this Colab demo](https://colab.research.google.com/github/eyaler/avatars4all/blob/master/yarok.ipynb) (built by [@eyaler](https://github.com/eyaler)) to blur the backgroud based on MODNet!

- **ONNX Version of MODNet**  
You can convert the pre-trained MODNet to an ONNX model by using [this code](onnx) (provided by [@manthan3C273](https://github.com/manthan3C273)). You can also try [this Colab demo](https://colab.research.google.com/drive/1P3cWtg8fnmu9karZHYDAtmm1vj1rgA-f?usp=sharing) for MODNet image matting (ONNX version).

- **TorchScript Version of MODNet**  
You can convert the pre-trained MODNet to an TorchScript model by using [this code](torchscript) (provided by [@yarkable](https://github.com/yarkable)).

- **TensorRT Version of MODNet**  
You can access [this Github repository](https://github.com/jkjung-avt/tensorrt_demos) to try the TensorRT version of MODNet (provided by [@jkjung-avt](https://github.com/jkjung-avt)).

- **Docker Container for MODnet**  
You can access [this Github repository](https://github.com/nahidalam/modnet_docker) for a containerized version of MODNet with the Docker environment (provided by [@nahidalam](https://github.com/nahidalam)).

There are some resources about MODNet from the community.
- [Video from What's AI YouTube Channel](https://youtu.be/rUo0wuVyefU)
- [Article from Louis Bouchard's Blog](https://www.louisbouchard.ai/remove-background/)

## Code
We provide the [code](src/trainer.py) of MODNet training iteration, including:
- **Supervised Training**: Train MODNet on a labeled matting dataset
- **SOC Adaptation**: Adapt a trained MODNet to an unlabeled dataset

In code comments, we provide examples for using the functions.

## PPM Benchmark
The PPM benchmark is released in a separate repository [PPM](https://github.com/ZHKKKe/PPM).

## License
The code, models, and demos in this repository (excluding GIF files under the folder `doc/gif`) are released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) license.

## Acknowledgement  
- We thank  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[@yzhou0919](https://github.com/yzhou0919), [@eyaler](https://github.com/eyaler), [@manthan3C273](https://github.com/manthan3C273),  [@yarkable](https://github.com/yarkable), [@jkjung-avt](https://github.com/jkjung-avt),  [@manzke](https://github.com/manzke),  [@nahidalam](https://github.com/nahidalam),  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[the Gradio team](https://github.com/gradio-app/gradio), [What's AI YouTube Channel](https://www.youtube.com/channel/UCUzGQrN-lyyc0BWTYoJM_Sg), [Louis Bouchard's Blog](https://www.louisbouchard.ai),  
for their contributions to this repository or their cool applications/extentions/resources of MODNet.

## Citation
If this work helps your research, please consider to cite:

```bibtex
@InProceedings{MODNet,
  author = {Zhanghan Ke and Jiayu Sun and Kaican Li and Qiong Yan and Rynson W.H. Lau},
  title = {MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition},
  booktitle = {AAAI},
  year = {2022},
}
```

## Contact
This repository is maintained by Zhanghan Ke ([@ZHKKKe](https://github.com/ZHKKKe)).  
For questions, please contact `kezhanghan@outlook.com`.