Color-NeuS - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #go #machine-learning #rust #deep-learning #linux #docker #nodejs #vue #awesome-list #mcp #ios #openai

Color-NeuS

(★ 146)

[3DV 2024] Color-NeuS: Reconstructing Neural Implicit Surfaces with Color

File Explorer

Download Latest Version (.zip)

- Color_NeuS_bmvs.yml
- Color_NeuS_dtu.yml
- Color_NeuS_iho.yml
- Color_NeuS_omniobject3d.yml
- NeuS_bmvs.yml
- NeuS_dtu.yml
- NeuS_iho.yml
- NeuS_omniobject3d.yml
- pipeline.png
- - __init__.py
  - bmvs.py
  - dtu.py
  - iho_video.py
  - omniobject3d.py
- - __init__.py
  - basic_metric.py
  - similarity.py
- - __init__.py
  - model_abstraction.py
  - NeuS_Trainer.py
- - __init__.py
  - builder.py
  - config.py
  - dist_utils.py
  - etqdm.py
  - io_utils.py
  - logger.py
  - mesh_tools.py
  - misc.py
  - net_utils.py
  - read_cameras.py
  - recorder.py
  - summary_writer.py
  - transform.py
- __init__.py
- opt.py
.gitignore
evaluation.py
LICENSE
pyrightconfig.json
README.md
requirements.txt
train.py

# Installation Guide

1. Get the code

git clone https://github.com/Colmar-zlicheng/Color-NeuS

Downloads the entire project code from GitHub to your computer.

cd Color-NeuS

Moves into the project folder you just downloaded.

2. Official Install Script

Easy Recommended

Prerequisites

Python 3 Python is required to use pip.

pip install "git+https://github.com/facebookresearch/pytorch3d.git"

Installs the package published on PyPI directly — no need to clone the source.

✅ After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.

Pulled directly from this repo's README.

3. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

pip install "git+https://github.com/facebookresearch/pytorch3d.git"

Installs the package published on PyPI directly — no need to clone the source.

✅ If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<p align="center">

<h1 align="center">Color-NeuS: Reconstructing Neural Implicit Surfaces with Color</h1>
  <p align="center">
    <a><strong>Licheng Zhong</strong></a>
    ·
    <a><strong>Lixin Yang</strong></a>
    ·
    <a><strong>Kailin Li</strong></a>
    ·
    <a><strong>Haoyu Zhen</strong></a>
    ·
    <a><strong>Mei Han</strong></a>
    ·
    <a><strong>Cewu Lu</strong></a>
  </p>

<h2 align="center">3DV 2024</h2>
</p>

<h3 align="center">
  <a href="https://colmar-zlicheng.github.io/color_neus"><strong>Project Page</strong></a>
  |
  <a href="https://arxiv.org/abs/2308.06962"><strong>arXiv</strong></a>
  |
  <a href="https://drive.google.com/drive/folders/1C6vcRiy_MFMtsaR8AydRAPrLIkok-tJm?usp=sharing"><strong>Data</strong></a>
</h3>

<div align="center">
  <img src="./docs/pipeline.png" alt="Logo" width="100%">
</div>

https://github.com/Colmar-zlicheng/Color-NeuS/assets/111580763/5b9a3ed9-7d4f-48d9-84b5-a60a0a6adef9

## :rocket: Dependencies
```shell
git clone https://github.com/Colmar-zlicheng/Color-NeuS.git
cd Color-NeuS
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt
pip install "git+https://github.com/facebookresearch/pytorch3d.git"
```
## :steam_locomotive: Train
### Dataset
- [IHO Video](https://drive.google.com/drive/folders/1C6vcRiy_MFMtsaR8AydRAPrLIkok-tJm?usp=sharing)
- DTU ([NeuS Preprocess Link](https://www.dropbox.com/sh/w0y8bbdmxzik3uk/AAAaZffBiJevxQzRskoOYcyja?dl=0) | [Raw Link](http://roboimagedata.compute.dtu.dk/))
- BlendedMVS ([NeuS Preprocess Link](https://www.dropbox.com/sh/w0y8bbdmxzik3uk/AAAaZffBiJevxQzRskoOYcyja?dl=0) | [Raw Link](https://github.com/YoYo000/BlendedMVS))
- [OmniObject3D](https://opendatalab.com/OpenXD-OmniObject3D-New)

### General command
- set `${DATASET}` as one in `[iho, dtu, bmvs, omniobject3d]`
- set `${OBJECT_NAME}` as the name of the object in the dataset

```shell 
python train.py -g 0 --cfg config/Color_NeuS_${DATASET}.yml -obj ${OBJECT_NAME} --exp_id ${EXP_ID}
```

### Command line arguments
- `-g, --gpu_id`, visible GPUs for training, e.g. `-g 0`. Only supports single GPU.
- `--exp_id` specify the name of experiment, e.g. `--exp_id ${EXP_ID}`. When `--exp_id` is provided, the code requires that no uncommitted change is remained in the git repo. Otherwise, it defaults to `'default'` for training and `'eval_{cfg}_{OBJECT_NAME}'` for evaluation. All results will be saved in `exp/${EXP_ID}*{timestamp}`.

### For example
```shell
# IHO Video: ghost_bear
python train.py -g 0 --cfg config/Color_NeuS_iho.yml -obj ghost_bear --exp_id Color_NeuS_iho_ghost_bear
# DTU: dtu_scan83
python train.py -g 0 --cfg config/Color_NeuS_dtu.yml -obj 83 --exp_id Color_NeuS_dtu_83
# BlendedMVS: bmvs_bear
python train.py -g 0 --cfg config/Color_NeuS_bmvs.yml -obj bear --exp_id Color_NeuS_bmvs_bear
# OmniObject3D: doll_002
python train.py -g 0 --cfg config/Color_NeuS_omniobject3d.yml -obj doll_002 --exp_id Color_NeuS_omniobject3d_doll_002
```

### Checkpoint
All the training checkpoints are saved at `exp/${EXP_ID}_{timestamp}/checkpoints/`

### Other Method
We also provide our implementation of [NeuS](https://github.com/Totoro97/NeuS) in this repo. To train NeuS, you can replace `Color_NeuS_${DATASET}.yml` with `NeuS_${DATASET}.yml` in the above command line, such as:
```shell
# IHO Video: ghost_bear
python train.py -g 0 --cfg config/NeuS_iho.yml -obj ghost_bear --exp_id NeuS_iho_ghost_bear
```

## :monocle_face: Inference  
- set corresponding `${DATASET}` and `${OBJECT_NAME}` as above
- set `${PATH_TO_CHECKPOINT}` as the path to the checkpoint (NeuS_Trainer.pth.tar) to be loaded
```shell
python evaluation.py -g 0 --cfg config/Color_NeuS_${DATASET}.yml -obj ${OBJECT_NAME} -rr 512 --reload ${PATH_TO_CHECKPOINT}
```
- `-rr, --recon_res` is the resolution of the reconstructed mesh. The default value is 512.
## :key: License

The code provided herein are available for usage as specified in the [LICENSE](LICENSE) file. By downloading and using the code you agree to the terms in the [LICENSE](LICENSE).

## :earth_asia: Citation

```bibtex
@inproceedings{zhong2024colorneus,
    title     = {Color-NeuS: Reconstructing Neural Implicit Surfaces with Color},
    author    = {Zhong, Licheng and Yang, Lixin and Li, Kailin and Zhen, Haoyu and Han, Mei and Lu, Cewu},
    booktitle = {International Conference on 3D Vision (3DV)},
    year      = {2024}
}
```

For more questions, please contact Licheng Zhong: zlicheng@sjtu.edu.cn