wesam - GitLite

KO | EN

gitlite — search

#python #react #ai #typescript #android #machine-learning #rust #deep-learning #linux #windows #awesome #kubernetes #security #pytorch

wesam

(★ 181)

[CVPR 2024] Code for "Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation"

File Explorer

Download Latest Version (.zip)

- Pipeline.webp
- teaser.webp
- VISUAL.webp
- base_config.py
- config.py
- __init__.py
- CAMO.py
- COCO.py
- COCONut.py
- COD10K.py
- GDD.py
- ISIC.py
- ISTD.py
- MSD.py
- OCID.py
- OSD.py
- PascalVOC.py
- Polyp.py
- SA_1B.py
- tools.py
- - __init__.py
  - common.py
  - image_encoder.py
  - mask_decoder.py
  - prompt_encoder.py
  - sam.py
  - transformer.py
- - __init__.py
  - amg.py
  - onnx.py
  - transforms.py
- __init__.py
- automatic_mask_generator.py
- build_sam.py
- predictor.py
- eval_utils.py
- sample_utils.py
- tools.py
- visualize.py
.gitignore
adaptation.py
INSTALL.md
LICENSE
losses.py
model.py
PREPARE.md
README.md
requirements.txt
sam_lora.py
validate.py

# Installation Guide

1. Get the code

git clone https://github.com/zhang-haojie/wesam

Downloads the entire project code from GitHub to your computer.

cd wesam

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

Runs the Python script (or module).

✅ If it runs without errors and prints output in the terminal, it worked.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<div align="center">

<h1> Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation </h1>

<a href='https://zhang-haojie.github.io/project-pages/wesam.html'><img src='https://img.shields.io/badge/Project-Page-green'></a> 
<a href='http://arxiv.org/abs/2312.03502'><img src='https://img.shields.io/badge/Technique-Report-red'></a>

</div>

## 🎈 News

- [2024.2.27] Our work has been accepted to CVPR 2024 🎉
- [2024.3.1] Training and inference code released

## 🚀 Introduction

<div align="center">
<img width="800" alt="image" src="asserts/teaser.webp?raw=true">
</div>

Segment Anything Model was pre-trained on a large-scale dataset but exhibits awkward performance on diverse downstream segmentation tasks. We adapt SAM through weak supervision to enhance its generalization capabilities.

## 📻 Overview

<div align="center">
<img width="800" alt="image" src="asserts/Pipeline.webp?raw=true">
</div>

The proposed self-training architecture with anchor network regularization and contrastive loss regularization. Red arrows indicates the backpropagation flow.

## 📆 TODO

- [x] Release code

## 🎮 Getting Started

### 1. Install Environment

see [INSTALL](INSTALL.md).

### 2. Prepare Dataset and Checkpoints

see [PREPARE](PREPARE.md).

### 3. Adapt with Weak Supervision

```
# 1 modify configs/config.py 
# Prompt type: box, point, coarse

# 2 adapt
python adaptation.py
```

### 4. Validation

```
python validate.py --ckpt /path/to/checkpoint
```

## 🖼️ Visualization

<div align="center">
<img width="800" alt="image" src="asserts/VISUAL.webp?raw=true">
</div>

## 🎫 License

The content of this project itself is licensed under [LICENSE](LICENSE).

## 💡 Acknowledgement

- [SAM](https://github.com/facebookresearch/segment-anything)

- [lightning-sam](https://github.com/luca-medeiros/lightning-sam)

- [SAM-LoRA](https://github.com/JamesQFreeman/Sam_LoRA)

## 🖊️ Citation

If you find this project useful in your research, please consider cite:

```BibTeX
@inproceedings{zhang2024improving,
  title={Improving the generalization of segmentation foundation model under distribution shift via weakly supervised adaptation},
  author={Zhang, Haojie and Su, Yongyi and Xu, Xun and Jia, Kui},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={23385--23395},
  year={2024}
}
```