SENSE

KO | EN

gitlite — search

#cli #react #shell #ios #api #llm #openai #css #artificial-intelligence #electron

(★ 9)

[ICML 2026] What Makes Synthetic Data Effective in Image Segmentation

File Explorer

ZIP Download

- ade20k_dpt.yaml
- ade20k_mask2former.yaml
- cityscapes_dpt.yaml
- cityscapes_mask2former.yaml
- coco_dpt.yaml
- coco_mask2former.yaml
- sampler.py
- semi.py
- transform.py
- - dinov2.py
  - dinov3.py
- - __init__.py
  - mask2former_head.py
  - mask2former_transformer_decoder.py
  - pixel_decoder.py
- - fpn.py
  - mask2former_transformer_decoder.py
  - ms_deform_attn.py
  - ms_deform_pixel_decoder.py
  - position_embedding.py
  - projector.py
- - dpt.py
  - mask2former.py
- - blocks.py
  - box_ops.py
  - common.py
  - criterion.py
  - matcher.py
  - misc.py
- - ade20k_gemini.jsonl
  - cityscapes_gemini.jsonl
- - ade20k_gemma3.jsonl
  - cityscapes_gemma3.jsonl
  - coco_gemma3.jsonl
- query_prompt.py
- train_dpt.sh
- train_mask2former.sh
- - val.txt
- - val.txt
- - val.txt
- classes.py
- dist_helper.py
- ohem.py
- utils.py
.gitignore
README.md
requirements.txt
sense_dpt.py
sense_mask2former.py
supervised_dpt.py
supervised_mask2former.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

By Jinjin Zhang, Xiefan Guo, Yizhou Jin, Nan Zhou and Di Huang.

## Introduction
We propose SENSE, a unified framework that leverages flexible and scalable synthetic data to substantially enhance segmentation performance. 
Notably, SENSE is model-agnostic, compatible with diverse architectures (e.g., DPT and Mask2Former), and scales effectively across models with varying parameter capacities. 
Extensive experiments on Cityscapes, COCO, and ADE20K validate the effectiveness and generalization capability of our approach.

<!-- ## What Makes Synthetic Data Effective?

* Holistic Scene Composition

| Sparse Composition | Dense Composition |
| ------ | ------ |
| <img src="asset/section3/composition/simple.png" width="90%" alt="teaser_page2"/> | <img src="asset/section3/composition/complex.png" width="90%" alt="teaser_page2"/> |

* Local Instance Fidelity

| Coarse Fidelity | Fine Fidelity |
| ------ | ------ |
| <img src="asset/section3/fidelity/coarse.png" width="90%" alt="teaser_page2"/> | <img src="asset/section3/fidelity/fine.png" width="90%" alt="teaser_page2"/> | -->

## Installation

Install the required packages:

```
pip install -r requirements.txt
```

Generate images with prompts in *prompts* folder and prepare data:

```
├── ./data
    ├── ADE20K
      ├── ADEChallengeData2016
        ├── images
          ├── training
          ├── validataion
          ├── generative_images
        ├── annotations
          ├── training
          ├── validataion
    ├── cityscapes
      ├── leftImg8bit
      ├── generative_images
    └── coco
      ├── train2017
      ├── val2017
      ├── masks
      ├── generative_images
```

Pre-trained Models
```
├── ./pretrained
    ├── dinov2_small.pth
    ├── dinov2_base.pth
    └── dinov3_large.pth
```

Prepare the generative image path for dataloader in the *splits/{dataset}/generative/unlabeled.txt*, for example:
```
generative_images/flux_seed1/cologne_000053_000019_leftImg8bit.png None
generative_images/flux_wlf_seed1/cologne_000053_000019_leftImg8bit.png None
```

## Train

For DPT training:
```
sh scripts/train_dpt.sh
```

For Mask2Former training:
```
sh scripts/train_mask2former.sh
```

## Acknowledgement

We are grateful for the following awesome projects and models when implementing SENSE:

* [Diffusers](https://github.com/huggingface/diffusers).
* [UniMatch V2](https://github.com/LiheYoung/UniMatch-V2).
* [Flux](https://github.com/black-forest-labs/flux).
* [GroundingDINO](https://github.com/idea-research/groundingdino).
* [Flux-WLF (Diffusion-4K)](https://github.com/zhang0jhon/diffusion-4k).

## Citation

If you find this project useful, please consider citing:

```
@inproceedings{zhang2026sense,
  title={What Makes Synthetic Data Effective in Image Segmentation},
  author={Zhang, Jinjin and Guo, Xiefan and Jin, Yizhou and Zhou, Nan and Huang, Di},
  booktitle={International Conference on Machine Learning},
  year={2026}
}
```

SENSE

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

SENSE

Owner Repos

File Explorer

# Use via CDN

Was this content helpful?

Recent Feedback

Download README