Hive - GitLite

KO | EN

gitlite — search

#bash #developer-tools #api #csharp #mcp #security #javascript #terminal #css #pdf

Hive

(★ 250)

A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation

파일 탐색기

ZIP 다운로드

- logo.png
- acoustic_guitar.wav
- laughing.wav
- noisy_speech.wav
- separated_audio.wav
- ticktok_piano.wav
- water_drops.wav
- - mix_data_curation.py
  - ontology.json
- - dataset_paths.json
  - mix_from_metadata.py
- README.md
- requirements.txt
- - utils.py
- - __init__.py
- - 01_audio_chunking.py
  - 02_filter_single_label.py
  - 03_filter_single_event_qwen.py
  - 04_audioset_label_audiotag.py
  - 05_leaf_label_qwen.py
  - 06_superres_apollo.py
- - audioset_ontology.json
  - hive_ontology.json
- Apollo
- icefall
- README.md
- requirements.txt
.gitattributes
.gitignore
.gitmodules
app.py
infer_audiosep.py
infer_flowsep.py
LICENCE
README.md
requirements-infer.txt
requirements.txt

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`pip install`

설명 보기 ▼

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

🔍

`python`

설명 보기 ▼

Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.

python

REPL(대화형 셸) 시작:

python {{path/to/file.py}}

특정 Python 파일 실행:

python -i {{path/to/file.py}}

특정 Python 파일 실행 후 REPL 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation</h1>
<p align="center">
  <img src="assert/logo.png" alt="Logo" width="250"/>
</p>
<p align="center">
  <strong>Kai Li<sup>*</sup>, Jintao Cheng<sup>*</sup>, Chang Zeng, Zijun Yan, Helin Wang, Zixiong Su, Bo Zheng, Xiaolin Hu</strong><br>
    <strong>Tsinghua University, Shanda AI, Johns Hopkins University</strong><br>
    <strong><sup>*</sup>Equal contribution</strong><br>
    <strong>Completed during Kai Li's internship at Shanda AI.</strong><br>
  <a href="https://arxiv.org/abs/2601.22599">📜 Arxiv 2026</a> | <a href="https://shandaai.github.io/Hive/">🎶 Demo</a> | <a href="https://huggingface.co/datasets/ShandaAI/Hive">🤗 Dataset</a> | <a href="https://huggingface.co/spaces/JusperLee/Hive">🤗 Space</a>
</p>

<p align="center">
  <img src="https://visitor-badge.laobi.icu/badge?page_id=ShandaAI.Hive" alt="访客统计" />
  <img src="https://img.shields.io/github/stars/ShandaAI/Hive?style=social" alt="GitHub stars">
  <img alt="Static Badge" src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" />
</p>

---

## 💥 News

- [2026-03-09] We release `infer_audiosep.py` for one-command AudioSep inference in Hive. The script automatically downloads config and checkpoints from [ShandaAI/AudioSep-hive](https://huggingface.co/ShandaAI/AudioSep-hive). 🚀
- [2026-03-09] We release `infer_flowsep.py` for one-command FlowSep inference in Hive, with automatic config and checkpoint download from [ShandaAI/FlowSep-hive](https://huggingface.co/ShandaAI/FlowSep-hive). 🚀
- [2026-03-09] We release `app.py`, a unified Gradio demo that supports both **AudioSep-hive** and **FlowSep-hive** in a single interface. 🚀
- [2026-03-09] Community Hugging Face Space is available at [JusperLee/Hive](https://huggingface.co/spaces/JusperLee/Hive) for quick interactive demo. 🚀
- [2026-02-09] Thanks to [@faiteamartaliius](https://huggingface.co/faiteamartaliius) for using this codebase to synthesize data and publicly sharing a third-party Hive-style dataset: [faiteamartaliius/Hive](https://huggingface.co/datasets/faiteamartaliius/Hive). 🙏

## 📌 Table of Contents

- [📄 Abstract](#-abstract)
- [📂 Repository Structure](#-repository-structure)
- [🍯 Hive Dataset](#-hive-dataset)
- [⚙️ Data Collection Pipeline](#️-data-collection-pipeline)
- [🎧 Inference](#-inference)
- [🖥️ Gradio App](#️-gradio-app)
- [📖 Citation](#-citation)
- [⚖️ License](#️-license)
- [🙏 Acknowledgments](#-acknowledgments)

## 📄 Abstract

<p align="justify">
Query-based universal sound separation is fundamental to intelligent auditory systems, aiming to isolate specific sources from unconstrained mixtures. 
Despite recent advances, existing methods continue to suffer from residual interference in complex acoustic scenes. 
This performance limitation stems largely from a data bottleneck: ubiquitous in-the-wild datasets contain weak labels and severe event co-occurrence. 
These flaws induce models to learn spurious correlations between background noise and target categories instead of robust acoustic features. 
To address this, we propose an automated pipeline that eliminates co-occurrence noise by mining high-purity single-event segments 
from unconstrained recordings and synthesizing mixtures via semantically consistent strategies. 
Utilizing this pipeline, we constructed <i>Hive</i>, a high-quality synthetic dataset comprising 2k hours of audio. 
<a style="color:blue;">Experimental results demonstrate that, despite using only <b>~0.2%</b> of the data scale of million-hour baselines, 
models trained on Hive achieve competitive separation accuracy and perceptual quality.</a> 
Moreover, these models exhibit remarkable zero-shot generalization on out-of-distribution evaluation benchmarks such as MUSDB18-HQ and USS-Bench. 
These findings highlight that <a style="background-color:LightYellow;color:red;">prioritizing supervision purity enables significant data efficiency</a>, 
offering a new paradigm for training robust auditory foundation models with reduced computational costs.
</p>

## 📂 Repository Structure

```
.
├── hive_dataset/                           # Hive Dataset generation and curation
│   ├── mix_from_metadata/                  # Generate mixtures from metadata
│   │   ├── mix_from_metadata.py
│   │   └── dataset_paths.json
│   ├── mix_curation/                       # Data curation for mix audio
│   │   ├── mix_data_curation.py
│   │   └── ontology.json
│   ├── README.md                           # Dataset documentation
│   ├── requirements.txt
│   └── LICENSE
├── pipeline/                               # Single-Event Data Collection Pipeline
│   ├── code/                               # Pipeline scripts
│   │   ├── 01_audio_chunking.py
│   │   ├── 02_filter_single_label.py
│   │   ├── 03_filter_single_event_qwen.py
│   │   ├── 04_audioset_label_audiotag.py
│   │   ├── 05_leaf_label_qwen.py
│   │   └── 06_superres_apollo.py
│   ├── data/                               # Pipeline data directories
│   ├── ontology/                           # AudioSet ontologies
│   ├── icefall/                            # AudioTag model repository
│   ├── Apollo/                             # Apollo model repository
│   ├── requirements.txt                    # Pipeline dependencies
│   └── README.md                           # Pipeline documentation
├── LICENCE                                 # MIT License
└── README.md                              
```

## 🍯 Hive Dataset

**Hive** is a high-quality synthetic dataset with 2,442 hours of raw audio and 19.6M mixtures for Universal Sound Separation.

**Features:**
- 283 sound categories from AudioSet ontology
- Semantically consistent mixing logic
- 44.1kHz sample rate

Please refer to [`hive_dataset/`](hive_dataset/) for details

## ⚙️ Data Collection Pipeline

An automated 6-step pipeline for mining high-purity single-event audio from weakly-labeled sources.

**Pipeline Stages:**
1. Audio Chunking - Split long audio into segments
2. Single Label Filtering - Remove multi-label samples
3. Single Event Filtering - Verify acoustic purity with Qwen3-Omni
4. AudioSet Label Tagging - Assign ontology labels with AudioTag
5. Leaf Label Classification - Refine to leaf nodes with Qwen3-Omni
6. Audio Super-Resolution - Upsample to 44.1kHz with Apollo

Please refer to [`pipeline/`](pipeline/) for details

## 🎧 Inference

Hive provides two inference scripts with automatic checkpoint/config download from Hugging Face:

- `infer_audiosep.py` -> [ShandaAI/AudioSep-hive](https://huggingface.co/ShandaAI/AudioSep-hive)
- `infer_flowsep.py` -> [ShandaAI/FlowSep-hive](https://huggingface.co/ShandaAI/FlowSep-hive)

### 1) Install dependencies

```bash
cd Hive
pip install torch torchaudio librosa pyyaml pytorch-lightning huggingface_hub gradio
```

### 2) AudioSep inference

```bash
python infer_audiosep.py \
  --audio_file /path/to/mixture.wav \
  --text "acoustic guitar" \
  --output_file /path/to/audiosep_output.wav
```

### 3) FlowSep inference

```bash
python infer_flowsep.py \
  --audio_file /path/to/mixture.wav \
  --text "acoustic guitar" \
  --output_file /path/to/flowsep_output.wav
```

## 🖥️ Gradio App

`app.py` launches an interactive local demo with both models in one UI:

- Model choices: `AudioSep-hive`, `FlowSep-hive`
- Input: mixed audio + text query
- Output: separated waveform

Run:

```bash
cd Hive
python app.py
```

Then open the local Gradio URL printed in terminal.

## 📖 Citation

If you use this code or the Hive Dataset, please cite:

```bibtex
@article{li2026semantically,
  title={A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation},
  author={Li, Kai and Cheng, Jintao and Zeng, Chang and Yan, Zijun and Wang, Helin and Su, Zixiong and Zheng, Bo and Hu, Xiaolin},
  journal={arXiv preprint arXiv:2601.22599},
  year={2026}
}
```

## ⚖️ License

### Project License

This project is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.

### Model Licenses

- **Qwen3-Omni**: Apache 2.0
- **AudioTag**: Apache 2.0
- **Apollo**: Check model repository for specific license

## 🙏 Acknowledgments

The Hive dataset is a collaborative achievement built upon the foundation of the open-source audio community. We extend our deepest gratitude to the researchers and organizations who curated the twelve foundational datasets. Their work provides the essential long-tailed acoustic space for advancing **Universal Sound Separation**.

### 🏛️ Foundational Data Sources

We gratefully acknowledge the following core datasets which provided the majority of our high-fidelity clips:

- **BBC Sound Effects** (369,603 clips, 1,020.62h) - Professional-grade recordings with broadcast-level fidelity under Remix License
- **AudioSet** (326,890 clips, 896.61h) - Large-scale benchmark from YouTube under CC BY (Google)
- **VGGSound** (115,191 clips, 319.10h) - Real-world acoustic diversity under CC BY 4.0 (University of Oxford)
- **FreeSound** (17,451 clips, 46.90h) - Rich crowdsourced soundscapes under CC0/BY/BY-NC (MTG-UPF)

### 🎯 Specialized Domain Contributors

Our sincere thanks go to the following datasets for providing the raw source audio that forms the specialized domains of the **Hive Dataset**:

**Music & Speech:**
- **MUSIC21** (32,701 clips, 90.28h) - Solo and ensemble instruments for harmonic structure modeling
- **Voicebank-DEMAND** (12,376 clips, 9.94h) - Clean speech signals under CC BY 4.0
- **FSD50K** (636 clips, 0.80h) - Finely annotated subset based on AudioSet ontology

**Environmental & Events:**
- **ClothoV2** (14,759 clips, 38.19h) - Audio captioning dataset with rich temporal evolution
- **AVE** (3,054 clips, 6.91h) - Audio-visual event localization under CC BY-NC-SA
- **SoundBible** (2,501 clips, 5.78h) - Curated short clips under CC BY 4.0
- **DCASE** (1,969 clips, 5.46h) - Acoustic scene detection challenges
- **ESC50** (1,433 clips, 1.99h) - Environmental sound classification benchmark under CC BY-NC 3.0

### ⚖️ License & Ethical Compliance

All source data were processed in strict accordance with their respective licenses (e.g., CC BY, CC0, Remix License). An automated data collection pipeline was employed to ensure that only semantically aligned and single-label pure segments were extracted, respecting the original intent of the data contributors while enhancing their utility for sound separation tasks.

**Important Note**: This repository releases only the **metadata** (JSON files containing mixing parameters and source references) for reproducibility. We do **not** redistribute the original audio files from the source datasets. Users must independently download and prepare the source datasets according to their respective licenses and terms of use.

*We thank all original contributors for their invaluable service to the scientific community.*