multi-source-diffusion-models - GitLite

KO | EN

gitlite — search

#react #video #ai #svg #linux #framework #markdown #swift #large-language-models #cloud-native

multi-source-diffusion-models

(★ 171)

No description available.

File Explorer

ZIP Download

- README.md
- convert_data_format.sh
- README.md
- __init__.py
- evaluate_separation.py
- experiments.py
- eval_msdm_dirac.yaml
- eval_msdm_gaussian.yaml
- eval_weakly_msdm_dirac.yaml
- eval_weakly_msdm_gaussian.yaml
- train_msdm.yaml
- train_weakly_msdm.yaml
- __init__.py
- data.py
- module_base.py
- separation.py
- utils.py
- MSDM-generation.ipynb
.env.tmp
.gitignore
config.yaml
env.yaml
evaluate.py
README.md
requirements.txt
train.py

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Version

File

Link

Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

`conda activate`

View Details ▼

Activate a conda environment.
See also: `conda deactivate`.

conda activate myenv

Activate an existing environment named `myenv`:

conda activate {{path/to/myenv}}

Activate an existing environment located at custom path:

conda activate --stack myenv

Stack `myenv` environment on top of a previous environment making libraries/commands/variables from both accessible:

🔍

`conda env`

View Details ▼

Manage conda environments.

conda env create {{[-f|--file]}} {{path/to/file}}

Create an environment from an environment file (YAML, TXT, etc.):

conda env remove {{[-n|--name]}} {{environment_name}}

Delete an environment and everything in it:

conda env update {{[-f|--file]}} {{path/to/file}} --prune

Update an environment based on an environment file:

🔍

`jupyter`

View Details ▼

Web application to create and share documents that contain code, visualizations, and notes.
Primarily used for data analysis, scientific computing, and machine learning.

jupyter notebook

Start a Jupyter notebook server in the current directory:

jupyter notebook {{path/to/file}}.ipynb

Open a specific Jupyter notebook:

jupyter nbconvert --to {{html|markdown|pdf|script|...}} {{path/to/file}}.ipynb

Export a specific Jupyter notebook into another format:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Official repository for [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://arxiv.org/abs/2302.02257).

Demo website available [here](https://gladia-research-group.github.io/multi-source-diffusion-models/).

# Installation
The environment for running our code can be installed using conda:
```bash
# Install environment
conda env create -f env.yaml

# Activate the environment
conda activate msdm
```

## Download dataset
This step is optional and required only if you are interested in training a model from scratch or reproducing the results from the paper.

Detailed instructions to download our dataset can be found [here](data/README.md).

## Download pretrained checkpoints
It is possible to download the pretrained models we used in our experiments. These are useful to run the generation, inpainting and separation scripts available in this repository.

Detailed instruction to download our pretrained models can be found [here](ckpts/README.md).

# Training

> ⚠️ **NOTE:**  
> Before executing the training script, you have to rename the file `.env.tmp` to `.env` and change the **WANDB_\*** environment variables to match your 
personal or institutional account.

You can run the model by invoking the `train.py` script:
```bash
# Activate environment
conda activate msdm

# Run training script
PYTHONPATH=. python train.py exp=train_msdm
```
The results will be logged into your **Weight & Biases** account, using the information provided before.

# Generation and Inpainting

## Run Notebook
Take a look at the notebook `notebook/MSDM-generation.ipynb` for generation and inpainting examples.

> ⚠️ **NOTE:**  
> Before running the notevook, download the `glorious-star-335` pretrained checkpoint.

The notebook can be executed using the command
```bash
jupyter notebook notebooks/MSDM-generation.ipynb
```
The example found in our [demo website](ttps://gladia-research-group.github.io/multi-source-diffusion-models/)
 were produced using a similar notebook.

# Separation evaluation
> ⚠️ **NOTE:**   
> You need to download the test set and the pretrained checkpoints in order to run the evaluation scripts.

Running the script
```bash
# Run evaluation script for MSDM model
PYTHONPATH=. python evaluate.py exp=eval_msdm_dirac
```
will separate the test set and evaluate its quality using the SI-SNRi metric. After executing the script, the separation results can be found inside of the `output/separations/<exp-name>` directory. The metrics are reported inside of the `metrics.csv` file, in the separation folder.

It is possible to use different configuration for the separation script, by changing the `exp=<config>` input argument.
You can choose between the following configurations:
```bash
exp=eval_msdm_dirac # < second best performance
exp=eval_msdm_gaussian
exp=eval_weakly_msdm_dirac # < best performance (4x slower)
exp=eval_weakly_msdm_gaussian
```