rl_singing_voice

KO | EN

gitlite — search

#python #ai #typescript #golang #machine-learning #deep-learning #linux #java #awesome #awesome-list #security #ios #openai #open-source

rl_singing_voice

(★ 22)

Unsupervised Representation Learning for Singing Voice Separation

rl_singing_voice Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Unsupervised Interpretable Representation Learning for Singing Voice Separation

This repository contains the PyTorch (1.4) implementation of our method for representation learning. Our method is based on (convolutional) neural networks, to learn representations from music signals that could be used for singing voice separation. The trick here is that the proposed method is employing cosine functions at the decoding stage. The resulting representation is non-negative and real-valued, and it could employed, fairly easily, by current supervised models for music source separation. The proposed method is inspired by [Sinc-Net](https://github.com/mravanelli/SincNet/) and [dDSP](https://github.com/magenta/ddsp).

### Authors

[S.I. Mimilakis](https://github.com/Js-Mim), [K. Drossos](https://www.tuni.fi/en/konstantinos-drossos), [G. Schuller](https://www.tu-ilmenau.de/mt-ams/personen/schuller-gerald/)

# What's inside?

* Code for the neural architectures used in our study and their corresponding minimization objectives (`nn_modules/`)
* Code for performing the unsupervised training (`scripts/exp_rl_*`)
* Code for reconstructing the signal(s) (`scripts/exp_fe_test.py`)
* Code for inspecting the outcome(s) of the training (`scripts/make_plots.py`)
* Code for visualizing loss functions, reading/writing audio files, and creating batches (`tools/`)
* Perks (unreported implementations/routines)
  * The discriminator-like objective, as a proxy to mutual information, reported [here](https://arxiv.org/pdf/1812.00271.pdf)

# What's not inside!

* Our papers: [EUSIPCO-2020 (accepted)](https://arxiv.org/pdf/2003.01567v4.pdf), [arXiv](https://arxiv.org/pdf/2007.02780)
* [Additional results](https://js-mim.github.io/rl_singing_voice/) that didn't fit in the EUSIPCO-2020 paper
* The used dataset &rarr; [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3338373.svg)](https://doi.org/10.5281/zenodo.3338373)
* The optimized models reported in [EUSIPCO-2020](https://arxiv.org/pdf/2003.01567v3.pdf)  &rarr; [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3707885.svg)](https://doi.org/10.5281/zenodo.3707885)
* The optimized models reported [here](https://arxiv.org/pdf/2007.02780) &rarr; [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4607439.svg)](https://doi.org/10.5281/zenodo.4607439)

# How to use
### Training
1. Download the dataset and declare the path of the downloaded dataset in `tools/helpers.py`
2. Apply any desired changes to the model by tweeking the parameters in `settings/rl_experiment_settings.py`
3. Execute `scripts/exp_rl_vanilla.py` for the basic method
4. Execute `scripts/exp_rl_sinkhorn.py` for the extended method, using Sinkhorn distances

### Testing
1. Download the dataset and declare the path of the downloaded dataset in `tools/helpers.py`
2. Download the results and place them under the `results` folder
3. Load up the desired model by declaring the experiment id in `settings/rl_experiment_settings.py` (e.g. `r-mcos8`)
4. Execute `scripts/exp_fe_test.py` (some arguments for plotting and file writing are necesary)

# Reference
If you find this code useful for your research, cite our papers:
```latex
  @inproceedings{mim20_uirl_eusipco,  
  author={S. I. Mimilakis and K. Drossos and G. Schuller},  
  title={Unsupervised Interpretable Representation Learning for Singing Voice Separation},  
  year={2020},
  booktitle={Proceedings of the 27th European Signal Processing Conference (EUSIPCO 2020)}
  }
  ```
  
  ```latex
    @misc{mimilakis2020revisiting,
    title={Revisiting Representation Learning for Singing Voice Separation with Sinkhorn Distances},
    author={S. I. Mimilakis and K. Drossos and G. Schuller},
    year={2020},
    eprint={2007.02780},
    archivePrefix={arXiv},
    primaryClass={cs.SD}
    }
  ```
  
# Acknowledgements

Stylianos Ioannis Mimilakis is supported in part by the German Research Foundation (AB 675/2-1, MU 2686/11-1).

# License

MIT

rl_singing_voice

rl_singing_voice Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

rl_singing_voice

rl_singing_voice Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README