w2v2-age-gender-how-to

KO | EN

gitlite — search

#hacktoberfest #ai #llm #deep-learning #linux #docker #nodejs #awesome #cli #kubernetes #awesome-list #ios #mcp #open-source

w2v2-age-gender-how-to

(★ 56)

How to use our public wav2vec2 age and gender model

파일 탐색기

최종 버전 다운로드 (.zip)

- - test.yml
- requirements.txt
.gitignore
LICENSE
notebook.ipynb
README.md
requirements.txt

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# How to use our public age and gender model

An introduction to our model for 
age and gender prediction based on
[wav2vec 2.0](https://ai.facebook.com/blog/wav2vec-20-learning-the-structure-of-speech-from-raw-audio/).
The model is available from 
[doi:10.5281/zenodo.7761387](https://doi.org/10.5281/zenodo.7761387)
and released under
[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
The model was created
by fine-tuning the pre-trained
[wav2vec2-large-robust](https://huggingface.co/facebook/wav2vec2-large-robust)
model on
[aGender](https://paperswithcode.com/dataset/agender), 
[Mozilla Common Voice](https://commonvoice.mozilla.org/), 
[Timit](https://catalog.ldc.upenn.edu/LDC93s1) and 
[Voxceleb 2](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html).
We provide two variants of the model: 
one with all 24 transformer layers and 
a stripped-down version with six transformer layers.
The models were exported to
[ONNX](https://onnx.ai/).
The original 
[Torch](https://pytorch.org/)
model is hosted at Hugging Face:
[6 layers](https://huggingface.co/audeering/wav2vec2-large-robust-6-ft-age-gender)
and
[24 layers](https://huggingface.co/audeering/wav2vec2-large-robust-24-ft-age-gender).
Further details are given in the associated 
[paper](https://arxiv.org/abs/2306.16962)
and [notebook](./notebook.ipynb).

## License

The model can be used for non-commercial purposes,
see [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
For commercial usage,
a license for
[devAIce](https://www.audeering.com/devaice/)
must be obtained.
The source code in this GitHub repository 
is released under the following
[license](./LICENSE).

## Quick start

Create / activate Python virtual environment and install 
[audonnx](https://github.com/audeering/audonnx).

```
$ pip install audonnx
```

Load the model with six layers and test on random signal.

```python
import audeer
import audonnx
import numpy as np

url = 'https://zenodo.org/record/7761387/files/w2v2-L-robust-6-age-gender.25c844af-1.1.1.zip'
cache_root = audeer.mkdir('cache')
model_root = audeer.mkdir('model')

archive_path = audeer.download_url(url, cache_root, verbose=True)
audeer.extract_archive(archive_path, model_root)
model = audonnx.load(model_root)

sampling_rate = 16000
signal = np.random.normal(size=sampling_rate).astype(np.float32)
model(signal, sampling_rate)
```
```
{'hidden_states': array([[ 0.02783544,  0.01402022,  0.03839185, ...,  0.00786646,
         -0.09332313,  0.0915948 ]], dtype=float32),
 'logits_age': array([[0.3961048]], dtype=float32),
 'logits_gender': array([[ 0.32810774, -0.56528044,  0.0317882 ]], dtype=float32)}
```

The 'hidden_states' are the pooled states of the last transformer layer, 
'logits_age' provides scores for age in a range of approximately 0...1 (== 100 years) 
and 'logits_gender' expresses the confidence for being female, male or child.

## Tutorial

For a detailed introduction, please check out the [notebook](./notebook.ipynb).

```bash
$ pip install -r requirements.txt
$ jupyter notebook notebook.ipynb 
```

## Citation

If you use our model in your own work, please cite the following
[paper](https://arxiv.org/abs/2306.16962):

``` bibtex
@inproceedings{,
   author = {Felix Burkhardt and Johannes Wagner and Hagen Wierstorf and Florian Eyben and Björn Schuller},
   editor = {Peter Jax and Sebastian Mölller},
   journal = {15th ITG conference on Speech Communication},
   title = {Speech-based Age and Gender Prediction with Transformers},
   year = {2023},
}
```

w2v2-age-gender-how-to

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

w2v2-age-gender-how-to

파일 탐색기

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README