BNTT-Batch-Normalization-Through-Time - GitLite

KO | EN

gitlite — search

#machine-learning #api #nextjs #postgresql #rust #html #awesome-list #agents #privacy #list

BNTT-Batch-Normalization-Through-Time

(★ 49)

BNTT - Frontiers in Neuroscience (2021)

파일 탐색기

ZIP 다운로드

model.py
README.md
train.py
utills.py

# CDN으로 사용하기

jsDelivr

jsDelivr는 공개 GitHub 리포지토리를 별도 설정 없이 CDN으로 즉시 서빙합니다. 버전과 파일을 고르면 웹페이지에 바로 붙일 수 있는 링크와 예시 코드가 만들어집니다.

버전

파일

링크

예시

명령어 용어집

이 문서에서 사용된 명령어를 모아봤습니다. 낯선 명령어가 있다면 펼쳐서 확인해보세요.

🔍

`conda create`

설명 보기 ▼

Create new conda environments.

conda create {{[-y|--yes]}} {{[-n|--name]}} py39 python=3.9 "numpy>=1.11" scipy

Create a new environment named `py39`, install Python 3.9, NumPy v1.11 or above in it, and the latest stable version of SciPy. Say yes to all confirmations:

conda create {{[-n|--name]}} myenv --file {{file1.yml}} --file {{file2.yml}}

Create a new environment named `myenv` and install packages listed in files:

conda create {{[-p|--prefix]}} {{path/to/myenv}}

Create a new environment at a custom path (i.e. prefix):

🔍

`conda install`

설명 보기 ▼

Install packages into an existing conda environment.

conda install {{package1 package2 ...}}

Install one or more package into the currently active conda environment:

conda install {{[-c|--channel]}} conda-forge {{package}}

Install a single package into the currently active conda environment using channel conda-forge:

conda install {{[-c|--channel]}} conda-forge --override-channels {{package}}

Install a single package into the currently active conda environment using channel conda-forge and ignoring other channels:

🔍

`python`

설명 보기 ▼

Python 언어 인터프리터.
더 많은 정보: <https://docs.python.org/using/cmdline.html>.

python

REPL(대화형 셸) 시작:

python {{path/to/file.py}}

특정 Python 파일 실행:

python -i {{path/to/file.py}}

특정 Python 파일 실행 후 REPL 시작:

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

This repository contains the source code associated with [arXiv preprint arXiv:2010.01729][arXiv preprint arXiv:2010.01729]

Accepted to Frontiers in Neuroscience (2021)

[arXiv preprint arXiv:2010.01729]: https://arxiv.org/abs/2010.01729

## Introduction

Spiking Neural Networks (SNNs) have recently emerged as an alternative to deep learning owing to sparse, asynchronous and binary event (or spike) driven processing, that can yield huge energy efficiency benefits on neuromorphic hardware. However, training high-accuracy and low-latency SNNs from scratch suffers from non-differentiable nature of a spiking neuron. To address this training issue in SNNs, we revisit batch normalization and propose a temporal Batch Normalization Through Time (BNTT) technique. Most prior SNN works till now have disregarded batch normalization deeming it ineffective for training temporal SNNs. Different from previous works, our proposed BNTT decouples the parameters in a BNTT layer along the time axis to capture the temporal dynamics of spikes. The temporally evolving learnable parameters in BNTT allow a neuron to control its spike rate through different time-steps, enabling low-latency and low-energy training from scratch. We conduct experiments on CIFAR-10, CIFAR-100, Tiny-ImageNet and event-driven DVS-CIFAR10 datasets. BNTT allows us to train deep SNN architectures from scratch, for the first time, on complex datasets with just few 25-30 time-steps. We also propose an early exit algorithm using the distribution of parameters in BNTT to reduce the latency at inference, that further improves the energy-efficiency.

## Prerequisites
* Ubuntu 18.04    
* Python 3.6+    
* PyTorch 1.5+ (recent version is recommended)     
* NVIDIA GPU (>= 12GB)

## Getting Started

### Installation
* Configure virtual (anaconda) environment
```
conda create -n env_name python=3.7
source activate env_name
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
```

## Training and testing

* We provide VGG9/VGG11 architectures on CIFAR10/CIAR100 datasets
* ```train.py```: code for training  
* ```model.py```: code for VGG9/VGG11 Spiking Neural Networks with BNTT  
* ```utill.py```: code for accuracy calculation / learning rate scheduler

*  Run the following command for VGG9 SNN on CIFAR10

```
python train.py --num_steps 25 --lr 0.3 --arch 'vgg9' --dataset 'cifar10' --batch_size 256 --leak_mem 0.95 --num_workers 4 --num_epochs 100
```

*  Run the following command for VGG11 SNN on CIFAR100

```
python train.py --num_steps 30 --lr 0.3 --arch 'vgg11' --dataset 'cifar100' --batch_size 128 --leak_mem 0.99 --num_workers 4 --num_epochs 100
```

## Citation
 
Please consider citing our paper:
 ```
 @article{kim2020revisiting,
  title={Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch},
  author={Kim, Youngeun and Panda, Priyadarshini},
  journal={arXiv preprint arXiv:2010.01729},
  year={2020}
}
 ```