RPSnet - GitLite

KO | EN

gitlite — search

#python #hacktoberfest #ai #llm #android #deep-learning #nodejs #java #windows #cli #awesome-list #security #mcp #openai

RPSnet

(★ 53)

Official Implementation of "Random Path Selection for Incremental Learning" paper. NeurIPS 2019

파일 탐색기

최종 버전 다운로드 (.zip)

- - cifar100_10.pkl
  - generate_cifar100_labels.py
  - generate_ILSVRC_labels.py
  - meta_CIFAR100.pkl
- - __init__.py
  - eval.py
  - logger.py
  - misc.py
  - visualize.py
- cifar_dataset.py
- iCaRL.py
- iCaRL.sh
- learner.py
- rps_net.py
- util.py
- cifar100_10.pkl
- generate_cifar100_labels.py
- generate_ILSVRC_labels.py
- meta_CIFAR100.pkl
- - .gitignore
  - demo.gif
  - LICENSE
  - MANIFEST.in
  - README.rst
  - setup.py
  - test_progress.py
- 10.png
- 20.png
- 50.png
- __init__.py
- eval.py
- imagenet_100.png
- logger.py
- misc.py
- path.png
- rpsnet.png
- visualize.py
.gitignore
cifar.py
CIFAR100.sh
cifar_dataset.py
imagenet.py
learner.py
mnist.py
MNIST.sh
plot.ipynb
README.md
rps_net.py
util.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/brjathu/RPSnet

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd RPSnet

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

⚠️ 이 프로젝트는 규모가 큰 저장소라, 이 방법이 실제 핵심 제품이 아니라 내부 하위 패키지를 가리키는 것일 수 있습니다. README 전체를 함께 확인해보세요.

pip install .

PyPI에 배포된 패키지를 바로 설치합니다. 소스 클론이 필요 없습니다.

jupyter notebook

브라우저에서 노트북(.ipynb) 파일들을 열람하고 실행할 수 있는 Jupyter 화면을 켭니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# RPSnet for Incremental Learning (NeurIPS'19)
Official Implementation of "Random Path Selection for Incremental Learning" (NeurIPS 2019) [(paper link)](http://papers.nips.cc/paper/9429-random-path-selection-for-continual-learning).

This code provides an implementation for RPSnet : Random Path Selection network for incremental learning (accepted in Nerual Information Processing Systems, Vancouver, 2019). This repository is implemented with pytorch and the scripts are written to run the experiments on multiple GPUs.

## Introduction

<p align="justify">Incremental life-long learning is the main challenge towards the long-standing goal of Artificial General Intelligence. In real-life settings, learning tasks arrive in a sequence and machine learning models must continually learn to increment already acquired knowledge. Existing incremental learning approaches, fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called RPS-Net, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse. Our approach avoids the overhead introduced by computationally expensive evolutionary and reinforcement learning based path selection strategies while achieving considerable performance gains. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity.  Through extensive experiments, we demonstrate that the proposed method surpasses the state-of-the-art performance on incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.</p>

<p align="center"><img src="./utils/rpsnet.png"></p>
<p align="center">(a) RPS network architecture</p>
<br/>
<br/> 
<p align="center"><img src="./utils/path.png"></p>
<p align="center">(b) Random path selection algorithm</p>

## Usage
### step 1 : Install dependencies
```
conda install -c pytorch pytorch
conda install -c pytorch torchvision 
conda install -c conda-forge matplotlib
conda install -c conda-forge pillow
```
### step 2 : Clone the repository
```
git clone https://github.com/brjathu/RPSnet.git
cd RPSnet
```

## Training

The algorithm requires `N` GPUs in parallel for random path selection. The script uses `N=8` with a DGX machine.  
```
./CIFAR100.sh
```

If you are training on single GPU change the `CIFAR100.sh` as well as other running scripts to use only one GPU (comment out the rest).

If you are training on for a specific task, then run,
```
python3 cifar.py TASK_NUM TEST_CASE
```

You can use the pre-trained models on [`CIFAR100`](https://drive.google.com/open?id=18kVpXv28b2nccwu-FzGNSyEb8k76Rdy_) and [`ILSVRC`](https://drive.google.com/open?id=1z0AcrjTTI2Kivs6PoQaB1SNTM_t1iyYI).

To test with several other datasets (ILSVRC, MNSIT etc) run the corresponding files `imagenet.py` or `mnist.py`.

If you need to change the settings (Eg. Number of tasks), run the `python3 prepare/generate_cifar100_labels.py` with corresponding settings.

for ILSVRC, download the pickle file  [meta_ILSVRC.pkl](https://drive.google.com/file/d/1VFxDq6CrAaIeQda_JWVb01erz8BJnlZk/view?usp=sharing) contains file names and labels. Place it inside `prepare` and run `python3 prepare/generate_ILSVRC_labels.py`

## Supported Datasets
 - `CIFAR100`
 - `ILSVRC`
 - `CIFAR10` 
 - `SVHN` 
 - `MNIST`

## Performance

<p align="center"><img src="./utils/10.png" width="500"></p>
<p align="center">(c) Performance on CIFAR100 with 10 classes per task</p>
<br/>
<br/>
<p align="center"><img src="./utils/20.png" width="500"></p>
<p align="center">(d) Performance on CIFAR100 with 20 classes per task</p>
<br/>
<br/>
<p align="center"><img src="./utils/50.png" width="500"></p>
<p align="center">(e) Performance on CIFAR100 with 50 classes per task</p>
<br/>
<br/>
<p align="center"><img src="./utils/imagenet_100.png" width="500"></p>
<p align="center">(f) Performance on Imagenet 100 classes with 10 classes per task</p>

## Reproducing iCaRL results

Although the last classiciation is not same as in iCaRL (uses examplar based nigherest neighbour), by setting `M=1, J=10` we can get almost the same basline results as iCaRL.

## Citation
```
@article{rajasegaran2019random,
  title={Random Path Selection for Incremental Learning},
  author={Rajasegaran, Jathushan and Hayat, Munawar and Khan, Salman and Khan, Fahad Shahbaz and Shao, Ling},
  journal={Advances in Neural Information Processing Systems},
  year={2019}
}
```

## We credit
We have used [this](https://github.com/kimhc6028/pathnet-pytorch) as the pathnet implementation. We thank and credit the contributors of this repository.

## Contact
Jathushan Rajasegaran - jathushan.rajasegaran@inceptioniai.org   or brjathu@gmail.com
<br/>
Discussions, suggestions and questions are welcome!