simclr

(★ 4,501)

SimCLRv2 - Big Self-Supervised Models are Strong Semi-Supervised Learners

Owner Repos

bert ★ 40,047

TensorFlow code and pre-trained models for BERT

Python
google-research ★ 38,592

Google Research

Jupyter Notebook
timesfm ★ 28,071

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

Python
vision_transformer ★ 12,675

No description provided.

Jupyter Notebook
arxiv-latex-cleaner ★ 7,017

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

Python
text-to-text-transfer-transformer ★ 6,544

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

Python

File Explorer

ZIP Download
  • CONTRIBUTING.md
  • data.py
  • data_util.py
  • lars_optimizer.py
  • LICENSE
  • model.py
  • model_util.py
  • objective.py
  • README.md
  • requirements.txt
  • resnet.py
  • run.py

# Installation Guide

python
Prerequisites

Setup Steps

pip install -r requirements.txt

Install required python package dependencies.

python run.py --train_mode=pretrain --dataset=cifar10 ...

Pretrain the model on a single GPU environment.

Key Commands

pip install -r requirements.txt

Install python library packages required for the project.

python run.py --train_mode=pretrain --train_batch_size=512 --train_epochs=1000 --learning_rate=1.0 --weight_decay=1e-4 --temperature=0.5 --dataset=cifar10 --image_size=32 --eval_split=test --resnet_depth=18 --use_blur=False --color_jitter_strength=0.5 --model_dir=/tmp/simclr_test --use_tpu=False

Pretrain the model using CIFAR-10 or ImageNet datasets.

python run.py --mode=train_then_eval --train_mode=finetune --fine_tune_after_block=4 --zero_init_logits_layer=True --variable_schema='(?!global_step|(?:.*/|^)Momentum|head)' --global_bn=False --optimizer=momentum --learning_rate=0.1 --weight_decay=0.0 --train_epochs=100 --train_batch_size=512 --warmup_epochs=0 --dataset=cifar10 --image_size=32 --eval_split=test --resnet_depth=18 --checkpoint=/tmp/simclr_test --model_dir=/tmp/simclr_test_ft --use_tpu=False

Fine-tune the linear head of the pretrained model.

Large-scale ImageNet training requires TPU setup and corresponding environment variables (TPU_NAME, STORAGE_BUCKET, DATA_DIR, MODEL_DIR).

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Link
Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

pip install

View Details ▼

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

🔍

python

View Details ▼

Python language interpreter.

python

Start a REPL (interactive shell):

python {{path/to/file.py}}

Execute a specific Python file:

python -i {{path/to/file.py}}

Execute a specific Python file and start a REPL:

// repository documentation