bert

(★ 40,047)

TensorFlow code and pre-trained models for BERT

Reference Links

🌐 Official Homepage `BERT-Large, Uncased (Whole Word Masking)` TensorFlow Hub Colab https://arxiv.org/abs/1810.04805 SQuAD v1.1

Owner Repos

google-research ★ 38,592

Google Research

Jupyter Notebook
timesfm ★ 28,071

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

Python
vision_transformer ★ 12,675

No description provided.

Jupyter Notebook
arxiv-latex-cleaner ★ 7,017

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

Python
text-to-text-transfer-transformer ★ 6,544

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

Python
simclr ★ 4,501

SimCLRv2 - Big Self-Supervised Models are Strong Semi-Supervised Learners

Jupyter Notebook

File Explorer

ZIP Download
  • .gitignore
  • __init__.py
  • CONTRIBUTING.md
  • create_pretraining_data.py
  • extract_features.py
  • LICENSE
  • modeling.py
  • modeling_test.py
  • multilingual.md
  • optimization.py
  • optimization_test.py
  • predicting_movie_reviews_with_bert_on_tf_hub.ipynb
  • README.md
  • requirements.txt
  • run_classifier.py
  • run_classifier_with_tfhub.py
  • run_pretraining.py
  • run_squad.py
  • sample_text.txt
  • tokenization.py
  • tokenization_test.py

# Installation Guide

python
Prerequisites

Setup Steps

pip install -r requirements.txt

Install project dependencies.

python create_pretraining_data.py

Generate pre-training data.

python run_pretraining.py

Run pre-training for BERT models.

python run_classifier.py

Fine-tune and run the classifier model.

Key Commands

pip install -r requirements.txt

Install python package dependencies specified in requirements.txt.

python run_classifier.py --task_name=SST-2 --do_train=true --do_eval=true --data_dir=$GLUE_DIR/SST-2 --vocab_file=$BERT_BASE_DIR/vocab.txt --bert_config_file=$BERT_BASE_DIR/bert_config.json --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt --max_seq_length=128 --train_batch_size=32 --learning_rate=2e-5 --num_train_epochs=3.0 --output_dir=/tmp/sst2_output

Fine-tune and evaluate the BERT model for classification tasks such as GLUE.

python run_squad.py --vocab_file=$BERT_BASE_DIR/vocab.txt --bert_config_file=$BERT_BASE_DIR/bert_config.json --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt --do_train=true --train_file=$SQUAD_DIR/train-v1.1.json --do_predict=true --predict_file=$SQUAD_DIR/dev-v1.1.json --train_batch_size=12 --learning_rate=3e-5 --num_train_epochs=2.0 --max_seq_length=384 --doc_stride=128 --output_dir=/tmp/squad_output

Train and evaluate the model using SQuAD 1.1 or 2.0 question answering datasets.

python extract_features.py --input_file=sample_text.txt --output_file=output.jsonl --vocab_file=$BERT_BASE_DIR/vocab.txt --bert_config_file=$BERT_BASE_DIR/bert_config.json --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt --layers=-1,-2 --max_seq_length=128 --batch_size=8

Extract high-dimensional vector embeddings from a pre-trained BERT model.

Pre-trained BERT model checkpoints and dataset paths must be prepared according to your environment.

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Link
Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

john

View Details ▼

Password cracker.

john {{path/to/hashes.txt}}

Crack password hashes:

john --show {{path/to/hashes.txt}}

Show passwords cracked:

john --show --users={{user_ids}} {{path/to/hashes1.txt path/to/hashes2.txt ...}}

Display users' cracked passwords by user identifier from multiple files:

🔍

python

View Details ▼

Python language interpreter.

python

Start a REPL (interactive shell):

python {{path/to/file.py}}

Execute a specific Python file:

python -i {{path/to/file.py}}

Execute a specific Python file and start a REPL:

// repository documentation