Expert_Sparsity

(★ 123)

[ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models

Owner Repos

TerDiT ★ 77

TerDiT: Ternary Diffusion Models with Transformers

Python
SPP ★ 22

[ICML 2024] SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models

Jupyter Notebook
OmniInteract ★ 20

OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants

Python
QAS-Bench ★ 12

[ICML 2023] QAS-Bench: Rethinking Quantum Architecture Search and A Benchmark

Python
PhoStream ★ 9

[ICML 2026] PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarios

Python

File Explorer

ZIP Download
  • .gitignore
  • LICENSE
  • main.py
  • README.md

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Link
Example

Command Glossary

Commands referenced in this DOCs, explained below.

🔍

accelerate

View Details ▼

A library that enables the same PyTorch code to be run across any distributed configuration.

accelerate env

Print environment information:

accelerate config

Interactively create a configuration file:

accelerate estimate-memory {{name/model}}

Print the estimated GPU memory cost of running a Hugging Face model with different data types:

🔍

conda activate

View Details ▼

Activate a conda environment.
See also: `conda deactivate`.

conda activate myenv

Activate an existing environment named `myenv`:

conda activate {{path/to/myenv}}

Activate an existing environment located at custom path:

conda activate --stack myenv

Stack `myenv` environment on top of a previous environment making libraries/commands/variables from both accessible:

🔍

conda create

View Details ▼

Create new conda environments.

conda create {{[-y|--yes]}} {{[-n|--name]}} py39 python=3.9 "numpy>=1.11" scipy

Create a new environment named `py39`, install Python 3.9, NumPy v1.11 or above in it, and the latest stable version of SciPy. Say yes to all confirmations:

conda create {{[-n|--name]}} myenv --file {{file1.yml}} --file {{file2.yml}}

Create a new environment named `myenv` and install packages listed in files:

conda create {{[-p|--prefix]}} {{path/to/myenv}}

Create a new environment at a custom path (i.e. prefix):

🔍

conda install

View Details ▼

Install packages into an existing conda environment.

conda install {{package1 package2 ...}}

Install one or more package into the currently active conda environment:

conda install {{[-c|--channel]}} conda-forge {{package}}

Install a single package into the currently active conda environment using channel conda-forge:

conda install {{[-c|--channel]}} conda-forge --override-channels {{package}}

Install a single package into the currently active conda environment using channel conda-forge and ignoring other channels:

🔍

python

View Details ▼

Python language interpreter.

python

Start a REPL (interactive shell):

python {{path/to/file.py}}

Execute a specific Python file:

python -i {{path/to/file.py}}

Execute a specific Python file and start a REPL:

// repository documentation