flux2-distributed-inference

(β˜… 13)

Unofficial distributed/sharded Flux2 inference using πŸ€— Accelerate device_map="auto" for multi-GPU model-parallel execution.

File Explorer

ZIP Download
  • inference.py
  • minimal_example.py
  • README.md
  • requirements.txt

# Use via CDN

jsDelivr

jsDelivr serves any public GitHub repository as a CDN with zero setup. Pick a version and a file to get a ready-to-paste link and snippet.

Command Glossary

Commands referenced in this DOCs, explained below.

πŸ”

conda activate

View Details β–Ό

Activate a conda environment.
See also: `conda deactivate`.

conda activate myenv

Activate an existing environment named `myenv`:

conda activate {{path/to/myenv}}

Activate an existing environment located at custom path:

conda activate --stack myenv

Stack `myenv` environment on top of a previous environment making libraries/commands/variables from both accessible:

πŸ”

conda create

View Details β–Ό

Create new conda environments.

conda create {{[-y|--yes]}} {{[-n|--name]}} py39 python=3.9 "numpy>=1.11" scipy

Create a new environment named `py39`, install Python 3.9, NumPy v1.11 or above in it, and the latest stable version of SciPy. Say yes to all confirmations:

conda create {{[-n|--name]}} myenv --file {{file1.yml}} --file {{file2.yml}}

Create a new environment named `myenv` and install packages listed in files:

conda create {{[-p|--prefix]}} {{path/to/myenv}}

Create a new environment at a custom path (i.e. prefix):

πŸ”

git clone

View Details β–Ό

Clone an existing repository.

git clone {{remote_repository_location}} {{path/to/directory}}

Clone an existing repository into a new directory (the default directory is the repository name):

git clone --recursive {{remote_repository_location}}

Clone an existing repository and its submodules:

git clone {{[-n|--no-checkout]}} {{remote_repository_location}}

Clone only the `.git` directory of an existing repository:

πŸ”

huggingface-cli

View Details β–Ό

This command is an alias of `hf`.

tldr hf

View documentation for the original command:

πŸ”

pip install

View Details β–Ό

Install Python packages.

pip install {{package1 package2 ...}}

Install one or more packages:

pip install {{package1 package2 ...}} {{[-U|--upgrade]}}

Upgrade all specified packages to the latest version, installing any that are not already present:

pip install {{package}}=={{version}}

Install a specific version of a package:

πŸ”

python

View Details β–Ό

Python language interpreter.

python

Start a REPL (interactive shell):

python {{path/to/file.py}}

Execute a specific Python file:

python -i {{path/to/file.py}}

Execute a specific Python file and start a REPL:

πŸ”

cls

View Details β–Ό

Clear the screen and reset colors.

CLS

Clear the screen resetting colors to white on black:

// repository documentation