halonet-pytorch - GitLite

KO | EN

gitlite — search

#javascript #python #react #machine-learning #docker #nodejs #kubernetes #awesome-list #pytorch #security #ios #mcp #openai #open-source

halonet-pytorch

(★ 199)

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

File Explorer

Download Latest Version (.zip)

- - python-publish.yml
- __init__.py
- halonet_pytorch.py
.gitignore
halonet.png
LICENSE
README.md
setup.py

# Installation Guide

1. Get the code

git clone https://github.com/lucidrains/halonet-pytorch

Downloads the entire project code from GitHub to your computer.

cd halonet-pytorch

Moves into the project folder you just downloaded.

2. Python

Easy Recommended

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

$ pip install halonet-pytorch

Installs the Python libraries listed in requirements.txt (or similar).

✅ If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<img src="./halonet.png" width="500px"></img>

## HaloNet - Pytorch

Implementation of the Attention layer from the paper, <a href="https://arxiv.org/abs/2103.12731">Scaling Local Self-Attention For Parameter Efficient Visual Backbones</a>. This repository will only house the attention layer and not much more.

## Install

```bash
$ pip install halonet-pytorch
```

## Usage

```python
import torch
from halonet_pytorch import HaloAttention

attn = HaloAttention(
    dim = 512,         # dimension of feature map
    block_size = 8,    # neighborhood block size (feature map must be divisible by this)
    halo_size = 4,     # halo size (block receptive field)
    dim_head = 64,     # dimension of each head
    heads = 4          # number of attention heads
).cuda()

fmap = torch.randn(1, 512, 32, 32).cuda()
attn(fmap) # (1, 512, 32, 32)
```

## Citations

```bibtex
@misc{vaswani2021scaling,
    title   = {Scaling Local Self-Attention For Parameter Efficient Visual Backbones}, 
    author  = {Ashish Vaswani and Prajit Ramachandran and Aravind Srinivas and Niki Parmar and Blake Hechtman and Jonathon Shlens},
    year    = {2021},
    eprint  = {2103.12731},
    archivePrefix = {arXiv},
    primaryClass = {cs.CV}
}
```