ultra-mem

KO | EN

gitlite — search

#ai #llm #go #android #rust #deep-learning #linux #windows #cli #kubernetes #security #ios #mcp #open-source

(★ 28)

Implementation of UltraMem, improved Product Key Memory design, from Bytedance AI labs

ultra-mem 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## UltraMem

Implementation of [UltraMem](https://arxiv.org/abs/2411.12364v1), improved Product Key Memory design, from Bytedance AI labs

## Install

```shell
$ pip install ultra-mem
```

## Usage

```python
import torch
from ultra_mem import UltraMem

ultra_mem = UltraMem(
    dim = 512,
    core_heads = 2,
    topk = 32,
)

tokens = torch.randn(1, 1024, 512)

out, mem_indices, aux_loss = ultra_mem(tokens) # (1, 1024, 512), (2, 1, 1024, 32), ()
```

## Char-level LM

```shell
$ uv run train.py
```

## Citations

```bibtex
@misc{huang2025ultrasparsememorynetwork,
    title   = {Ultra-Sparse Memory Network}, 
    author  = {Zihao Huang and Qiyang Min and Hongzhi Huang and Defa Zhu and Yutao Zeng and Ran Guo and Xun Zhou},
    year    = {2025},
    eprint  = {2411.12364},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url     = {https://arxiv.org/abs/2411.12364}, 
}
```

```bibtex
@inproceedings{Lin2025ContinualLV,
    title    = {Continual Learning via Sparse Memory Finetuning},
    author   = {Jessy Lin and Luke S. Zettlemoyer and Gargi Ghosh and Wen-tau Yih and Aram H. Markosyan and Vincent-Pierre Berges and Barlas Ouguz},
    year     = {2025},
    url      = {https://api.semanticscholar.org/CorpusID:282203348},
    blog_url = {https://jessylin.com/2025/10/20/continual-learning/}
}
```

ultra-mem

ultra-mem 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

ultra-mem

ultra-mem 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README