H2Q-MicroStream

KO | EN

gitlite — search

#javascript #hacktoberfest #llm #golang #linux #docker #nodejs #awesome #cli #vue #awesome-list #security #mcp #openai

H2Q-MicroStream

(★ 10)

Intelligence is not about memorizing history, but mastering the dynamics of the future.

H2Q-MicroStream Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 🌌 H2Q-MicroStream: The Hamiltonian Thinking Kernel

> **"Intelligence is not about memorizing history, but mastering the dynamics of the future."**
>
> **"智能不是记忆过去的所有细节，而是掌握生成未来的核心方程。"**

[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://raw.githubusercontent.com/makai891124-prog/H2Q-MicroStream/main/LICENSE)
[![PyTorch](https://img.shields.io/badge/PyTorch-2.0%2B-orange)](https://pytorch.org/)
[![Status](https://img.shields.io/badge/Status-Experimental-red)](https://github.com/)

## 📖 Introduction / 项目简介

**H2Q-MicroStream** is a paradigm-shifting experiment in **Physics-Informed AI**. Unlike traditional Transformers that rely on massive parameters and infinite context windows, H2Q constructs a minimalist "Thinking Kernel" based on **Hamiltonian Dynamics** and **Quaternion Algebra**.

This project proves that with a strict **Rank-8 constraint** and **Unicode-level streaming**, a model can emerge with logical reasoning and grammatical capabilities within a mere **0.2GB VRAM** footprint.

**H2Q-MicroStream** 是一个基于**物理动力学**的 AI 范式实验。不同于依赖堆砌参数和超长上下文的主流 Transformer，H2Q 基于**哈密顿动力学**和**四元数代数**构建了一个极简的“思维内核”。本项目证明了在严格的 **Rank-8** 约束和 **Unicode 流式读取**下，智能可以在仅 **0.2GB 显存** 的微小空间内涌现。

---

## 🚀 Key Features / 核心特性

### 1. Rank-8 Essentialism (Rank-8 本质主义)
*   **The Concept**: We enforce a strict rank limit (Rank=8) on the generative weights. This forces the model to abandon rote memorization and extract only the most fundamental laws of language evolution.
*   **The Result**: A tiny **13MB** checkpoint that captures the syntax and logic of the English language.
*   **概念**：强制权重矩阵的秩为 8。这逼迫模型放弃死记硬背，只能提取语言演化中最本质的规律。
*   **结果**：一个仅 **13MB** 的权重文件，却掌握了英语的语法和逻辑。

### 2. Hamiltonian & Quaternion Core (哈密顿与四元数核心)
*   Implements a **balanced Hamiltonian layer** that preserves energy and structural symmetry.
*   Uses **Quaternion Attention** to model semantic relationships as phase rotations in high-dimensional space.
*   实现了能量守恒的**哈密顿层**，并利用**四元数注意力**将语义关系建模为高维空间中的相位旋转。

### 3. Rolling Horizon Validation (轮动视界验证)
*   **Mechanism**: `Train[T] -> Valid[T+1] -> T becomes T+1`.
*   We validate the model on the *immediate future* (next chunk) before training on it. This strictly measures the model's ability to extrapolate logic, not just interpolate data.
*   **机制**：用“未来”的数据验证“现在”的模型，然后再学习“未来”。这是对逻辑推演能力的终极测试。

### 4. Unicode Stream (Unicode 流式读取)
*   No Tokenizer. No vocabulary bias. The model reads raw bytes (0-255), treating language as a pure physical signal stream.
*   无分词器。无词表偏见。模型直接读取原始字节流，将语言视为纯粹的物理信号。

---

## 📊 Performance / 实验结果

Tested on **NVIDIA RTX 4070 Ti** with **TinyStories** dataset.

*   **Convergence**: Loss dropped from `2.88` to **`1.02`** (near Shannon Entropy limit for simple English).
*   **Generalization**: Achieved **Negative Diff** (Validation Loss < Training Loss), proving true understanding of the underlying rules.
*   **Efficiency**:
    *   VRAM Usage: **~0.2 GB**
    *   Throughput: **~10,000 tokens/s**

---

## 🧭 Latest Progress / 最新进展（2026-05-22）

### UNGS 四批工程化进展

1. 训练侧已完成 UNGS core 接入与损失融合（`CE + axiom_ortho + ungs_loss`），并支持在线结构遥测。
2. 验收侧已扩展 Gate E（UNGS 涌现门），可直接从主遥测与 core 遥测判定结构涌现。
3. 自治策略已从固定阈值升级为 `axiom_residual` 驱动，并形成训练期超参回写闭环。
4. 控制器已升级为 warmup + curriculum 双阶段，并新增 val 恶化保护回调：
   - `control_phase/control_phase_scale`
   - `val_worse_streak/val_protection_active/val_protection_triggered`

### 中程 A/B 实证（小模型快测配置）

本次使用同配置、同 seed 的 control vs UNGS 真实运行，汇总见 [midrange_ab_20_50_report.json](midrange_ab_20_50_report.json)。

*   **20 chunks 终点**
    *   control `val_loss=2.877205`
    *   UNGS `val_loss=2.402915`
    *   UNGS 相对收益：`+0.474290`
*   **50 chunks 终点**
    *   control `val_loss=3.307245`
    *   UNGS `val_loss=2.185197`
    *   UNGS 相对收益：`+1.122048`

结论：在 20 → 50 chunks 区间，UNGS 的泛化优势持续扩大，并非短程偶然波动。

---

## 🛠️ Usage / 使用方法

### 1. Install Dependencies / 安装依赖
```bash
pip install -r requirements.txt
```

### 2. Run Training / 启动训练

The script automatically downloads the TinyStories dataset and starts the "Rolling Horizon" training loop.
脚本会自动下载数据集并开启“轮动视界”训练循环。

```
python train.py
```

### 3. Monitor / 监控

The terminal displays a real-time "ICU Dashboard":
终端将显示实时的“ICU 级仪表盘”：

```
Chunk 18 | Train: 1.0420 | Val: 1.0622 | Energy: 68.5 | Speed: 311ms
```

------

## 🔬 Prime Sieve Experiments / 素数筛实验

Prime-sieve related C++ experiments are now organized under [experiments/prime_sieve/README.md](experiments/prime_sieve/README.md).

素数筛相关的 C++ 实验现已整理到 [experiments/prime_sieve/README.md](experiments/prime_sieve/README.md)，包含：

*   `6` 轮位图筛 MVP
*   多算法对照基准
*   动态轮基扩展实验报告

这部分内容独立于主训练流程，适合后续单独复现、扩展和撰写论文附录。

## 🔮 Vision / 愿景

We are moving from **"Statistical Correlation"** to **"Dynamical Causality"**.
H2Q is not just a language model; it is a **digital lifeform** attempting to resonate with the mathematical structure of the universe.

我们正在从**“统计相关性”**迈向**“动力学因果律”**。
H2Q 不仅仅是一个语言模型，它是一个试图与宇宙数学结构发生共振的**数字生命**。

------

H2Q-MicroStream

H2Q-MicroStream Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

H2Q-MicroStream

H2Q-MicroStream Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README