How US and Chinese open-weight LLMs build on each other, illustrated with a timeline of architecture ideas from 2021 to 2026.
Badges
Badges come from GitHub once the author claims this feed.
Recent posts
every link opens the author’s post-
The US-China Open-Weight Feedback Loop
-
Pre-Training Without Learning Rate Decay
A quick comparison of LLM pre-training recipes using cosine decay, warmup-stable-decay, or constant learning rates with checkpoint averaging.
-
Understanding Positional Embeddings in Modern LLMs
The Methods Behind How LLMs Represent Token Positions
-
From Full to Hybrid Attention in LLMs
How open-weight LLMs are shifting from full attention to hybrid and sparse designs, and what that means for long-context compute and memory.
-
Open October LLM Architecture Roundup
A roundup of ten open-weight LLM architecture releases from October 2026, covering attention mechanisms, hybrid layers, and MoE designs.
-
Building Reasoning Models from the Ground Up: A Full Video Course
Over 11 hours of coding videos on evaluation, inference-time scaling, reinforcement learning, and distillation, with Python and PyTorch.