Taalas-Style On-Chip Weights on a $250 FPGA: a Language Model at 60k tok/s
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260.
Badges come from GitHub once the author claims this feed.
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260.