Hunting My Hand-Crafted Induction Head in Real Models
Last time I hand-crafted an induction head, and I ended on a question I could not answer from inside the toy: if I go looking in a real trained model, will I find the circuit I just built?