Code for "Learning to Read Out: Unembedding Dynamics in Language Model Pretraining" — trajectory crosscoders on the unembedding matrix W_U across pretraining checkpoints. Trained dictionaries: hf.co/hematteo/parameter-trajectory-crosscoders
language-models pythia sparse-autoencoders interpretability mechanistic-interpretability crosscoders pretraining-dynamics
-
Updated
Sep 3, 2026 - Python