Projection and non-Gaussianity (figure design v2). (A) A seeded sample of 300 centered GPT-2 Layer-0 token activations, together with 5 related-token activations, in the plane spanned by a seeded random direction $\mathbf{w}$ and the ICA direction $\mathbf{v}$, shown in an arbitrary orthonormal coordinate system $(u_1,u_2)$ after a 60-degree clockwise display rotation. Orthogonal projection maps each activation to a scalar coordinate along a direction. Panels (B) and (C) show projected points along $\mathbf{w}$ and $\mathbf{v}$, respectively; panels (D) and (E) show their full projection distributions. All distributions use the same 50,256 non-special GPT-2 token activations.
