Probe for a concept with a linear classifier

The setup is small: collect hidden states from a chosen layer for a labeled dataset, then fit logistic regression on top. High accuracy means the concept is linearly available at that layer.

Sweep the probe across layers and you can watch where a concept first becomes readable. Abstract properties tend to sharpen in the middle layers, which is a clue about where the model is doing the relevant work.