Sparse autoencoders to untangle neurons

The trick is over-completeness plus sparsity. You map activations into a much wider space but force only a few of those dimensions to be active at once. The pressure pushes tangled concepts apart into separate, more interpretable features.

Once you have the features you can do surgery: amplify one and watch the model fixate on a bridge, suppress another and watch a behavior fade. It is the closest thing we have to named knobs inside the network.