BertViz for watching attention move

BertViz draws the attention pattern as lines connecting tokens, split by head and layer. It runs in a notebook in a couple of lines and works on most Hugging Face transformer models.

The value is not the pretty picture, it is the habit of looking. Once you have seen a head that exclusively tracks matching brackets, you stop thinking of the model as a black box and start thinking of it as a stack of small, legible mechanisms.