Abstract

Existing mechanistic interpretability efforts seem severely threatened by superposition, an effect in which a neural network represents more “features” than it has neurons. Previous papers have used toy models with MLP architectures to study both representational superposition (caused by passing higher dimensional data through a lower dimensional hidden layer) and computation in superposition. Here, for the first time, toy models are used to study how superposition arises in graph neural networks (GNNs). We demonstrate, (i) that superposition in GNNs can arise similarly to MLPs through compression, yet different aggregation functions distinctly impact this phenomenon, with max pooling notably discouraging superposition; (ii) we find that the inherent topology of graphs enables the construction of toy models where superposition arises even in the absence of compression and we discuss the algorithms the model finds to do this; (iii) we identify that graph isomorphism networks (GIN) can lead to the emergence of superposition within a lower-dimensional subspace of a larger embedding, suggesting that superposition inadvertently creates metastable minima; and (iv) we look at how superposition emerges in real life binary classification datasets.