Dieu, Patrick-Olivier: Empirical Decoupling of Representation Similarity and Transformation Stability in Neural Networks

Representation similarity metrics such as CKA are widely used to compare neural networks by quantifying geometric alignment in representation space. These measures are often interpreted as proxies for functional similarity across models, architectures, and training conditions. In this work, we investigate whether representation similarity is predictive of transformation-dependent stability in neural networks. We introduce a stability functional S(M), defined as the expected sensitivity of a mode