Click here to flash read.
Within the framework of deep learning we demonstrate the emergence of the
singular value decomposition (SVD) of the weight matrix as a tool for
interpretation of neural networks (NN) when combined with the descrambling
transformation--a recently-developed technique for addressing interpretability
in noisy parameter estimation neural networks \cite{amey2021neural}. By
considering the averaging effect of the data passed to the descrambling
minimization problem, we show that descrambling transformations--in the large
data limit--can be expressed in terms of the SVD of the NN weights and the
input autocorrelation matrix. Using this fact, we show that within the class of
noisy parameter estimation problems the SVD may be the structure through which
trained networks encode a signal model. We substantiate our theoretical
findings with empirical evidence from both linear and non-linear signal models.
Our results also illuminate the connections between a mathematical theory of
semantic development \cite{saxe2019mathematical} and neural network
interpretability.
No creative common's license