Click here to flash read.
Lateral predictive coding is a recurrent neural network which creates
energy-efficient internal representations by exploiting statistical regularity
in sensory inputs. Here we investigate the trade-off between information
robustness and energy in a linear model of lateral predictive coding
analytically and by numerical minimization of a free energy. We observe several
phase transitions in the synaptic weight matrix, especially a continuous
transition which breaks reciprocity and permutation symmetry and builds cyclic
dominance and a discontinuous transition with the associated sudden emergence
of tight balance between excitatory and inhibitory interactions. The optimal
network follows an ideal-gas law in an extended temperature range and saturates
the efficiency upper-bound of energy utilization. These results bring
theoretical insights on the emergence and evolution of complex internal models
in predictive processing systems.
No creative common's license