Binarized neural networks converge toward algorithmic simplicity: empirical support for the learning-as-compression hypothesis

Understanding and controlling the complexity of neural networks is a central challenge in machine learning, with implications for generalization, optimization, and model capacity. While most approaches rely on entropy-based loss functions and statistical metrics, these measures often fail to capture deeper, causally relevant algorithmic regularities embedded in network structure. We propose a shift toward algorithmic information theory, using binarized neural networks (BNNs) as a first proxy. Gr