Projects / Mathematical foundations
Compressibility and Multifractal Properties of Self-Compressing Neural Networks
M.S. thesis (SDSU, 2026). Networks that compress their own internal states train faster and more accurately, and their representations turn out to be multifractal.
My master’s thesis in Applied Mathematics (Dynamical Systems concentration) at San Diego State University, advised by Uduak George.
The idea
A self-model is a representation a system forms of its own internal states. This thesis recasts self-modeling as compression. A network is given an internal autoencoder that compresses one of its hidden layers, and the network is trained to move that layer toward its own compressed reconstruction, so its states become more compressible. A matched null model has the same autoencoder but no feedback into the network.
Results
- Faster, better training. On MNIST, validation accuracy was 97.03% versus 96.46% for the null model, reaching median accuracy 50% faster. On CIFAR-10 it was 66.34% versus 63.73%, 22% faster.
- Why it works. A linear analysis shows the method keeps the original fixed points. Participation ratio and explained variance show the network settling into a compact basis early, which amplifies the known spectral bias of neural networks.
- Multifractal representations. Fixed-mass (Badii–Broggi) estimators match box-counting on iterated function systems with known spectra, and they scale to high dimensions. Applied to network states, which to my knowledge has not been done before, they show multifractal structure. That structure is the same with or without internal compression, which suggests it is set by the data rather than the training method.
This grew out of an earlier, negative-result experiment on self-compressing representations.