Self-compressing representations

An original experiment: does pushing a latent code toward its own compressed reconstruction disentangle it? (It doesn't; it becomes more compressible and more entangled.)

Benchmarked against FactorVAE. Entanglement is measured as total correlation, estimated with histograms over vector-quantized codes. This is a negative result, reported as one.

The idea of self-compression carried on into my thesis, where it turned out to be a useful training regularizer: Compressibility and Multifractal Properties of Self-Compressing Neural Networks.