Alternatives to backpropagation
Hebbian learning, predictive coding, forward-forward, PEPITA, reservoirs and evolutionary methods, benchmarked against backprop on sample efficiency and wall-clock time.
Projects
A curated list of the projects from my GitHub. I've mostly stopped posting since 2025 because I have been using AI tools so extensively since then. Even if I'm the architect it doesn't quite feel right for a personal project GitHub. Since then, I've been working on similar things: alternatives to backpropagation, JEPA-style learning, and intrinsically motivated RL.
I've had Claude write the descriptions for the projects below, this gives more neutral descriptions than I can!
- original
- My own idea or architecture
- implementation
- A paper, built from scratch
- tool
- A reusable library
- tutorial
- Written to teach (including teach myself)
Learning without backprop
Hebbian rules, predictive coding, forward-only and evolutionary learning, benchmarked fairly.
Memory & the hippocampus
Grid cells, attractor networks, and associative memory that learns online.
Hippocampus models
A continuous-attractor grid-cell simulator and Vector-HaSH associative memory, extended with online Hebbian learning and metaplasticity tuned by evolutionary search.
Ars Memoria: transformers with memory
An original memory-augmented transformer: a separate memory module trained with recall and hippocampus-inspired pattern-separation losses instead of full backprop through time.
Self-supervised world models
Joint-embedding predictive architectures, from images to agents that learn skills.
JEPA & Saccade JEPA
I-JEPA built from the paper before the official release, trained on ImageNet with one GPU, plus Saccade JEPA: an original variant that learns by predicting small eye movements.
Fire: artificial agency
Experiments in agents that learn on their own: streaming deep RL, unsupervised skill discovery (METRA, CSF), cognitive map learners, and TD-JEPA.
Dynamical systems & time
Koopman operators, HiPPO memories, and forecasting chaotic systems.
Khresmoi: forecasting dynamical systems
Neural Koopman operators, HiPPO memories, synthetic gradients and symbolic dynamics, tested on synthetic oscillators and chaotic systems.
Global workspace test
An original experiment: do modules predicting a coupled chaotic system do better when they share a compressed "workspace"?
Generation & pattern formation
Audio codecs, self-organizing codebooks, diffusion, and growth models from biology.
Neural audio codec experiments
A SoundStream/EnCodec-style VQ-VAE with original additions: wavelet upsampling layers, self-organizing codebooks, and an energy-transformer bottleneck that outperformed RVQ.
quantization-maps
A pip-installable library for vector quantization with modern and differentiable self-organizing maps.
energy_transformer
A PyTorch port of the Energy Transformer, a transformer that runs attention alongside a Hopfield network and descends an energy function.
Morphogenesis
Leaf-venation growth by space colonization in 2D and 3D, grown inside arbitrary meshes and explored in a dashboard.
Baby diffusion
A minimal, heavily annotated diffusion model, written to understand diffusion and shared as a teaching resource.
Neural file sorter
Sorts image collections by meaning using CLIP or a custom autoencoder, with an interactive 3D embedding explorer.
Mathematical foundations
Compression, fractal geometry and information geometry of learning systems.
Compressibility and Multifractal Properties of Self-Compressing Neural Networks
M.S. thesis (SDSU, 2026). Networks that compress their own internal states train faster and more accurately, and their representations turn out to be multifractal.
Information geometry, from scratch
A JAX tutorial that builds differential geometry from primitives (metrics, connections, curvature, parallel transport) up to Fisher metrics and α-connections.
Self-compressing representations
An original experiment: does pushing a latent code toward its own compressed reconstruction disentangle it? (It doesn't; it becomes more compressible and more entangled.)