Ars Memoria: transformers with memory

An original memory-augmented transformer: a separate memory module trained with recall and hippocampus-inspired pattern-separation losses instead of full backprop through time.

Loosely follows the recurrent memory transformer, but splits the model into a language model and a memory encoder. The recurrence is kept cheap: the language model trains with none, while the memory model has a short recurrence that makes memories compose. A VICReg-style pattern-separation loss keeps memories from collapsing.