A generative model of memory construction and consolidation

Episodic memories are (re)constructed, share neural substrates with imagination, combine unique features with schema-based predictions and show schema-based distortions that increase with consolidation. Here we present a computational model in which hippocampal replay (from an autoassociative network) trains generative models (variational autoencoders) to (re)create sensory experiences from latent variable representations in entorhinal, medial prefrontal and anterolateral temporal cortices via the hippocampal formation. Simulations show effects of memory age and hippocampal lesions in agreement with previous models, but also provide mechanisms for semantic memory, imagination, episodic future thinking, relational inference and schema-based distortions including boundary extension. The model explains how unique sensory and predictable conceptual elements of memories are stored and reconstructed by efficiently combining both hippocampal and neocortical systems, optimizing the use of limited hippocampal storage for new and unusual information. Overall, we believe hippocampal replay training generative models provides a comprehensive account of memory construction, imagination and consolidation.

Figure 1 shows results for the 18 remaining Deese-Roediger-McDermott task word lists not shown in Figure 7.As in the human data, lure words are often but not always recalled when the model is presented with 'id n'.The model also forgets some words, and produces additional semantic intrusions.See Methods for further details.
Figure 2 shows that latent representations support few-shot learning better than intermediate representations extracted from the encoder or the 'sensory' image features.Decoding accuracy is measured by training a support vector machine to classify the central object's shape, varying the input features and the amount of data, and evaluating the resulting model on a held-out test set.The intermediate features tested are the outputs of four convolutional layers in the encoder, flattened to one-dimensional vectors.

A.2.1 Variational autoencoders
The generative networks used in the model are variational autoencoders.An autoencoder is a neural network which encodes an input into a shorter vector, and then decodes this compressed representation back to the original.It learns by minimising the difference between the inputs and outputs.There is no guarantee that decoding an arbitrary compressed representation produces a sensible output, so standard autoencoders do not perform well as generative models.In other words, there are many regions in the vector space of the compressed representations which do not correspond to anything meaningful.However, one can train an autoencoder with special properties, such that each latent variable is normally distributed for a given input, which allow one to sample realistic items.The result is called a variational autoencoder. 2,3 atent variables can be thought of as hidden factors behind the observed data, and directions in the latent space can correspond to meaningful transformations -see Figure 8b for an example from Hou et al. 4 The VAEs in these simulations use convolutional layers to better encode and decode image features.Convolutional layers learn sliding windows that scan the image for a relevant feature, outputting a stack of feature maps. 5Applying such a layer to the output of a preceding convolutional layer has the effect of finding higher-level features in the stacked feature maps, i.e. if the first convolutional layer learns to identify simple features such as lines at different orientations, the second convolutional layer might learn features consisting of combinations of lines.
A large VAE was used for the Shapes3D dataset (containing RGB images of size 64x64 pixels), and a small VAE was used for the MNIST dataset (containing greyscale images of size 28x28 pixels).In the large model's encoder, four convolutional layers gradually decrease the width and height of the representation and increase the depth (as is standard when using convolutional neural networks to encode images), followed by a pooling layer and dense layers to represent the mean and log variance of the latent representation.In addition, a dropout layer immediately after the input layer is added to improve the denoising abilities of the model. 6In the decoder, four convolutional layers alternate with up-sampling layers to increase the width and height of the representation and decrease the depth.The smaller VAE used for the MNIST simulations has a latent dimension of 20,  The classifier is a simple support vector machine as in Figure 2a.and a reduced architecture with fewer convolutional layers for efficiency (specifically, there are two convolutional layers in the encoder and two transposed convolutional layers in the decoder).
The following list describes the sequence of operations within the large VAE's encoder network, using the layer names from the TensorFlow Keras API 7 (see also Figure 3): 1. Input layer for arrays of shape (n, 64, 64, 3), representing n 64x64 RGB images 2. Dropout layer with a dropout rate of 0.2 (during training, dropout randomly sets a fraction of the input units to 0 at each step, reducing overfitting and encouraging robustness)  only a temporary store is required until consolidation occurs.In addition, they frequently recall incorrect memories, as the energy function can get 'stuck' in a local minimum.
However, recent research has shown that the storage capacity of a Hopfield network can be increased in several ways.Krotov and Hopfield 10 devise a new energy function involving a polynomial function, and a corresponding update rule to minimise this; the activation of a node flips from -1 to 1 or vice versa if the energy is lower in the flipped state.Ref. 11 develops this idea further, increasing the capacity from approximately 0.14d to 2 d/2 with the use of an exponential energy function.Ramsauer et al. 9 extend this to memories involving continuous variables and further amend the energy function, enabling the recall of much more complex data.(For example, whilst classical Hopfield networks can only recall black and white images, the modern variant can recall greyscale ones.) However, understanding these new variants of Hopfield networks in terms of neural networks is less straightforward.To recap, Equation (1) gives the energy of a standard Hopfield Network. 10 During recall, a node's value is updated to the sign of the weighted sum of its inputs; in other words, a node's value is flipped if it decreases the energy.The matrix T gives the weights of the network, and the calculation of T is simply Hebbian learning.(In these equations, σ gives the state of the network as a vector, and ξ gives a stored pattern.) gives the energy of a dense Hopfield network. 12In this example F(x) is x 3 , but it can be any polynomial function.As above, at recall time a node's value flips if it decreases the energy.When F(x) is x 2 , Equation (2) reduces to Equation (1) for a standard Hopfield network.In any other case, the tensor T has more than two indices, and can no longer be thought of a matrix produced by Hebbian learning.This means the energy is no longer a function of weights and activations in a neural network.Modern Hopfield networks 12 suffer from the same problem. ( Krotov and Hopfield 12 suggest a way to overcome this problem by using hidden units (which they call 'memory units') in addition to the 'feature units' which represent the input.As a result, a modern Hopfield network can be understood as a neural network, like its predecessor.The authors provide two equations for the evolution of the feature neurons and hidden neurons over time.Rather than using discrete time steps as in a classical Hopfield network, time is modelled as continuous.
They therefore give a pair of differential equations, in which change to each set of currents is driven by the weighted sum of currents in the other layer.They then define an energy function, chosen 'so that the energy function decreases on the dynamical trajectory'.The energy function has three terms: energy in the feature neurons, energy in the hidden neurons, and energy from the interaction between the two groups.Importantly, the interaction term can be described in terms of two-body synapses, so once again the energy is a function of weights and activations in a neural network.
The authors state that 'the memory patterns . . .can be interpreted as the strengths of the synapses connecting feature and memory neurons'.To understand the intuition behind this, suppose we set the weights connecting a particular hidden node with the feature neurons to the values of the pattern to be memorised.Then activating the hidden node results in the pattern being reinstated in the feature neurons.In other words, each hidden node represents a memory, and each memory could be encoded using Hebbian learning.The key point is that the energy does not require a matrix of stored patterns, unlike in earlier formulations of modern Hopfield networks -the patterns are encoded in the weights, and the energy is a function of weights and activations as explained above.
Krotov and Hopfield 12 show that under different circumstances, their formulation can be simplified to dense associative memory, 10 or modern Hopfield networks. 9Having established that modern Hopfield networks increase memory performance and are biologically plausible (in the sense that they involve only 'two-body synapses', and that memories can be stored as weights), we use them to model the initial learning in the hippocampus.
An important question is how the memories get encoded as the weights of a bipartite graph in the ref. 12 formulation of a modern Hopfield network.Each memory is bound together by a single node, which connects the features that comprise that memory.The weights between a given memory node and the feature nodes are simply the values of the features for that memory; these weights can be learned by Hebbian learning.Therefore encoding in a modern Hopfield network is similar to previous models of the hippocampus as 'indexing', or binding together, a set of memory components. 13The innovative aspect of modern Hopfield networks is the update rule, which is cleverly designed to guarantee the desired properties.The equation below gives the new state pattern ξ new in terms of the previous state ξ, stored patterns X T , and inverse temperature β: (3) ξ new = Xsof tmax(βX T ξ) In modern Hopfield networks, the inverse temperature parameter β determines whether individual attractors or metastable states (superpositions of stored attractors) are retrieved.In our simulations we set β very high in order to ensure that only individual 'memories' are recalled.
It should be noted that the modern Hopfield network could be swapped out for other computational models of associative memory, providing they i) are high capacity, ii) can retrieve memories from noise, and iii) are capable of one-shot memorisation.

Figure 1 :
Figure 1: Additional results for the Deese-Roediger-McDermott task.In the extended model, gist-based semantic intrusions arise as a consequence of learning the co-occurrence statistics of words.First the VAE is trained to reconstruct simple stories 1 converted to vectors of word counts, representing background knowledge.The system then encodes the lists as the combination of an 'id n' term capturing unique spatiotemporal context, and the VAE's latent representation of the word list.In each plot, recalled stimuli when the system is presented with 'id n' are shown, with output scores treated as probabilities so that words with a score of above 0.5 are recalled.Words from the stimulus list are shown in blue, and lures in red.

Figure 2 :
Figure 2: Latent representations support few-shot category learning.The accuracy of an object shape classifier on a held-out test set is shown for different amounts of training data, with different layers of the VAE as input features.The classifier is a simple support vector machine as in Figure2a.

Figure 3 :
Figure 3: Additional model details.a) Variational autoencoder architecture.Trainable layers (plus the input, output, and sampled latent vector) are shown in boxes, along with the dimensions of their outputs, and nontrainable operations such as activation functions, batch normalisation, and upsampling are shown as annotations.See the SI for more details.b) Figure adapted from Hou et al. 4 with permission, showing the effect of adding and subtracting a proportion α of various different vectors in the latent space of their VAE.(Diagrams were created using BioRender.com.) 9owever one issue is their limited capacity; a Hopfield network can only recall approximately 0.14d states, where d is the dimension of the input data.9Ittherefore seems unlikely that classical Hopfield networks are a good model of hippocampal memory encoding -even if we assume that 3. Conv2D layer with 32 filters (i.e.convolutional windows, or feature detectors) and kernel size of 4 (i.e.windows of 4x4 pixels)4.Batch normalisation layer (batch normalisation is a common technique which computes the mean and variance of each feature in a mini-batch and uses them to normalise the activations) 5. LeakyReLU activation layer (LeakyReLU is an activation function that is a variant of the