Abstract
Exponential growth in data generation and largescale data science has created an unprecedented need for inexpensive, lowpower, lowlatency, highdensity information storage. This need has motivated significant research into multilevel memory devices that are capable of storing multiple bits of information per device. The memory state of these devices is intrinsically analog. Furthermore, much of the data they will store, along with the subsequent operations on the majority of this data, are all intrinsically analogvalued. Ironically though, in the current storage paradigm, both the devices and data are quantized for use with digital systems and digital errorcorrecting codes. Here, we recast the storage problem as a communication problem. This then allows us to use ideas from analog coding and show, using phase change memory as a prototypical multilevel storage technology, that analogvalued emerging memory devices can achieve higher capacities when paired with analog codes. Further, we show that storing analog signals directly through joint coding can achieve low distortion with reduced coding complexity. Specifically, by jointly optimizing for signal statistics, device statistics, and a distortion metric, we demonstrate that singlesymbol analog codings can perform comparably to digital codings with asymptotically large code lengths. These results show that endtoend analog memory systems have the potential to not only reach higher storage capacities than discrete systems but also to significantly lower coding complexity, leading to faster and more energy efficient data storage.
Introduction
A paradigm shift in the type and quantity of storable information is currently underway. Internetconnected devices are projected to reach 50 billion, more than 6 devices per person, by the year 2020^{1}. Much of the data produced by these devices, such as pixel intensities from cameras, sound recordings from microphones, and timeseries from sensors, comes from signals that are intrinsically analogvalued or manyvalued (>100 values) and ordinal. These signals are also highly redundant and compressible, with a large degree of their joint activity explained by a smaller number of factors than the intrinsic dimensionality of the signal. Furthermore, an increasing portion of the operations performed on the data are analog, with much of it being used for statistical inference or human perception (e.g. objectdetection for images).
A concurrent paradigm shift is underway in the media we use to store data. Early storage media such as phonograph records and VCR tapes relied on perturbing an analogvalued state (wax height and magnetic polarization, respectively). Digital computation led to the popularity of binary storage representations that inhibit noise propagation and utilize the concurrently developed theories of binary errorcorrecting codes^{2}. However, many emerging memory technologies have shifted back to analogvalued media to create multilevel devices that fill the need for inexpensive, highdensity storage. MLCFlash, Phase Change Memory (PCM), Resistive RAM (RRAM), and Conductive Bridge RAM (CBRAM) are all examples of technologies that have an analog state (threshold voltage or resistance) determined by the gatecharge, resistive amorphous capping region, and conducting filament, respectively^{3,4}. One of the goals of many researchers and developers studying emerging memory technologies is to create a “Storage Class Memory”^{5}. This is, essentially, a system of memory that combines the benefits of storage systems like HDDs (e.g. high density, long life) with the benefits of “high performance” systems like DRAM (e.g. fast access time). From this traditional perspective, the emerging memory device we will be considering – PCM – is approaching a Storage Class Memory system in that it has the ability to reach high densities (through 3D integration) with very fast access times (~100 ns).
The aforementioned trends of (i) analog data generation and operation and (ii) analog storage media development motivated us to investigate the use of analog coding in emerging memory systems. We examine here the potential for memory systems that utilize analog representations–analogvalued signals stored in analogvalued devices–to achieve higher capacity and reduced coding complexity, leading to reduced latency and increased energy efficiency.
To create a fair comparison between digital and analog systems, we need to compare their endtoend performance on a storage task. Ratedistortion theory, introduced by Shannon in 1948, provides a common framework to evaluate both approaches, as it allows us to analyze the tradeoff between representation size and performance (i.e., rate and distortion, respectively, described in the following paragraphs)^{6}.
As diagrammed in Fig. 1a, the general coding framework can be set up as follows: a source signal, S^{k}, of dimensionality k, is encoded (F) into a representation, V^{m}, of dimensionality m. The encoded signal is then stored in memory devices, equivalent in the communications literature to passing through a noisy channel, characterized by P(RV), resulting in a noisy recalled signal, R^{m}. A decoder (G) then creates an estimate of the original source, Ŝ^{k}.
The channel noise characteristics, P(RV), uniquely determine the capacity of the channel (Eq. 1). Crucially, for memory storage, the noise distribution is determined by the dynamics of both the device response and of the memory control circuitry. For example, closedloop memory controllers can increase channel capacities through multipulsed readverify schemes^{7}. Such schemes help increase capacity and compensate for devicetodevice variations and temporal drift^{8}, and can be described by information theoretic treatments without loss of generality^{9}. For clarity, we restrict our current investigation to PCM devices controlled with a single readwrite pulse, where V and R directly correspond to the write voltages and read resistances of m devices used to store k symbols of data.
The distortion (d) is determined by comparing the source and its reconstruction with a taskrelevant metric. For example, if mean squared error (MSE) is the relevant metric, then \(d=\frac{1}{N}{\sum }_{i=0}^{N}{({S}_{i}{\hat{S}}_{i})}^{2}\), where N is the total number of examples. For media such as images, perceptual distortion metrics such as multiscale structural similarity index (MSSIM) may be more appropriate^{10}. For a given source, the rate (r, not to be confused with resistance) is given by the compression ratio k/m. For example, in a binary system with rate r, a source signal of k bits can be stored in m bits of memory and reliably recalled with a distortion d.
In contrast to the joint approach shown in Fig. 1a, Fig. 1b shows the traditional, separate approach to coding: encoding and decoding are each divided into separate coding procedures for the source and the channel (F_{src}, F_{ch}, G_{src}, G_{ch}). In this separate coding paradigm, source coding aims to remove statistical redundancy from the data. The resulting compressed representation requires fewer resources to store but is extremely fragile with respect to noise. Consequently, channel coding introduces redundancy, in the form of statistical dependencies, through an errorcorrecting code (ECC), typically with addition of parity bits dependent on data bits. Thus, if the channel code used is a graphical code, this enables the original source coded data to be decoded/inferred from the noisy version through a graphical model^{11}.
Since small errors in a source coded signal can produce large distortions, sufficient redundancy must be added to ensure that no uncorrected errors remain, leaving source coding as the sole source of distortion. The degree of required redundancy (devices/bit) is determined by the inverse of the information capacity (C, bits/device) of the storage devices themselves, which is the maximum rate achievable with an optimal ECC. Thus, the capacity of an analogvalued memory device provides a fair means of comparison between digital and analog implementations. For each symbol produced by a source, a lower bound on r(d) (devices/symbol) for a separatecoded digital system is given by r(d) (bits/symbol) of the source coder divided by the capacity of the memory device (bits/device).
Shannon proved that errorless transmission of a source over a channel is only assured in the limit that the code word length, also known as the blocklength, approaches infinity^{6}. Gastpar^{12} and Rimoldi^{13} shed new light on the problem by showing that the reason why separate codes often require large blocklengths is that they are effectively creating a deterministic channel, whereas a joint approach can keep the mapping random. More specifically, rather than trying to make the channel behavior asymptotically deterministic, joint coding attempts to map the statistics of the source through the stochasticity of the channel such as to produce a low average distortion. Under certain conditions, such mappings can achieve optimal r(d) with finite blocklength. Simple but common examples include the Gaussian source through a Gaussian channel with MSE distortion and the Bernoulli source through a binary symmetric channel with Hamming distance distortion. In both cases, the optimal solution is achieved by performing no coding at all (just simple scaling) and sending each symbol individually (blocklength of one)^{12}. Further, even when the optimal solution is not reached, joint coding can significantly outperform separate coding schemes in the finite blocklength regime^{14}. These examples demonstrate how matching the statistics of the source, channel, and cost can significantly increase the efficiency of reliable communication.
On the flip side, this attempt to create an effectively deterministic channel is why modern binary ECCs with rates that approach the Shannon capacity, such as LDPC codes^{15}, often do so at the expense of large blocklengths (>1kB). While the degree of redundancy remains constant for large blocklengths, the expense of the decoder increases dramatically as it must perform inference on problems of higher and higher dimensionality. Even welldesigned LDPC decoders with large blocklengths can consume significant power (~100 mW–1 W) and area (~10 mm^{2})^{16,17}. As code lengths increase, so do latencies due to decoding, in one case rising from 20 us to 220 us for an increase in BCH (BoseChaudhuriHocquenghem) code length from 16B to 2048B^{18}. While increasing the dimensionality, k, of the signal improves coding performance, it also requires reading from larger segments of memory at once and increases the complexity of the decoder circuitry, resulting in larger access latency and greater energy consumption. Furthermore, these separate digital systems typically lack a robustness to varying channel conditions: the performance saturates if the channel SNR increases beyond the value for which the system was designed and, in the other direction, the system does not “gracefully degrade”, like joint systems can, with decreasing channel SNR^{19,20}.
Again, we can reformulate the problem of storing analog data on an emerging memory device as that of transmitting a discrete time, continuous alphabet source over a discrete time, additive noise channel. When thought of in this manner, we can then look at the general problem of analog coding in the field of communications and see what ideas can be brought to bear on this specific problem. This field has a rich history, with the original treatment of this problem dating back to Shannon^{21} and Kotelnikov^{22}. Their proposed analog coding scheme was based on the use of spacefilling curves. The use of spacefilling curves was then significantly extended in the work of^{23,24,25,26,27,28,29,30}. In these works, spacefilling curves were investigated for the transmission of Gaussian sources over additive white Gaussian noise (AWGN) channels. These ideas, primarily using variants of the “Archimedean spiral”, were further extended by^{31,32,33,34} to the case of different kinds of channels. Following these, a major breakthrough was made in 2014^{35} when it was shown that instead of using parameterized spirals, a functional optimization approach could be used to achieve stateoftheart performance on the problem of Gaussian sources over AWGN channels.
Here, inspired by the general success of optimizationbased methods and utilizing ideas from the analog communications literature to frame the problem, we expand and build on our previous work^{36} and show that using analog codes with analog emerging memory devices can improve system performance over digital codes in both the separate coding and joint coding regimes. Using PCM as an example of analogvalued emerging memory, we demonstrate that separate coding systems can reach a higher rate of information transmission when paired with analog ECCs, as has been demonstrated in the literature for standard communications channels. In the joint coding regime, we show that analog codes can achieve r(d) performance comparable to digital codes, but at a lower complexity cost. Specifically, we examine the simple case of developing optimal mappings for storing analog values directly in the analog resistances of the PCM devices. For symbolbysymbol storage of a Gaussian source with an MSE distortion metric, we find comparable r(d) performance to digital implementations employing optimal capacityachieving channel coding. This surprising result is achieved with a blocklength of one, indicating the potential for even better performance with higher dimensional encodings utilizing the joint statistics of multidimensional sources. This work opens the door to further research into analog coding schemes and shows how concepts from analog coding can be applied to the problem of storing information on emerging memory systems, with the potential to outperform digital approaches. We have seen in previous work^{37,38} that certain analog coding schemes for emerging memory devices can empirically outperform digital ones, and this current work helps to provide an explanation as to why.
Background
To measure the capacity of analogvalued PCM devices, we recast the storage problem as a communication problem (Fig. 2a). The devices are perturbed by voltage pulses of different magnitudes drawn from the input probability distribution P(V). These pulses modulate a device’s resistance, resulting in an output resistance distribution \({\rm{P}}({\rm{R}})=\int {\rm{P}}({\rm{R}}{\rm{V}}){\rm{P}}({\rm{V}}){\rm{dV}}\). The conditional dependence of the read resistance on the write voltage, P(RV), defines the noisy channel through which information must be communicated. In the limit of an infinite blocklength code, the capacity is the number of bits of information per use of the device that can be communicated without error^{6}. It is equivalently given by the maximum mutual information between the input and output distributions,
The capacity achieving input distribution given by Eq. 1 exhibits an optimal tradeoff between having as many input states as possible and having as little overlap in the output distributions as possible. As a simple example, consider a twostate system shown in Fig. 2b, with input voltages V_{0} and V_{1}. If there is no overlap between the output distributions P(RV_{0})P(V_{0}) and P(RV_{1})P(V_{1}) (left panel), the capacity is the log (base two) of the number of input states (1 bit). If there is complete overlap (right panel), the capacity is 0 bits, as it is impossible to infer V from R. For cases of partial overlap, redundancy needs to be added to the signal to correctly infer the inputs, reducing the effective bits/device (center panel). Increasing the number of read states can also increase the capacity for a given number of write states (Fig. 2c). Practically, having more read states than write states gives additional information in the form of greyscale belief as to which input state they belong. This ‘soft information’ is currently used by MLCFlash LDPC decoders to improve inference during belief propagation decoding^{16}. Intuitively, analog representations that use the resistance values directly can have higher capacity because they operate at higher granularity (ultimately limited by circuit noise).
Device operation
Device operation is carried out in the following manner: as diagrammed in Fig. 3a, a PCM material can either be in a conductive crystalline structure (Poly GST, light blue) or a resistive amorphous structure (αGST “cap”, light purple). The resistance of the PCM cell is determined by the size and structure of the resistive amorphous cap, with larger caps yielding higher resistances. The resistance is set to a high resistance state by a short current pulse (Fig. 3a, top panel, leftmost light blue pulse) that first melts the film through Joule heating. The short pulse then quickly quenches the material, causing it to form a resistive amorphous cap. The resistance can be decreased by slowly passing a lower current pulse (Fig. 3a top panel, second pulse from the left, light red) that heats the material above its crystallization temperature, annealing the amorphous cap, allowing some of the crystalline structure to slowly reform while cooling. Different resistance levels can then be achieved by applying different magnitudes of current pulses (e.g. the two different light red pulses in Fig. 3a, top panel), resulting in different volumes of resistive amorphous and conducting crystalline phases^{3}. In general, closed form mathematical models do not do a good job of describing the dynamics of PCM devices. To describe the dynamics of these devices requires highly detailed finiteelement modeling (e.g. COMSOL^{39}), and even this is not very accurate in most circumstances because there are intricate interactions between the electric field, current, and temperaturedependent conductivity of the GST that all must be simulated selfconsistently^{40,41}. For these reasons, we construct a numerical model of these devices for our study based on empirical measurements. We describe the data collection and model in the following paragraphs.
For this study, we performed pulsed resistance measurements of seven different devices (Fig. 3b, circuit diagram for one such device) on a 100device PCM array. Details of fabrication and characterization of these arrays can be found in previous reports^{42,43,44}. Each cell is equipped with its own access transistor (Fig. 3b) to prevent crosstalk and sneak paths in the array. Current flowing through the cell is controlled by the gate ‘word line’ voltage (V_{WL}), which we pulse to control the shape and magnitude of current pulses. To ensure independence between pulses, we first ‘RESET’ the cell to a high resistance state with a short melting pulse. We then apply a longer variablemagnitude ‘Partial SET’ pulse to decrease the resistance of the cell. This facilitates analysis by constraining device behavior to a memoryless zerothorder Markov process. For example, let us say that at t = 1 second the value of V_{WL} = 1.0 Volts was applied across the device. This will result in a resistance that is drawn from the conditional distribution P(RV_{WL} = 1.0). Now because of the way the device is being pulsed, this conditional distribution will be the same regardless of what the previous resistance value was. I.e. if the process had memory, P(RV_{WL} = 1.0) could be different depending on what R at t = 0 seconds was. But since the process does not have memory, P(RV_{WL} = 1.0) will always be the same [Note: It should be noted that a digital system would need to apply a similar pulse scheme if it was to achieve greater than 1 bit of information transmission across the channel.]
Results
Capacity
As mentioned earlier, devicetodevice variation and resistance drift will have a dramatic effect on the capacity of PCM arrays. The channel noise characteristics are determined by both the memory controller and memory devices, and indeed multipulse readverify control schemes have proven effective at achieving high capacities despite devicetodevice variation and resistance drift^{7,8}. As a compromise between simplicity and realism, we ignore drift in our current treatment and approximate a simple memory controller to measure the responses of seven different devices on this 100device array.
For each of the seven different devices, we first collect 120 trials at each voltage level. With seven devices, this corresponds to 840 points at each voltage level. For this study, we used 100 voltage levels. To then calculate a continuous probability density for P(RV_{WL}), we first performed Gaussian kernel density estimation (KDE) of the distribution of resistances resulting from each voltage level. That is, for each of the 100 voltage levels, v, a Gaussian KDE of P(RV_{WL} = v) was made from the 840 measured points. To estimate P(RV_{WL} = v) for v’s that were not measured, we simply interpolate between the two closest P(RV_{WL} = v) that we have measurements for.
Plotted in Fig. 4a are one standard deviation above and below the mean for each of the seven devices measured (interpolating between the 100 measured v’s to get the smooth estimates shown). As can be seen, the different devices exhibit qualitatively similar behavior with slightly offset RESET resistances and slopes of annealing. We approximate the very simple singlepulse memory controller, mentioned above, by normalizing the RESET resistances between devices through measuring the relative resistance (to the RESET resistance) rather than the absolute resistance, which can be accomplished with two resistance reads and a divide operation.
Another way of thinking about this with Fig. 4a in mind is that we’re simply applying a vertical shift to each of the curves so that their left most points align, yielding the curves seen in Fig. 4b. Now, in this new space, the relative resistance value of a point is the logarithmic ratio between R_{Reset} and the measured resistance, i.e. log_{10}(R_{RESET}/R) (yaxis, Fig. 4b). Thus, all the points to the farthest left are centered around 0.0, by construction (as this is log_{10}(R_{Reset}/R_{Reset})), while all the points that are, for example, 10 times smaller than R_{Reset} are at 1.0.
The effect of this normalization on the actual measured resistances is that it brings all of the points for each voltage closer together. Thus, the seven curves described previously from Fig. 4a now lie on top of each other (Fig. 4b) as the seven sets of 120 points that yielded each curve are now much closer together. Finally, when the Gaussian KDE (with interpolation) is done on this new, renormalized set of points, the P(RV_{WL}) [Note: More precisely, this is P(log_{10}(R_{RESET}/R)V_{WL}), but for brevity and clarity we’ll refer to log_{10}(R_{RESET}/R) simply as R in the text.] shown in Fig. 4c results [Note: Raw data and the code to reproduce the parameters of this Gaussian KDE can be found at https://github.com/rzarcone/Analog_Coding_Final].
From this empirically obtained P(RV_{WL}) we then solve for the capacityachieving input distribution, P_{cap}(V_{WL}), using the BlahutArimoto algorithm^{2}. The effect of the simple singlepulse memory controller is that increases the capacity of the channel from 1.54 bits to 2.08 bits (for reference, the average capacity for any individual device is ~2.5 bits). While more realistic arrays would contain more variation and drift, they would typically also be paired with more powerful controllers, regardless of the coding scheme (either analog or digital). Thus, as they would affect both schemes, we will not factor them into our central comparison of analog vs. digital coding for emerging memory devices. Furthermore, opportunities exist to jointly learn the behavior of the controller with the encoder and decoder – e.g. with a neural network – which is in fact an avenue we have explored in another paper^{37}.
As the BlahutArimoto algorithm technically only applies to discrete distributions, we use a discrete approximation to the channel, finely discretizing the P(RV_{WL}) that we calculated as discussed above. Our criterion for sufficient discretization is that the capacity no longer increases with the addition of more states (e.g. >1000 states). As the goal is to optimize the tradeoff between the number of output states and these states’ overlap (Fig. 5a), the number of nonzero values in P_{cap}(V_{WL}) eventually saturates. That is, beyond a certain number of discrete input states, the addition of more states does not increase the mutual information (if the full analog output range is used). For this channel, saturation is reached at 13 states [Note: Nonuniform input state probabilities like those here are difficult to achieve for many applications, but setting them equal only decreases the channel capacity by ~5%]. Note that the output distributions for these states partially overlap and cover the full range of R (Fig. 5a, bottom panel), meaning that higher capacity can only be achieved if the full analog output range is used (as opposed to the traditional discrete case where there is no overlap). Put another way, any decoding algorithm that is to achieve the channel capacity must perform inference over the entire analog range of R.
As mentioned previously, the capacity is the maximum possible rate of information transmission over a given channel. To gain a better understanding of the effect of discretization on the rate of information transmission, we examine the case where there are a finite number of read and write states. This analysis is done to illustrate and make quantitative, for the case of the device we’re considering, the idea of “soft information” presented in Fig. 2. We model this discretization behavior by marginalizing the continuous probability P(RV_{WL}) between discrete read levels, creating a reduced discrete channel. We then use simulated annealing to search for optimal values of the read levels to maximize the rate of information transmission. Figure 5b demonstrates how the capacity of this reduced channel increases with the number of read levels for a given number of write levels. In each case, the capacity monotonically asymptotes as the number of read levels increases, corresponding to the case of an analog read circuit. As the number of allowed input levels increases above 13, the optimal number for the full analog channel, the discrete capacity asymptotes to the analog capacity of 2.08 bits. Put another way, the maximum possible rate of information transmission is only possible in the limit that the full analog range for the output is used. As analog systems inherently operate over the full continuous range of values, it is possible for them to achieve the capacity. This use of the full continuous output range can be thought of as a necessary condition to achieve the maximum possible rate of information transmission over the channel. Of course, developing the specific code that achieves this maximum information transmission is an unsolved problem. I.e. the capacity tells you what the upper bound is, but not how to achieve it. This discretization analysis shows that it is necessary to consider the full continuous range of outputs to achieve this bound, but it too does not show you how to achieve it. Thus, if the goal is to perform separate source and channel coding, say, to allow the channel coder to work with data from a wider range of sources, then analog channel coding schemes have the ability to outperform digital ones.
Joint coding
Finally, we explore an approach to directly store analogvalued information in the resistance values of the PCM devices. While many interesting opportunities exist for exploiting the rich statistics of natural signals, for this preliminary study, we restrain ourselves to the canonical case of a single Gaussian source signal, P(S), with an MSE distortion metric. The most naïve approach to store analog values would be to employ no coding and directly map source values, S, into voltages and directly provide reconstructions, Ŝ, from resistances. Such a naïve approach leads to an “effective channel” (conditional probability distribution of reconstruction, P(ŜS)) identical to the original channel, P(RV_{WL}). While this is the ideal approach for a Gaussian source with Gaussian channel and MSE distortion^{12}, the nonlinear PCM device is far from a Gaussian channel. Correspondingly, the naïve implementation exhibits a low SNR (< 0 dB) at a rate of 1 device/symbol.
Given that we are working with a Gaussian source and MSE distortion, we would like a parameterized model to learn an encoding and decoding function that minimizes the average cost. We start with a parameterized model
where F and G are nonlinear mappings corresponding to the encoder and decoder, respectively. We then solve numerically for the mappings that minimize the distortion,
The encoding and decoding functions are parameterized by a sum of unnormalized Gaussians. Specifically, F(.) and G(.) are described by
Here, the Gaussian centers, {x_{i}}, as well as their widths, \(\sigma \), are fixed, and the only variables to be adapted are the weights of each Gaussian, {\({y}_{{\alpha }_{i}}\)} [Note: see appendix for details on the fixed parameters]. Ultimately, two different \({y}_{\alpha }\) are learned, one for the encoder and one for the decoder. The {\({y}_{{\alpha }_{i}}\)} for each of these are learned by minimizing Eq. (4) via gradient descent. A sum of Gaussians is chosen because they are smooth (with wellbehaved derivatives for the gradientbased optimization), are easily parameterizable, have support over the whole region considered, and are highly nonlinear and thus, when combined, able to approximate a large class of functions.
The results of this minimization are presented in Fig. 6. Here, the Gaussian distributed variable, S (blue distribution, top of Fig. 6a), is passed through the learned encoding function F (red line, Fig. 6a). That is, each sample from P(S) is mapped, deterministically, through F into a specific voltage, V_{WL}. For example, a sample of S = 0.0 is mapped to approximately 0.9 V. This mapping transforms the original Gaussian distribution into a highly nonGaussian distribution (red distribution, Fig. 6a right and Fig. 6b, top). Interestingly, and encouragingly, this distribution is qualitatively similar to P_{cap}(V_{WL}) (Fig. 5a). For an easier comparison between the two we have taken the resulting P(V_{WL}) and overlaid P_{cap}(V_{WL}) (Fig. 7a).[Note: It is difficult to make a quantitative comparison here as the capacity achieving source distribution is over a finite number of elements whereas this distribution has compact support in different sections.] It is also interesting that the parameterization for the encoding function, the sum of Gaussians, was chosen for its flexibility in function approximation, and not handdesigned to try and approach P_{cap}(V_{WL}). Thus, the fact that algorithm was able to learn a mapping that produced a distribution that approaches P_{cap}(V_{WL}) is support for the algorithm’s generality.
The encoded value then gets stochastically mapped through the channel. That is, each sample voltage results in a draw from the conditional distribution given by P(RV_{WL}). For example, V_{WL}(S = 0.0) results in a draw from P(RV_{WL}=0.9). The distribution of resistances that results from this mapping is again highly nonGaussian (green distribution Fig. 6b, right and Fig. 6c, top) [Note: Note that the encoder learns to map a large amount of mass in the region of the channel where inputoutput pairs are easiest to distinguish (red portion of histogram, where the noise is smallest) and none in the “indistinguishable” region of the channel (region with positive slope, approximately 1.2–1.6 V).] Finally, the readout resistance is passed through the learned decoding function, G (green line, Fig. 6c). The resulting distribution for Ŝ is very close to the original Gaussian (with the target distribution for S given by the dashed line). For an easier comparison between these distributions we have also taken the original P(S) and overlaid P(Ŝ) (Fig. 7b).
To quantitatively assess the performance of this approach, we can draw many samples from P(S) and check what the corresponding Ŝ is. This creates an ‘effective channel’, P(ŜS) (Fig. 6d). This analysis shows that samples from P(S) are mapped through the channel and recovered by Ŝ with a small amount of scatter around the straight dashed line (corresponding to Ŝ = S), yielding an SNR of 9.4 dB. Note how the region between [−1,1], where the majority of S’s mass is, is where the encoder and decoder learn to do the best job of transmitting the signal over the channel. This is a consequence of the MSE distortion metric: the encoder and decoder are penalized the most in this region and thus devote their resources to performing good reconstructions here.
As another point of comparison, we can look at a quantity from the analog coding literature: the Optimal Performance Theoretically Attainable (OPTA). This quantity is obtained by equating the channel capacity to the rate distortion function of the source considered, and then solving for the resulting minimum achievable distortion (usually in decibels). Thus, for the Gaussian source and the channel being considered we have:
This implies that the OPTA is 12.5 dB (3.1 dB from our method). While this is a useful quantity to consider as it provides an upper bound to the performance, we prefer instead to think about the effective number of bits transmitted across the device (obtainable via a similar manipulation as above, but solving for rate instead of distortion). From this perspective, we think that the most surprising and important result of this work is that even with this very simple encoding/decoding scheme (amounting to a single dimensional lookup table), joint coding with these devices achieves a similar SNR to separate coding schemes of high complexity. Specifically, this joint method performs as well as a separate system that first compresses strings of samples from the Gaussian to 1.56 bits per sample (e.g. through vector quantization) and then uses a hypothetical channel coder capable of getting 1.56 bits of errorless communication across the device. Said channel coder is “hypothetical” in that there does not currently exist a channel coder capable of achieving 1.56 bits across the device. In reality, the design of channel coders for specific channels is a challenging task. For example, though the capacity of the standard AWGN was worked out by Shannon over 70 years ago^{6}, it wasn’t until the 1990s that a class of codes was created that could approach the capacity of these channels^{15}.
Though a channel coder for the separate coding scheme does not exist for these channels, we can assume one does for the purposes of a comparison. We can then further quantify the benefits of our approach by comparing the energy and latency required to perform the above storage task with separate vs. joint coding schemes. For the comparison, an ‘8KB’ PCM array is used, with both separate and joint coding schemes implemented in Samsung 28 nm technology. As discussed above, the separate approach requires the use of digital errorcorrecting codes. These codes require large blocklengths for even modest code rates (e.g., ~0.5). The current state of the art – both in terms of performance and efficiency – are LDPC codes. Thus, we use these as the hypothetical channel coder for the comparison. An LDPC code with good decoding performance typically requires a blocklength of ~2000 at a code rate of ~0.5 (with significantly longer blocklengths for higher rates)^{45}. The decoding step alone for this requires ~44 pJ/sample. For this storage task, the total energy expenditure for a separate scheme with an LDPC code is ~128 pJ/sample. Furthermore, the LDPC decoder requires 10 (or more) iterations to converge. By comparison, a joint coding scheme, achieving the same SNR (i.e. at the same distance from the OPTA, described previously), implemented using a simple lookup table requires only ~40 pJ/sample and a single iteration. The decrease in required energy and latency come mainly from (i) the elimination of the LDPC coder – requiring long blocklengths – and (ii) the reduction in PCM write operations – 2 writes/sample → 1 write/sample – due to the higher coding rate achievable with joint coding in the small blocklength regime (via the intrinsic errorcorrection provided by the learned encoding/decoding functions).
While we have only examined the case of single dimensional encodings here, the design landscape for analog encoders/decoders is rich for higher dimensional signals and other compression ratios, making it a promising area for the development of coding circuitry that is low in energy and latency. Lastly, we should emphasize that because the joint framework we proposed is adaptive, it can learn to store different types of analog data on different types of channels, as diagramed in Fig. 2 (e.g., RRAM, MRAM, or other PCM devices).
Discussion
We have explored a new approach for storing analogvalued media in emerging analogvalued devices showing that analog coding strategies have the potential to create robust storage with relatively low complexity. Designing systems to match the statistics of the source, channel, and taskathand provides unique opportunities not captured by systems that try to enforce determinism. However, there is no free lunch, and what such systems gain in efficiency they likely lose in generality.
Universal memory design has been a great boon to the development of digital technology and designing memory systems for specific tasks has many unexplored risks in terms of backward compatibility. That said, with the rise of machine learning applications, the nature of computation is changing, and many of the datacentric applications of the present and future do not require perfect data retrieval. With the data deluge faced by internet companies, it is often not the storage of data that is important, but the storage of information which performs well when retrieved for human or statistical machine inference (e.g., perception of audio and video). While losing some generality, representations that reduce the redundancy of signals to match the statistics of storage media and applications can be more efficient than universal representations.
It is interesting to note that from the perspective of neuroscience, neural systems are faced with a very similar task. Given the statistical redundancy of natural stimuli and the stochasticity of perception and signal transmission, efficient coding strategies must be devised to transmit compressed representations of stimuli that are intrinsically robust to neural noise and resource efficient^{46,47}.
Finally, the results presented here represent only a single model applied to a single type of signal and device. While we only looked at one type of signal and device for illustrative purposes, the methods we used here are very general and can be applied to a wide range of devices and signals. The future design space to explore in this field is extremely rich.
From the experimental perspective, other devices (RRAM, CBRAM, and MLCFlash) can exhibit dramatically different behavior. Furthermore, various pulsing schemes can produce different types of statistics. For example, iterative pulsing introduces rich conditional dependencies through time that are not currently exploited by multibit memory systems and present an interesting opportunity from the perspective of modeling. Celltocell dependencies within the array are also a fact of life that must usually be avoided through handengineered systems. Measuring these statistical dependencies can enable codes to be adaptive to the flaws and statistics of individual chips, both at the factory during test time and over the lifetime of use, improving the reliability and uniformity of performance.
From an algorithmic perspective, many powerful analogvalued highdimensional models such as artificial neural networks exist, leaving a rich space to explore for learning highdimensional parameterized encodings and compression. Since such models have been able to successfully utilize the statistics of natural media such as images^{48} and sound^{49}, they would seem wellsuited for encoding such media in analogvalued devices. While these neural network methods have been successfully applied to the problem of source coding^{50,51}, there has still yet to be a thorough treatment of the problem for doing both source and channel coding with emerging memory devices, and we think this is an exciting open area for research. Our own initial work in this direction^{37,38} has shown that a joint approach with neural networks can outperform traditional separate approaches when storing images on an array of emerging memory devices. In a related work^{52}, researchers have looked at the problem of using neural networks for source and channel coding with standard channels, such as the Gaussian and Rayleigh channels. As mentioned above, many of the concepts we have discussed here are regularly applied to analog coding problems in the communications literature, typically in the context of communicating over standard channels. Our focus in this work has been on bringing these ideas into the problem of storage with emerging memory devices.
Efficient integrated circuit implementations of such circuits are also an active area of investigation^{53}, lending further weight to the idea of analogvalued ECCs that can learn onchip dependencies and adapt to nonstationary statistics as devices age. Another active area of research is in using analogvalued variable resistors as synaptic elements in such neural circuits^{54,55,56}, leading to the possibility that storage and errorcorrection could someday be integrated within a single memory substrate.
Methods
Device operation
Pulsed resistance experiments were conducted using a Cascade probe station, measuring 1T1R (one transistor per resistance element) PCM arrays with approximately 40 nm Bottom Electrode (BE) contact diameter. Details on device fabrication and characterization of these arrays can be found in^{42,43,44}. Wafers of arrays, provided by IBM, were contacted by a custom probe card and connected to a Keithley 700B Switch Matrix, enabling access to 100 unique devices. Voltage pulses were produced by an Agilent 81110 A Pulse Generator. ‘RESET’ pulses applied 3 V to bitline and wordline with a 5 ns rise time, 60 ns width, and 5 ns fall time. ‘Partial SET’ pulses applied 3 V to the bitline and a variable voltage to the wordline with 500 ns rise time, 2000ns width, and 500 ns tail. Cell resistances were measured with an Agilent 4155 C Semiconductor Parameter Analyzer, applying 3 V to the wordline and 0.1 V to the bitline. All equipment was controlled via custom Python software utilizing PyVisa and wxPython (available at http://www.github.com/jesseengel/PythonProbestation).
Capacity comparison
Estimating P(RV)
Estimates of P(RV) were calculated from experimental data using Gaussian kernel density estimation (KDE) and linear interpolation, implemented using SciPy. Python code for this is available at https://github.com/rzarcone/Analog_Coding_Final.
Calculating capacity
Capacity calculations and ratedistortion simulations were conducted in Python using NumPy. The implementation of the BlahutArimoto algorithm was adapted from MATLAB code by Kenneth Shum (http://home.ie.cuhk.edu.hk/~wkshum/wordpress/p=825). Code for this Python implementation is available at https://github.com/rzarcone/Analog_Coding_Final.
Capacity for finite states
For examining the case of a finite number of read and write states, we employ a basin hoping search strategy, as outlined in^{57}. Python code for this is available at https://github.com/rzarcone/Analog_Coding_Final
Joint coding
Learning encoding/decoding functions
The encoding and decoding functions are parameterized by a sum of unnormalized Gaussians, as described in the main text. The fixed parameters, {x_{i}}, \(\sigma \), and N, are selected via a standard gridsearch method. That is, an initial guess for each of these is chosen. The optimization is then done with these parameters resulting in a given performance level. The parameters are then modified, and the optimization is carried out again. The parameters that result in the best performance are then used in the final encoding/decoding functions. The objective for learning the variable parameters, the weights of these Gaussians, is minimized using Adam^{58}, a version of gradient decent.
In order to be able to take derivatives through the stochastic devices during training, we use a version of the reparameterization trick^{59}. That is, the numerical model described in the Capacity section of the main text is further simplified (only during training, but not during the actual evaluation) to be conditionally Gaussian, where the mean and variance of each Gaussian depends, highly nonlinear, on the voltage applied. Thus,
The functions describing how the mean and variance of the conditional resistance distribution change with V, \(\mu (V)\) and \(\sigma (V)\), respectively, are simply calculated from the measured mean and variance at each of the voltages probed, and then interpolated between these. Raw data and the parameters for the \(\mu (V)\) and \(\sigma (V)\) models can be found at https://github.com/rzarcone/Analog_Coding_Final.
Separate vs joint comparison
The input signal is assumed to be Gaussian distributed and to have 1.56 effective number of bits per sample. Both joint coding and separate coding systems are compared using Samsung 28 nm technology. Specifically, the PCM array is an ‘8KB’ PCM array, with 256 rows x 256 columns, simulated using NVSIM. C++ code for this simulation is available at https://github.com/SEALUCSB/NVSim. In the separate coding system, the input is first converted by a 2bit ADC into digital values (though in a more realistic scenario even higher precision ADC would likely be required). The digital values are then encoded by an LDPC encoder. The encoded signals are then written into the PCM array. In the separate coding scheme, each PCM cell can store 2 bits, and one write pulse is adequate to program the PCM cell into one of four states. Therefore, with a coderate 0.5, each input sample is encoded into 4 bits (two for the signal and two for the required error correction) written over two PCM devices. When retrieving stored information, it is decoded by an LDPC decoder (energy efficiency statistics can be found in Table 2.3 of^{45}) and finally converted back into analog form using a 2bit DAC. This LDPC decoding step typically requires at least 10 iterations to converge. With such an implementation, the energy efficiency is ~128 pJ/sample. The most energyconsuming stage is PCM write (70pJ/sample) – necessitated by the redundant devices required for digital error correction – followed by LDPC decoding (44pJ/sample).
In the joint coding implementation, the input analog signal is first converted by an ADC into digital values. The values then pass through the encoding to determine the voltage to be applied to the PCM. As in the separate case, under the joint coding scheme each PCM cell can store 2 bits. However, in this case, as error correction is built in to the encoding and decoding functions, we do not require redundant, errorcorrecting bits. Therefore, one device is adequate to store one input sample (i.e. code rate = 1). The PCM is then read by an ADC of the same precision as the input ADC. The resulting values are transformed by the decoding function and finally converted back to an estimate of the analog signal by a DAC. Under such scheme, the energy efficiency is highly dependent on the required bitprecision of ADC/DACs. We used the ADC/DAC survey from^{60} and found that the energy per conversion (at 28 nm) scales roughly exponentially with the effective number of bits (Fig. M1). Therefore, the joint coding implementation offers more benefit when the required ADC/DAC precision is low (Fig. M2).
As the input signal has only 1.56 bits per/sample, a 4bit ADC/DAC suffices for the joint coding scheme. If the input samples contained more information, a higher precision ADC/DAC would be required, eventually reaching the point where the separate scheme outperformed the joint. This is an experimental demonstration of the principle laid out by Rahul Sarpeshkar in^{61}, namely that analog computation is efficient at lowprecision, while digital becomes efficient at highprecision. Thus, analog representations are best in this case when the required precision is low to moderate (precisely the kinds of scenarios where the majority of the analog data being produced is used – e.g. human or statistical machine inference).
On a final note, as we are trying to make a comparison between analog and digital strategies, implementing the analog version using ADC/DAC might seem inappropriate. However, this can actually be seen as a sort of easily implementable “intermediate” between fully analog and fully digital, as this scheme is using the analog coding strategy – namely, the functions learned for encoding/decoding – but digitizing it for easy implementation with existing technology. That being said, a faithful analog implementation would perform even better than what we have presented here as it would remove the need for ADC/DAC. Specifically, this analysis could have been done by building custom opamps to truly implement the analog functions. But this would have required extensive work (likely necessitating an additional paper for characterization and construction details) as comparatively less time has been spent on optimizing analog electronics due to the ubiquity of digital hardware.
Change history
04 August 2020
An amendment to this paper has been published and can be accessed via a link at the top of the paper.
References
 1.
Evans, D. The internet of things: how the next evolution of the internet is changing everything. CISCO white paper, 111 (2011).
 2.
MacKay, D. J. C. Information Theory, Inference, and Learning Algorithms. (Cambridge University Press, 2003).
 3.
Wong, H.S. P. et al. Phase change memory. Proc. IEEE 98, 2201–2227 (2010).
 4.
Wong, H.S. P. et al. Metal–oxide rram. Proc. IEEE 100, 1951–1970 (2012).
 5.
Burr, G. et al. Phase change memory technology. Journal of Vacuum Science & Technology B, Nanotechnology and Microelectonics: Materials, Processing, Measurement, and Phenomena 28.2, 223–262 (2010).
 6.
Shannon, C. E. A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948).
 7.
Nirschl, T. et al. Write strategies for 2 and 4bit multilevel phasechange memory. IEEE International Electron Devices Meeting, 461–464 (2007).
 8.
Papandreou, N. et al. Drifttolerant multilevel phasechange memory. IEEE International Memory Workshop, 1–4 (2011).
 9.
LastrasMontano, L. A., Franceschini, M., Mittelholzer, T. & Sharma, M. Rewritable storage channels. International Symposium on Information Theory and Its Applications, 1–6 (2008).
 10.
Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).
 11.
Motwani, R., Kwok, Z. & Nelson, S. Low density parity check (ldpc) codes and the need for stronger ecc. Flash Memory Summit, 4150 (2011).
 12.
Gastpar, M., Rimoldi, B. & Vetterli, M. To code, or not to code: lossy sourcechannel communication revisited. IEEE Trans. Inf. Theory 49, 1147–1158 (2003).
 13.
Rimoldi, B. Beyond the separation principle: a broader approach to sourcechannel coding. International ITG conference on source and channel coding (2002).
 14.
Kostina, V. & Verdu, S. Lossy joint sourcechannel coding in the finite blocklength regime. IEEE Transactions on Information Theory 59.5, 2545–2575 (2013).
 15.
MacKay, D. J. C. & Neal, R. M. Near shannon limit performance of low density parity check codes. Electron. Lett. 32.18, 1645–1646 (1997).
 16.
Kim, J. & Sung, W. Rate0.96 ldpc decoding vlsi for softdecision error correction of nand flash memory. IEEE Trans. Very Large Scale Integr. Syst. 22, 1004–1015 (2014).
 17.
Mansour, M. M. & Shanbhag, N. R. A 640Mb/s 2048bit programmable ldpc decoder chip. IEEE J. SolidState Circuits 41, 684–698 (2006).
 18.
Micheloni, R. et al. A 4Gb 2b/cell nand flash memory with embedded 5b bch ecc for 36mb/s system read throughput. SolidState Circuits Conf. 2006. ISSCC 2006. Dig. Tech. Pap. IEEE Int. 32, 497–506 (2006).
 19.
Mittal, U. & Phamdo, N. Hybrid digital analog (hda) joint sourcechannel codes for broadcasting and robust communications. IEEE Trans. Inf. Theory 48.5, 1082–1102 (2002).
 20.
Skoglund, M., Phamdo, N. & Alajaji, F. Hybrid digitalanalog sourcechannel coding for bandwidth compression/expansion. IEEE Tran. Inf. Theory 52.8, 3757–3763 (2006).
 21.
Shannon, C. E. Communications in the presence of noise. Proceedings of the IRE 37.1, 10–21 (1949).
 22.
Kotel’nikov, V. The Theory of Optimum Noise Immunity. (McGrawHill, 1959)
 23.
Fuldseth, A. & Ramstad, T. Bandwidth compression for continuous amplitude channels based on vector approximation to a continuous subset of the source signal space. IEEE International Conference on Acoustics, Speech, and Signal Processing 4, 3093–3096 (1997).
 24.
Chung, S.Y. On the construction of some capacityapproaching coding schemes. PhD Diss. MIT (2000).
 25.
Ramstad, T. Shannon mappings for robust communication. Telektronikk 98.1, 114–128 (2002).
 26.
Vaishampayan, V., Vinay, A. & Sueli, I. R. C. Curves on a sphere, shiftmap dynamics, and error control for continuous alphabet sources. IEEE Trans. Inf. Theory 49.7, 1658–1672 (2003).
 27.
Floor, P. A. & Ramstad, T. Optimality of dimension expanding shannonkotel’nikov mappings. IEEE Information Theory Workshop, 289294 (2007).
 28.
Hekland, F. On the design and analysis of shannonkotel’nikov mappings for joint sourcechannel coding. PhD Diss. NTNU (2007)
 29.
Hekland, F., Floor, P. A. & Ramstad, T. Shannonkotel’nikov mappings in joint sourcechannel coding. IEEE Trans. Comm. 57.1, 94–105 (2009).
 30.
Hu, Y., GarciaFrias, J. & Lamarca, M. Analog joint sourcechannel coding using nonlinear curves and mmse decoding. IEEE Trans. Comm. 59.11, 3016–3026 (2013).
 31.
Floor, P. A. et al. Zerodelay joint sourcechannel coding for a bivariate gaussian on a gaussian mac. IEEE Trans. Comm. 60.10, 3091–3102 (2012).
 32.
Floor, P. A., Kim, A. N., Ramstad, T. & Balasingham, I. Zero delay joint source channel coding for multivariate gaussian sources over orthogonal gaussian channels. Entropy 15.6, 2129–2161 (2013).
 33.
Hassanin, M. & GarciaFrias, J. Analog joint source channel coding over nonlinear acoustic channels. Conference on Information Sciences and Systems. 1–6 (2013).
 34.
Romero, S. M., Hassanin, M., GarciaFrias, J. & Arce, G. R. Analog joint source channel coding for wireless optical communications and image transmission. Journal of Lightwave Technology 32.9, 1654–1662 (2014).
 35.
Akoyl, E., Viswanatha, K. B., Rose, K. & Ramstad, T. On zerodelay sourcechannel coding. IEEE Trans. Inf. Theory 60.12, 7473–7489 (2014).
 36.
Engel, J. H. et al. Capacity optimization of emerging memory systems: a shannoninspired approach to device characterization. IEEE International Electron Devices Meeting, 29–4 (2014).
 37.
Zheng, X. et al. Errorresilient analog image storage and compression with analogvalued rram arrays: an adaptive joint sourcechannel coding approach. IEEE International Electron Devices Meeting, 3–5 (2018).
 38.
Zarcone, R. et al. Joint sourcechannel coding with neural networks for analog data compression and storage. Data Compression Conference, 147–156 (2018).
 39.
Dickinson, E., Ekstrom, H. & Fontes, E. COMSOL multiphysics: finite element software for electrochemical analysis. a minireview. Eletrochemistry communications 40, 71–74 (2014).
 40.
Lacaita, A. et al. Electrothermal and phasechange dynamics in chalcogenidebased memories. IEEE International Electron Devices Meeting, 911–914 (2004).
 41.
Lelmini, D. & Zhang, Y. Analytical model for subthreshold conduction and threshold switching in chalcogenidebased memory devices. Journal of Applied Physics 102.5, 054517113 (2007).
 42.
Breitwisch, M. et al. Novel lithographyindependent pore phase change memory. IEEE Symposium on VLSI Technology, 100–101 (2007).
 43.
Close, G. F. et al. Device, circuit and systemlevel analysis of noise in multibit phasechange memory. IEEE International Electron Devices Meeting, 29.5.1–29.5.4 (2010).
 44.
Eryilmaz, S. B. et al. Experimental demonstration of arraylevel learning with phase change synaptic devices. IEEE International Electron Devices Meeting 25, 1–4 (2013).
 45.
Ly, T. T. N. Efficient hardware implementations of ldpc decoders, through exploiting impreciseness in messagepassing decoding algorithms. PhD Diss. (2017)
 46.
Karklin, Y. & Simoncelli, E. Efficient coding of natural images with a population of noisy linearnonlinear neurons. Advances in neural information processing systems, 9991007 (2011).
 47.
Varshney, L. R., Sjöström, P. J. & Chklovskii, D. B. Optimal information storage in noisy synapses under resource constraints. Neuron 52, 409–23 (2006).
 48.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 10971105 (2012).
 49.
Hinton, G. et al. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal processing magazine 29.6, 82–97 (2012).
 50.
Toderici, G. et al. Full resolution image compression with recurrent neural networks. IEEE Conference on Computer Vision and Pattern Recognition, 53065314 (2017).
 51.
Ballé, J., Laparra, V., & Simoncelli, E. P. Endtoend optimized image compression. 5th International Conference on Learning Representations (2017).
 52.
Bourtsoulatze, E., Kurka, D. B. & Gündüz, D. Deep joint sourcechannel coding for wireless image transmission. IEEE Transactions on Cognitive Communications and Networking 5.3, 567–579 (2019).
 53.
Neftci, E., Das, S., Pedroni, B., KreutzDelgado, K. & Cauwenberghs, G. Eventdriven contrastive divergence for spiking neuromorphic systems. Front. Neurosci. 7, 272 (2013).
 54.
Kuzum, D., Yu, S. & Wong, H.S. P. Synaptic electronics: materials, devices and applications. Nanotechnology 24, 382001 (2013).
 55.
Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metaloxide memristors. Nature 521, 61–64 (2015).
 56.
Burr, G. W. et al. Experimental demonstration and tolerancing of a largescale neural network (165 000 synapses) using phasechange memory as the synaptic weight element. IEEE Trans. Electron Devices 62.11, 3498–3507 (2015).
 57.
Wales, D. J. & Doye, J. P. K. Global optimization by basin hopping and the lowest energy structures of lennardjones clusters containing up to 110 atoms. J. Phys. Chem. A 101, 5111–5116 (1997).
 58.
Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. 3rd International Conference on Learning Representations (2015).
 59.
Kingma, D. P., & Welling, M. Autoencoding variational bayes. Preprint at, https://arxiv.org/abs/1312.6114 (2013).
 60.
Murmann, B. ADC performance survey. CoRR 1404, 7736 (2014).
 61.
Sarpeshkar, R. Analog versus digital: extrapolating from electronics to neurobiology. Neural computation 10.7, 1601–1638 (1998).
Acknowledgements
This work was supported in part by SONIC, one of six centers of STARnet, a Semiconductor Research Corporation program sponsored by MARCO and DARPA, by member companies of the Stanford SystemX Alliance, and member companies of the Stanford NonVolatile Memory Technology Research Initiative (NMTRI). R.Z. was supported by a National Science Foundation Graduate Research Fellowship under Grant No. DGE 1752814
Author information
Affiliations
Contributions
R.Z. and J.H.E. performed all simulations, conducted the analysis, and wrote the manuscript. J.H.E. performed the experiments. J.H.E., H.S.P.W. and B.A.O. conceived of the project. S.B.E. assisted with experimental setup and design. W.W. provided the numbers for the comparison of the separate vs. joint schemes for the 8KB PCM array at 28 nm. S.K., M.B., C.L. and H.L.L., provided the test wafer of memory devices and helpful discussions.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zarcone, R., Engel, J.H., Burc Eryilmaz, S. et al. Analog Coding in Emerging Memory Systems. Sci Rep 10, 6831 (2020). https://doi.org/10.1038/s4159802063723z
Received:
Accepted:
Published:
Further reading

Chemical order relaxation in a substitutional solid alloy around the critical temperature
Physical Review B (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.