Electronic Implementation of Associative Memory Based on Neural Network Models

From the Publisher: Neural Nets and Chaotic Carriers Peter Whittle University of Cambridge Neural Nets and Chaotic Carriers is an innovatory text, in that it develops rational principles for the design of associative memories with a view to applying these principles to models with the irregularly oscillatory operation so evident in biological neural systems. It thus bridges studies of artificial and of biological neural networks, with new results for both. The text has a strong research character, but a concise exposition from the basics makes it accessible to non-specialists. Design is based on the criterion that an associative memory must be able to cope with 'fading data', i.e. to form an inference from data even as its memory of that data degrades. The resultant net shows striking biological parallels, suggesting testable anatomical predictions. Many questions concerning composite or 'spurious' traces and memory capacity are clarified. The approach taken to models of the biological neuron and oscillation in systems of such neurons follows the pioneering ideas of W.J. Freeman, and develops these. In particular, when the associative memory principles are combined with oscillatory operation, some remarkable effects emerge. For example, the system shows a low-frequency square-wave oscillation (the 'escapement oscillation') with gamma-range bursts at its peaks, much as is observed in electroencephalograms. The text will be invaluable for researchers and graduate workers with a primary interest in artificial of biological neural nets. However, it is also accessible and interesting to anyone with the mathematical background usual in artificial intelligence, computer science, systems studies or statistics,for example.

Lattice associative memories also known as morphological associative memories are fully connected feedforward neural networks with no hidden layers, whose computation at each node is carried out with lattice algebra operations. These networks are a relatively recent development in the field of associative memories that has proven to be an alternative way to work with sets of pattern pairs for which the storage and retrieval stages use minimax algebra. Different associative memory models have been proposed to cope with the problem of pattern recall under input degradations, such as occlusions or random noise, where input patterns can be composed of binary or real valued entries. In comparison to these and other artificial neural network memories, lattice algebra based memories display better performance for storage and recall capability; however, the computational techniques devised to achieve that purpose require additional processing or provide partial success when inputs are presented with undetermined noise levels. Robust retrieval capability of an associative memory model is usually expressed by a high percentage of perfect recalls from non-perfect input. The procedure described here uses noise masking defined by simple lattice operations together with appropriate metrics, such as the normalized mean squared error or signal to noise ratio, to boost the recall performance of either the min or max lattice auto-associative memories. Using a single lattice associative memory, illustrative examples are given that demonstrate the enhanced retrieval of correct gray-scale image associations from inputs corrupted with random noise.

Binary matrices under the microscope: A tomographical problem

In this note we show how a binary memory can be used to recall gray-level patterns. We take as example the � β associative memories recently proposed in Yanez, Associative Memories based on order Relations and Binary Operators(In Spanish), PhD Thesis, Center for computing Research, February of 2002, only useful in the binary case. Basically, the idea consists on that given a set of gray-level patterns to be first memorized: (1) Decompose them into their corresponding binary patterns, and (2) Build the corresponding binary associative memory (one memory for each binary layer) with each training pattern set (by layers). A given pattern or a distorted version of it, it is recalled in three steps: (1) Decomposition of the pattern by layers into its binary patterns, (2) Recalling of each one of its binary components, layer by layer also, and (3) Reconstruction of the pattern from the binary patterns already recalled in step 2. The proposed methodology operates at two phases: training and recalling. Conditions for perfect recall of a pattern either from the fundamental set or from a distorted version of one them are also given. Experiments where the efficiency of the proposal is tested are also given.

Application of neural networks to predict the transient performance of a Run-Around Membrane Energy Exchanger for yearly non-stop operation

An associative memory is a special type of artificial neural network that has the purpose of store input patterns with their corresponding output patterns and efficiently recall a pattern from a noise-distorted version. Presented in this article is a new framework for constructing a binary associative memory model based on two new autoinverse operations called extended XOR/XNOR; these new operations are generated from the XOR/XNOR operations, respectively. Two types of associative memory are generated with this model: the max type (XOR-AM max), which is constructed with the maximum of the extended XOR operation, and the min type (XOR-AM min), which is constructed with the minimum of the extended XNOR operation. The XOR-AM max exhibits tolerance against the presence of patterns distorted by dilative noise, whereas the XOR-AM min exhibits tolerance against the presence of patterns distorted by erosive noise; both types of memory converge in a single step, use the same extended XOR/XNOR operator for learning and recalling phases, operate in heteroassociative and autoassociative modes, and show infinite storage capacity for the autoassociative mode. Finally, computer simulation results are presented for the new memories based on the extended XOR/XNOR (XOR-AM), which have better or equal performance compared to other associative memories. For the experiments with mixed noise, the conditions established by the kernel method proposed by Ritter for Morphological Associative Memories were conserved, and the solution algorithm proposed by Hattori for the construction of the kernel patterns of these memories was modified.

A robustness analysis is conducted for a large class of discrete-time recurrent neural networks for associative memories under perturbations of system parameters. The present paper aims to provide an answer to the following question: given a discrete-time neural network with specified stable memories (specified asymptotically stable equilibria), under what conditions will a perturbed model of the discrete-time neural network possess stable memories that are close (in distance) to the stable memories of the unperturbed discrete-time neural network model? Robustness stability results for the perturbed discrete-time neural network model are established and conditions are obtained for the existence of asymptotically stable equilibria of the perturbed discrete-time neural network model which are near the asymptotically stable equilibria of the original unperturbed neural network. In the present results, quantitative estimates (explicit estimates of bounds) are established for the distance between the corresponding equilibrium points of the unperturbed and perturbed discrete-time neural network models considered herein.

A conventional associative memory employs correlation as a similarity measure in storage and retrieval operations. We had previously described a variant of this linear associative memory that contains a controllable, adaptive nonlinearity in the correlation domain to suppress cross-correlation noise and increase the memory capacity.1 Work by Maxwell et al.2 and Psaltis and Park3 utilized higher-order correlations to achieve the same effect. A comparative study between the use of isolated higher-order correlation terms and an adaptive nonlinearity will be carried out via computer simulations. Particular emphasis will be placed on the memory capacity and error-tolerance characteristics of the nonlinear associative memory models using different forms of nonlinearities.

A unipolar binary bidirectional associative memory (UBBAM) has been designed to associate input and output vectors in a manner of a content addressable memory. The neural network thus formed does not require inhibitory neurons and is well suited for optoelectronic implementa tion. The performance of the UBBAM has been shown to match that of the original BAM. The key device in the implementation of the UBBAM is a Smart Advanced Spatial Light Modulator (SASLM1), a ferroelectric liquid crystal (FELC) SLM driven by a silicon CMOS backplane. The smart pixel of the SASLM serves as a local processing unit which sets the state of its FELC modula tor according to some function of the optical signals incident upon the pixel's photodetector. The ac tion of each smart pixel is hence akin to a neuron's soma, and a matrix of such smart pixels therefore forms a powerful parallel processing hardware, significant in representing neuron fields, in real time, with minimal off-chip control and processing. The UBBAM model, the SASLM1 smart pixels and a proposed optical system will be described in this paper.

A significant bottleneck to the use of associative memories in real-time systems is the amount of data that requires processing. The aim of this paper is to present the work that produced a dedicated hardware design that will run a major part of the ADAM algorithm. The work selected a portion of the algorithm to implement using FPGA technology. The Sum And Threshold (SAT) processor has been simulated and shown to process data fifty times faster than the current DSP based system that uses a dedicated peripheral processor. The paper includes analysis of the speed characteristics of the SAT processor, along with an analysis of the design. The design analysis highlights a bottleneck in the design to which a solution is proposed for future work.

Aging impairs visual associative memories. Up to date, little is known about whether aging impairs auditory associative memories. Using the head-related-transfer function to induce perceived spatial locations of auditory phonemes, this study used an audiospatial paired-associates-learning (PAL) paradigm to assess the auditory associative memory for phoneme-location pairs in both younger and older adults. Both aging groups completed the PAL task with various levels of difficulty, which were defined by the number of items to be remembered. The results showed that compared with younger participants' performance, older participants passed fewer stages and had lower capacity of auditory associative memory. For maintaining a single audiospatial pair, no significant behavioral differences between the two aging grous werefound. However, when multiple sound-location pairs were required to be remembered, older adults made more errors and demonstrated a lower working memory capacity than younger adults. Our study indicates aging impairs audiospatial associative learning and memory.

Niigata UniversityFaculty of Engineering8050 Ikarashi 2Niigata-shi 950-21JapanE-mail: osami@eng.niigata-u.ac.jpAbstract. A method is proposed in which handwritten characters arerecognized using feature extraction and an associative memory matrix.In feature extraction, simple processes such as shifting and superimpos-ing patterns are executed. A memory matrix is generated with singularvalue decomposition and by modifying small singular values. Themethod is optically implemented with two liquid crystal displays. Experi-mental results for the recognition of 26 handwritten alphabet charactersclearly show the effectiveness of the method.

The neural-network-like scheme of data storage and processing of N-bit sequences of information needs a memory of about N2 elements (interconnections). In digital auto-associative memories /1/ the useful data is presented usually as a set of S different words, v( s )(s= 1,…, S), each word being a sequence of N bits. Simple mathematical rule /2/, given originally by Hopfield /3/, can serve as an algorithm to calculate the values of the N2 elements of the memory matrix, T. Recall of the memory gives an output word, v(out), which results from a thresholded inner product between the interrogating input word, v( in ) and the memory matrix.

Memristor is a nonlinear resistor with memory, which has the characteristics of neuron synapses and can be used to design a new generation of memristive neural networks. Based on the Pavlov associative memory, a novel memristive associative memory neural network model is designed by using the charge-controlled nanoscale HP memristor model as the electronic synapse. This model includes neurons, memristor synapses, and weighted input feedback learning rule. Based on the proposed memristive associative memory model, a memristor-based neural network structure for simulating human emotions is further designed. The emotion simulation takes into account the excitation and inhibition between different neurons, making it more bionic. In order to simulate the memristive neural network structure, a relatively simplified emotional simulation circuit is constructed, which effectively reduces the network complexity and circuit power consumption. Finally, PSPICE is used to simulate the circuit. The simulation results not only verify the correctness of the model but also achieve a simple simulation of human emotions, which is helpful for the further development of the artificial neural network in the field of artificial intelligence.

Multimodal emotion recognition can more accurately understand human emotional expressions by fusing multi-source information such as text, audio, and video. Addressing the deficiencies of existing methods in modal alignment, context modeling, and cross-modal interaction, this paper proposes a multimodal deep fusion framework based on extended Long Short-Term Memory networks (xLSTM). The framework introduces exponential gating mechanisms and novel memory structures, including sLSTM with scalar memory and mLSTM with matrix memory, effectively enhancing the model's memory capacity and temporal modeling capabilities. The proposed method integrates differential convolution for dynamic video feature extraction and hierarchical attention mechanisms for adaptive cross-modal fusion.