A hybrid learning network for shift, orientation, and scaling invariant pattern recognition

The objective of this work is to identify an object out of a given set of objects. Shift invariance is automatically achieved by employing spatial filters in the Fourier transform plane. To relax the strict alignment and stability requirements of the holographic filter, we use bipolar amplitude filters with minimum information content that is still sufficient for the required pattern recognition. For complete rotation invariance one may use filters with rotation symmetry, while for scale invariance, filters that are independent on the radial coordinate may be employed. Partial invariances combined in a single filter are also investigated. Theoretical considerations augmented with computer simulation are used for the determination of acceptable recognition criteria and for the synthesis of practical filters. The feasibility of the method was demonstrated by computer simulations and some preliminary experiments.

Ridge polynomial networks (RPNs) are a special class of high order neural networks, which are computationally much more efficient than traditional high order neural networks. Motivated by the ability of high order NNs to perform shift and rotation invariant pattern recognition tasks, we apply RPNs for function approximation and character recognition. We are using features from the image block representation of the characters and traditional invariant moments. The results are quite favoring and show that RPNs can be used effectively in pattern recognition problems.

For the task of position-, scale-, and rotation-invariant pattern recognition higher order neural networks have shown good separation results for different object classes. However, their use is limited by the large number of monomials that have to be calculated to get the invariant features. Even with the application of methods like coarse coding to reduce the number of monomials the computational requirements are still large. Here a new method is proposed where the averaging of the monomials to form the invariant features is performed adaptively with a network architecture that uses coupled nodes. Thereby the features are adapted to a specific application. This reduces significantly the number and order of the monomials necessary for the invariant recognition of patterns. Moreover, the classification part of the network becomes superfluous since the output of the network is already one-dimensional and binary. The properties of the new method in comparison to higher order neural networks are shown for position and 90-degree-rotation invariance and for invariance with respect to different object backgrounds. The application to rotations of smaller angles is outlined.

The investigation of nonlinear processes in brain systems based on the oscillatory-wave approach is currently one of the most consequential domains of researching signal generation and processing mechanisms in the brain. These investigations may offer insights into various phenomena, such as the mechanism of associative memory, based on network models of biologically relevant neurons. A neural network architecture is proposed in this paper to solve applied problems, such as signal filtering, processing, and recognition of information images. Mechanisms for incomplete fragments were implemented, enabling the extraction and complete restoration of objects from memory. Each neuron in this neural network comprises a Hodgkin–Huxley biophysical model with a Mainen modification, the dynamics of which most plausibly reproduce the processes in the neural cells of the brain. In the research on individual neurons, a constant external current with varying amplitudes was applied. The study determined the average periods of self-oscillations and identified parameter values for the Andronov–Hopf bifurcation. In particular, a stable limit cycle is created and destroyed depending on different scenarios when the bias current is increased or decreased. A stable limit cycle arises via the Andronov–Hopf bifurcation and is terminated via the saddle-node bifurcation on the cycle. The study identified parameter regions whereby two neurons with excitatory and inhibitory synaptic connections synchronize. The neurons were all in self-oscillating mode and exhibited a stable limit cycle in the phase space. A three-layer neural network was constructed, comprising of a reference neuron, sensory, control, and interneurons. The control and intermediate layer connections are arranged based on the Hebbian learning rule. Areas of excitation for various types of neurons were found. Experiments were conducted using the neural network architecture to distinguish binary information patterns encoded through signal phase. Excitatory and inhibitory synaptic connections were employed to encode the images stored in memory. The dynamic patterns are determined through the in-phase or anti-phase mode of phase locking, i.e., phase synchronization relative to the global rhythm. Before reaching the output layer of neurons, the signals are summed on the interneuron layer with the signal of the reference neuron, resulting in the filtering out of a certain information segment (in-phase or anti-phase). The network accurately recognized the patterns after parameter adjustments were made. Areas of neuron phase synchronization were discovered, enabling control of network activity modes. The confirmation of the viability of phase capture and pattern recognition modes using Hodgkin–Huxley–Mainen neurons was achieved. In total, as a result of the implementation of the spike neural network modeling project: 1) two-parameter diagrams of regions of neuronal excitation in various modes are calculated; 2) the effect of excitatory and inhibitory coupling in synaptic currents is used to represent the input signal; 3) an abstract mathematical algorithm for calculating the Hebbian matrix has been transferred to the implementation of synaptic currents between layers of neurons; 4) the effect of phase clusters is used to represent the output pattern; 5) the neurons’ topology in the visual-cerebral department served as a working model for recognizing graphic images through the Hopfield network on a spike neural network, using the Hebbian rule.

A new learning algorithm is proposed for the training of single layer linear networks. The network studied has an input layer of N units and an output layer of M units. The input and output layer are fully connected via an M/spl times/N weight matrix. It is well known that such a network of linear processing units will generate M principal components of the input distribution when it is trained by Hebbian type learning algorithms, such as general Hebbian algorithm (GHA). It is also known that the same network structure of winner-take-all (WTA) units will produce M cluster centres of the input space when it is trained by gradient based competitive learning, such as Kohonen learning. The new algorithm also uses a Hebbian type learning mechanism, but unlike the previous algorithms such as GHA, it simultaneously classifies the input distribution into M subclasses and extracts the principal component of each subclass distribution. To achieve robust performance, a frequency sensitive competitive learning mechanism is incorporated into the process, hence the new algorithm is called frequency sensitive Hebbian learning (FSHL). We have applied the new algorithm to image data compression applications and simulation results are presented which indicate that the new FSHL will consistently outperform the optimal Karhunen-Loeve transform (KLT) and is competitive to Kohonen networks.

Pattern recognition by optical spatial filtering procedures is discussed using general considerations with the objective of reducing the information content in the spatial filter. The achievement of this objective is very useful toward the wide application of spatial light modulators and also for facilitating distortion invariant recognition. The proposed novel approach is demonstrated by an example employing bipolar spatial filters for rotation invariant pattern recognition.

Distributed memory and localized memory in neural networks

We propose a new method to find the proper center of the circular harmonic filter using simulated annealing. The new method is two orders of magnitude faster than the previous method. The proper center is a chosen expansion center which ensures the maximum peak of the correlation output to be at its center, thus a truly shift and rotation invariant pattern recognition system. The proper center is to be used in a new synthesis of filters: the geometrical approach of multiple circular harmonic filters. In this approach, the distances between the correlation peaks in multiple correlation outputs are used as features for pattern recognition. This method has good discrimination capability and can be implemented for practical applications.© (1992) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A three-layer neural network implemented by optical and electronic techniques for pattern recognition is described. The principle, the architecture, and the experimental results of the hardware system for pattern recognition are presented. A multiplex hologram is used as the interconnections between neurons of the input layer and the middle layer. The winner-take-all (WTA) operation of the intermediate layer is performed by an electronic circuit. The interconnection between neurons of the middle layer and the output layer is simply implemented with an optical transparency. By changing the patterns on the transparency, this system can be used for both auto- or heteroassociative memories. In addition to noise and partial hindrance tolerances, it also has illuminance invariance because of the WTA layer of the system.

Abstract-Hebbian learning is widely accepted in the fields of psychology, neurology, and neurobiology. It is one of the fundamental premises of neuroscience. The LMS (least mean square) algorithm of Widrow and Hoff is the world's most widely used adaptive algorithm, fundamental in the fields of signal processing, control systems, pattern recognition, and artificial neural networks. These are very different learning paradigms. Hebbian learning is unsupervised. LMS learning is supervised. However, a form of LMS can be constructed to perform unsupervised learning and, as such, LMS can be used in a natural way to implement Hebbian learning. Combining the two paradigms creates a new unsupervised learning algorithm that has practical engineering applications and provides insight into learning in living neural networks. A fundamental question is, how does learning take place in living neural networks? Nature's little secret, the learning algorithm practiced by nature at the neuron and synapse level, may well be the Hebbian-LMS algorithm.

The classification and recognition of two-dimensional patterns independently of their position, orientation, and size by using high-order networks are discussed. A method is introduced for reducing and controlling the number of weights of a third-order network used for invariant pattern recognition. The method leads to economical networks that exhibit high recognition rates for translated, rotated, and scaled, as well as locally distorted, patterns. The performance of these networks at recognizing types and handwritten numerals independently of their position, size, and orientation is compared with and found superior to the performance of a layered feedforward network to which image features extracted by the method of moments are presented as input.

Making Artificial Brains: Components, Topology, and Optimization.

A novel method for early diagnosis of Alzheimer’s disease based on pseudo Zernike moment from structural MRI

Handwriting number recognition was being challenge problem to do in the recent years. The main objective for our research waso recognized handwritten account number. The original data was bank deposit slip that acquired by scanner. Before do the recognition of account number handwritten, first step that must be done was located account number on the bank deposit slip. After the location was found then the account number was segmented to cut up each numbern. After cutting the stem then performed feature extraction to obtain a vector which was fed to the neural network system for recognition rate. System back propagation neural network for handwritten digit pattern recognition was designed by 168neuron consists of input layer, 70 neurons in the hidden layer and 10 neurons in the output layer. The results obtained in this study were 83.78% of the data slip can be recognized correctly.

Translational, rotational, and scaling invariant (TRSI) pattern recognition is a third-order problem encountered frequently in real-world applications. But neither traditional image processing/pattern recognition algorithms nor artificial neural networks have yet provided satisfactory solutions for this problem after years of study. Recent research has shown that a higher-order neural network (HONN), of order three with built-in invariances, can effectively achieve TRSI pattern recognition. For an N X N image, the memory needed to store the connections is proportional to N⁶. This huge memory requirement limits the HONNs application to large-scale images. To solve this problem the authors first adapt edge detection and log-spiral mapping algorithms to preprocess the image so that the problem is converted into a second-order one. This reduces the HONN memory requirement to O(N⁴). Second, the authors modified the second-order HONN architecture to further reduce the memory size to O(N²). Synthetic and real images with resolution 256 X 256 have been used for simulation. The training samples are noise free, and the testing samples are rotated, translated, scaled, or noise-corrupted versions of the training patterns. Simulation results show that this system can indeed achieve TRSI pattern classification. In addition, its high robustness to noise and pattern deformation makes it very useful for real-world applications.