A massively parallel architecture for a self-organizing neural pattern recognition machine

Neural Dynamics of Category Learning and Recognition: Attention, Memory Consolidation, and Amnesia

This paper proposes enhancement of the search space (SSS) method attempted in [Y. Yang, et al., (2003)] to further accelerate the recognition speed. It consists of structuring the search space into two layers, improving the candidate selection algorithm and selecting candidates depending on the top candidate. For two-layered search space, we divide all of the prototypes into smaller clusters and derive the centroid of each cluster as a pivot, then again cluster all of the pivots and derive the centroid of each cluster (super cluster) as a super pivot. An input pattern is compared with all the super pivots and several super clusters are selected whose super pivots are close to the input pattern. Then, the input pattern is compared with pivots in the selected super clusters, close pivots are selected and prototypes within the clusters of the selected pivots are treated as candidates for fine classification. Thus, the number of prototypes compared with the input pattern is greatly reduced. Moreover, we employ a synthetic candidate selection algorithm and a top candidate dependent candidate selection method. Since the top candidate suggests where the input pattern is mapped in the feature space, it can provide the information on how candidates should be selected in coarse classification. Thus, this information is specified in each prototype for the case when it is selected as the top candidate and specified values are employed for selecting a variable number of candidates. These improvements have been incorporated into a practical off-line Japanese character recognizer consisting of coarse classification and fine classification with the result that the coarse classification time is reduced to 28.6% and the whole recognition time is reduced to 31.3% from the original time while sacrificing a very limited recognition rate (98. 1 % to 97.7%).

A neural network model of visual pattern recognition, called the Necognitron, was previously proposed by Kunihiko Fukushima (1988). After training, it can recognize the input pattern without being affected by a change in size or a shift in position. The Fukushima model possesses several appealing categories so that its application would not be restricted to the process of pattern recognition. It can be applied to many other fields if its details are modified properly. With the model's ability of selective attention, gain control and perfect recall for deformed patterns, as a model of associative memory, it would be possible to endow the data compression problem. This article proposes a learning scheme that embeds the Karhunen-Loeve (K-L) transform basis technique into the structure of a Fukushima based neural network to compress the Arabic alphabetic patterns data as well as to reduce the input dimensionality for the network training. Despite that, the input pattern can still be recognized and reconstructed from a few local features. The proposed scheme converges the connections weight vectors to the principal eigenvectors, that retains the maximum information contained in the Arabic alphabetic patterns set into a few significant local features and reduces the redundancies present among the inputs to the perceptual layer. The learning process not only leads to efficient data compression and reconstruction but also enhances the network's ability of feature extraction and Arabic alphabetic recognition.

Introduction A growing body of research suggests that sleep has an important role in consolidating newly learnt information and generalising this knowledge to novel instances (e.g. Djonlagic et al., 2009). The effects of sleep on early categorisation abilities haven’t been investigated so far and thus the aim of this study is to examine the role of sleep in formation of category representations. Two experiments were conducted. Experiment 1 set out to determine 4-months-old infants’ ability to form novel visual categories when category formation test immediately followed learning. Experiment 2 investigated the impact of sleep on category consolidation by testing category formation after a 2 hours delay. Methods Both experiments employed eye-tracking familiarisation-novelty-preference paradigm. In Experiment 1, infants were familiarised with a set of exemplars from a novel category. Immediately after the familiarisation phase, infants’ category formation was tested and their looking references were used as an index of category learning. Experiment 2 tested consolidation of the newly acquired categories by introducing a delay between the familiarisation and the test phase. Half of the participants took a nap during the delay, whereas the other half remained awake. Polysomnography was recorded using a standard PSG protocol. Sleep stages were scored according to the AASM guidelines, and sleep spindles were detected using an adapted algorithm for children. Results Results of the Experiment 1 revealed that infants were able to learn novel visual categories when category formation was tested immediately after learning suggesting therefore that infants are able to extract relevant information and generalise to novel instances. Preliminary results of Experiment 2 (data analysis ongoing) suggested that infants who napped performed better than infants who stayed awake. Conclusion The present study suggests that infants are able to rapidly form novel categories, and that consolidation of this knowledge is shaped by the activity following learning. Support N/A

The basal ganglia (BG), and more specifically the striatum, have long been proposed to play an essential role in action-selection based on a reinforcement learning (RL) paradigm. However, some recent findings, such as striatal spike-timing-dependent plasticity (STDP) or striatal lateral connectivity, require further research and modelling as their respective roles are still not well understood. Theoretical models of spiking neurons with homeostatic mechanisms, lateral connectivity, and reward-modulated STDP have demonstrated a remarkable capability to learn sensorial patterns that statistically correlate with a rewarding signal. In this article, we implement a functional and biologically inspired network model of the striatum, where learning is based on a previously proposed learning rule called spike-timing-dependent eligibility (STDE), which captures important experimental features in the striatum. The proposed computational model can recognize complex input patterns and consistently choose rewarded actions to respond to such sensorial inputs. Moreover, we assess the role different neuronal and network features, such as homeostatic mechanisms and lateral inhibitory connections, play in action-selection with the proposed model. The homeostatic mechanisms make learning more robust (in terms of suitable parameters) and facilitate recovery after rewarding policy swapping, while lateral inhibitory connections are important when multiple input patterns are associated with the same rewarded action. Finally, according to our simulations, the optimal delay between the action and the dopaminergic feedback is obtained around 300 ms, as demonstrated in previous studies of RL and in biological studies.

We investigate techniques to combine multiple representations of a handwritten digit to increase classification accuracy without significantly increasing system complexity or recognition time. We compare multiexpert and multistage combination techniques and discuss in detail in a comparative manner methods for combining multiple learners: voting, mixture of experts, stacking, boosting and cascading. In pen based handwritten character recognition, the input is the dynamic movement of the pentip over the pressure sensitive tablet. There is also the image formed as a result of this movement. On a real world database, we notice that the two multi layer perceptron (MLP) neural network based classifiers using these representations separately make errors on different patterns, implying that a suitable combination of the two would lead to higher accuracy. Thus we implement and compare voting, mixture of experts, stacking and cascading. Combined classifiers have an error percentage less than individual ones. The final combined system of two MLPs has less complexity and memory requirement than a single k nearest neighbor using one of the representations.

Multimode optical fibers, with their high spatial-mode capacity, present a promising platform for high-fidelity image transmission. However, environmental instabilities and complex input patterns often result in intricate speckle outputs, complicating accurate image reconstruction. In this work, we encode grayscale images through orbital angular momentum (OAM) mode superpositions and achieve high-fidelity reconstruction using a ResNet-based decoding network. By incorporating transfer learning, the model demonstrates strong generalization across different wavelengths and spectral linewidths, attaining a reconstruction accuracy of up to 99%. Furthermore, we develop an attention-enhanced DoubleU-Net to reconstruct complex images with rich edge structures. Experimental results verify that OAM filtering substantially enhances edge fidelity, improving the reconstruction accuracy by approximately 4% over the non-OAM case, reaching a final accuracy of 95%. These findings open avenues toward high-dimensional optical information processing, multimode fiber communications, and advanced computational imaging.

An application of neocognitron to the formation of a closed boundary contour of the left ventricle (LV) from line segments in echocardiographic images is presented. In echocardiographic image processing, the extraction of a closed boundary contour is a very difficult but important task for both automatic diagnosing and monitoring of heart functions. The two major problems are the extraction of complex contour edge pattern structures from noisy edge streaks and the reconstruction of missing edges. The first problem can be cast as a feature extraction problem, and the second one can be solved by the top-down model-based approach using the existing contextual information. The neocognitron model which has a hierarchical structure and forward and backward paths provides solutions to these two problems. The recognition of an input pattern is carried out sequentially, layer by layer along the forward path. The forward signals route is later retraced by the backward signals. The noise edges in the input pattern, which are not retraced in the backward route, are eliminated by the gain control mechanism. The incomplete features, which appear in the backward route, but might not be extracted in the forward route, also can be extracted by lowering the threshold value. The missing edge in the initial pattern can be filled back once the feature corresponding to that edge has been reconstructed. This neocognitron network has three layers: the lower, the intermediate, and the highest, containing the templates for basic geometrical features, basic curve features, and whole boundary contour, respectively. The models of the left ventricle boundary are created semi-automatically from real echocardiographic images. A model is established for each class of images based on the shape of the boundary contour which is defined according to age, gender, disease, probe position, and the time of contraction or expansion. The curve features are extracted from the boundary contour model semi-automatically.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Development of orientation maps in ferret and cat primary visual cortex (V1) has been shown to be stable, in that the earliest measurable maps are similar in form to the eventual adult map [1], robust, in that similar maps develop in both dark rearing and in a variety of normal visual environments [2], and yet adaptive, in that the final map pattern reflects the statistics of the specific visual environment [3]. How can these three properties be reconciled? Using a mechanistic model of the development of neural connectivity in V1, we show how including two low-level mechanisms originally motivated from single-neuron results makes development stable, robust, and adaptive. Specifically, contrast gain control in the retinal ganglion cells and the LGN reduces variation in the pre-synaptic drive due to differences in input patterns, while homeostatic plasticity of V1 neuron excitability reduces the post-synaptic variability in firing rates. Together these two mechanisms lead to maps that develop stably and robustly, yet adapt to the visual environment. The modeling results suggest that topographic map stability is a natural outcome of low-level processes of adaptation and normalization. The resulting GCAL model is also significantly simpler yet more robust and more biologically plausible than previous mechanistic models of cortical map, receptive field, and connection development, and thus represents a good platform for future cortical modeling. The simulator and the model code can be freely downloaded from topographica.org. Figure ​Figure11. Figure 1 (A) Polar orientation maps recorded in a ferret using chronic optical imaging at the postnatal ages shown (in days; reprinted from [1]). (B) GCAL model polar orientation maps. Spontaneous activity patterns drive the map development until 6000 image presentation ...

A new way of measuring generalization in unsupervised learning is presented. The measure is based on an exclusive allocation, or credit assignment, criterion. In a classifier that satisfies the criterion, input patterns are parsed so that the credit for each input feature is assigned exclusively to one of multiple, possibly overlapping, output categories. Such a classifier achieves context-sensitive, global representations of pattern data. Two additional constraints, sequence masking and uncertainty multiplexing, are described; these can be used to refine the measure of generalization. The generalization performance of EXIN networks, winner-take-all competitive learning networks, linear decorrelator networks, and Nigrin's SONNET-2 network are compared.

Geometric moments have been widely used in many shape recognition and object classification tasks. These monomials are usually computed from binary or gray-level images for the object shape recognition invariant to rotation, translation, and scaling. In this paper, we attempt to calculate the shape related moments from color images, and study their noise immunity and color invariance property for the application areas of face recognition and content based image retrieval. To this end, we describe a computationally efficient method of converting a vector-valued color image into a gray scale for robust moment computation. Geometric moments are calculated from the resultant scalar representation of a color image data, and proven to be robust shape descriptors for the face and flower images. The generated shape invariants appear to have better noise immunity than the Hu moments and exhibit characteristics invariant to hue changes in the object colors. As compared to the Zernike polynomials, the proposed feature set has higher discriminatory power although the Zernike polynomials present superior noise rejection capability. Robust performance, computational efficiency, high noise immunity, and hue invariance property of the new approach are particularly useful for fast image retrieval tasks requiring high query accuracy.

Summary. People working in Didactics of Mathematics have constantly regarded statistical implicative analysis as a profitable and heuristic method of data analysis. First we intend to show the reasons for this interest: implicative links that S.I.A. has pointed out may be interpreted as rules and regulations connecting actions, discourses, ..., or as a group’s characteristics. We develop some examples showing how S.I.A. can be used and what special research results it can provide. We insist upon some points that may be interesting methodologically to focus on: asymmetric links, nodes and separate implicative ways. Theorizing the relations between these two fields of research, didactics of mathematics and Statistical Implicative Analysis, in terms of producers of models and techniques on one hand and in the field of application on the other is without doubt too reductive, from a historical point of view. In fact the still very brief history of the emergence of these two scientific domains shows us some more complex connections: the coincidence of the time of emergence and recognition, that of their geographical and institutional situations- some universities and associations of researchers in France —and finally, and above all, the presence of common actors — Regis Gras particularly [4‐6]— imply a dynamic which is specific to these relations. Thus the implicative analysis of data has been able to be identified as a preferred method of analysis in mathematical didactics and reciprocally in some way, some problems of didactics have been able to raise questions in implicative analysis. First we seek to clarify the reasons why we see it as a fruitful cooperation, in seeking to understand how the clarification of rules (or quasi-rules) that implicative analysis allows proves to be a valuable and pertinent tool for mathematical didacticians. Then we will explain two of the main problems in which these rules have meaning in mathematical didactics: that of the D. Lahanier-Reuter: Didactics of Mathematics and Implicative Statistical Analysis, Studies in

The main goal of the present research was to demonstrate the interaction between category and causal induction in causal model learning. We used a two-phase learning procedure in which learners were presented with learning input referring to two interconnected causal relations forming a causal chain (Experiment 1) or a common-cause model (Experiments 2a, b). One of the three events (i.e., the intermediate event of the chain, or the common cause) was presented as a set of uncategorized exemplars. Although participants were not provided with any feedback about category labels, they tended to induce categories in the first phase that maximized the predictability of their causes or effects. In the second causal learning phase, participants had the choice between transferring the newly learned categories from the first phase at the cost of suboptimal predictions, or they could induce a new set of optimally predictive categories for the second causal relation, but at the cost of proliferating different category schemes for the same set of events. It turned out that in all three experiments learners tended to transfer the categories entailed by the first causal relation to the second causal relation.

Image registration is an essential step in many image processing applications that need visual information from multiple images for comparison, integration, or analysis. Recently, researchers have introduced image registration techniques using the log-polar transform (LPT) for its rotation and scale invariant properties. However, it suffers from nonuniform sampling which makes it not suitable for applications in which the registered images are altered or occluded. Inspired by LPT, this paper presents a new registration algorithm that addresses the problems of the conventional LPT while maintaining the robustness to scale and rotation. We introduce a novel adaptive polar transform (APT) technique that evenly and effectively samples the image in the Cartesian coordinates. Combining APT with an innovative projection transform along with a matching mechanism, the proposed method yields less computational load and more accurate registration than that of the conventional LPT. Translation between the registered images is recovered with the new search scheme using Gabor feature extraction to accelerate the localization procedure. Moreover an image comparison scheme is proposed for locating the area where the image pairs differ. Experiments on real images demonstrate the effectiveness and robustness of the proposed approach for registering images that are subjected to occlusion and alteration in addition to scale, rotation, and translation.

A hybrid utilisation of the Fuzzy ARTMAP (FAM) neural network and the Probabilistic Neural Network (PNN) is proposed for online learning and prediction tasks. FAM is used as an underlying clustering algorithm to classify the input patterns into different recognition categories during the learning phase. Subsequently, a non parametric probability estimation procedure in accordance with the PNN paradigm is employed during the prediction phase. This hybrid approach realises an incremental learning network with implementation of the Bayes strategy for online applications. The effectiveness of this network is assessed with statistical classification problems in both stationary and non stationary environments. Simulation studies illustrate that the network is capable of asymptotically approaching the Bayes optimal classification rates.