Hardware-Based Hopfield Neuromorphic Computing for Fall Detection

There are numerous advantages of deep neural network surrogate modeling for response time-history prediction. However, due to the high cost of refined numerical simulations and actual experiments, the lack of data has become an unavoidable bottleneck in practical applications. An iterative self-transfer learning method for training neural networks based on small datasets is proposed in this study. A new mapping-based transfer learning network, named as deep adaptation network with three branches for regression (DAN-TR), is proposed. A general iterative network training strategy is developed by coupling DAN-TR and the pseudo-label strategy, and the establishment of corresponding datasets is also discussed. Finally, a complex component is selected as a case study. The results show that the proposed method can improve the model performance by near an order of magnitude on small datasets without the need of external labeled samples, well behaved pre-trained models, additional artificial labeling, and complex physical/mathematical analysis.

This paper addresses the design and experimental characterization of a novel hybrid neural network, in which two distinct classical architectures interact: the Hopfield neural network and the Multi-Layer Perceptron. This hybrid neural system, named MLP+H (from MLP + Hopfield), presents a better performance than each one of the two classical architectures when considered in separate. In addition, it has the ability to deal with different classes of data than that normally allowed by the two conventional architectures. For example, while the Hopfield networks deal with binary patterns and the MLPs with information that always have some analog characteristics (due to the continuous nature of the MLP nodes), the MLP+H deals with analog inputs and purely digital outputs. Moreover, the MLP+H allows reduced training times when compared with the MLP architecture, also presenting compactness and flexibility in dealing with different applications, as implementation of anti-noise filters, an application currently under study.KeywordsConfusion MatrixInput PatternAnalog InputOutput PatternHybrid NetworkThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In 1984 Hopfield showed that the time evolution of a symmetric Hopfield neural networks are a motion in state space that seeks out minima in the energy function (i.e., equilibrium point set of Hopfield neural networks). Because high-order Hopfield neural networks have more extensive applications than Hopfield neural networks, and have been discussed on the convergence of the networks. In practice, a neural network is often subject to environmental noise. It is therefore useful and interesting to find out whether the high-order neural network system still approacher some limit set under stochastic perturbation. In this paper, we will give a number of useful bounds for the noise intensity under which the stochastic high-order neural network will approach its limit set. Our result cancels the requirement of symmetry of the connection weight matrix and includes the classic result on Hopfield neural networks, which is a special case of stochastic high-order Hopfield neural networks. In the end, A example is given to verify the effective of our results.

Iterative learning control problem based on improved discrete-time Hopfield neural networks is considered in this paper. For the every process of iterative learning control, the neural networks execute a cycle that includes variable terms of learning time and training iterative number. The iterative learning control with improved Hopfield neural networks is formulated that can be described as a two-dimensional (2-D) Roesser model with variable coefficients. In terms of 2-D systems theory, sufficient conditions that iterative learning error approaches to zero are given. It has been shown that convergence of iterative learning control problem based on Hopfield neural networks is derived by 2-D systems theory instead of conventional algorithms that minimize a cost function.KeywordsRecurrent Neural NetworkIterative Learning ControlHopfield Neural NetworkNeural Network ControlRoesser ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The Boolean Satisfiability Problem (BSAT) is one of the most important decision problems in mathematical logic and computational sciences for determining whether or not a solution to a Boolean formula.. Hopfield neural network (HNN) is one of the major type artificial neural network (NN) popularly known for it used in solving various optimization and decision problems based on its energy minimization machinism. The existing models that incorporate standalone network projected non-versatile framework as fundamental Hopfield type of neural network (HNN) employs random search in its training stages and sometimes get trapped at local optimal solution. In this study, Ants Colony Optimzation Algorithm (ACO) as a novel variant of probabilistic metaheuristic algorithm (MA) inspired by the behavior of real Ants, has been incorporated in the training phase of Hopfield types of the neural network (HNN) to accelerate the training process for Random Boolean kSatisfiability reverse analysis (RANkSATRA) based for logic mining. The proposed hybrid model has been evaluated according to robustness and accuracy of the induced logic obtained based on the agricultural soil fertility data set (ASFDS). Based on the experimental simulation results, it reveals that the ACO can effectively work with the Hopfield type of neural network (HNN) for Random 3 Satisfiability Reverse Analysis with 87.5 % classification accuracy

ADANOISE: Training neural networks with adaptive noise for imbalanced data classification

Research on artificial neural networks (ANNs) has been carried out for more than five decades. A renewed interest appeared in the 80's with the finding of powerful models like J. Hopfield's recurrent networks, T. Kohonen's self-organizing feature maps, and the back-propagation rule. At that time, there was no platform that was at the same time versatile enough for any ANN model to be implemented and fast enough to solve large problems. Super-computers were the sole exception to this rule, but were prohibitively expensive for most applications. However, both research scientists and application engineers clearly identified the need for such a computing power. This triggered many projects in the field. In parallel, research on multi-processor systems started during the 60's. Systolic arrays have been proposed in 1979 as a means to fully exploit the possibilities of VLSI. Two previous theses, by F. Blayo and C. Lehmann, have studied the use of bi-dimensional systolic arrays for neural computation. At first, the presented system, called GENES, has been designed for the Hopfield model. Extensions to other ANNs have also been proposed. The goal of the present thesis is to study, design, build, and analyze an efficient accelerator for neural computation. In a first step, the GENES architecture has been extended towards generality and efficiency. This includes a thorough analysis of ANN models, of other neural computers, and of previous GENES implementations. The result of this work is the GENES IV integrated circuit, whose architecture has been co-designed by P. Ienne and the author. The main part of this thesis discusses the architecture, the design and the analysis of the MANTRA I machine, a neural computer based on a GENES IV array with up to 40 × 40 processing elements (PEs). The delta rule (and hence the Perceptron and Adaline rules), the back-propagation rule, the Hopfield model, and the Kohonen model can be implemented on this system. Although not a generic system, such a machine may be regarded as a multi-model neural computer. A prototype has been running for a year and is used daily by software designers. Several novel features distinguish the MANTRA I machine from other neural systems. First, it belongs to the few existing neural computers, contrary to the majority of implementations, which are specific to an application or to an algorithm. The machine does not hard-wire any algorithm, but provides the necessary primitives to implement the target models. This is a key feature for research, since several algorithms or versions of an algorithm can be tested on a problem. It is an important aspect for applications as well, because different ANN models are often cascaded to solve a problem. The GENES IV array — that is, the computing core of the MANTRA I machine — features synapse-level parallelism (i.e., one real or virtual PE is allocated per synapse or neural connection), while most other systems exploit only neuron-level parallelism (i.e., one PE per neuron). Hence, this system aims at a much finer parallelism grain and is well suited for massively parallel architectures. The problem size that can be computed by a neural accelerator should not be limited by the hardware (except for memory size). Therefore, it is essential to support time-sharing of PEs. On the MANTRA I machine, this is achieved by the concept of virtual arrays. Matrices are divided into sub-matrices that can be mapped onto the physical array, which is then time-shared among them. An efficient mechanism has been implemented to swap sub-matrices in background, while some other computation is performed. Since systolic arrays are pipelined systems, it is important to avoid emptying and re-filling them too often, in order to keep the hardware utilization rate high. Therefore, a systolic instruction flow has been implemented, so that each instruction follows the data for which it has been issued. Like any SIMD system, the MANTRA I machine is composed of a parallel or SIMD module and a control module. The SIMD module consists of the GENES IV array and a set of dedicated units designed for the computation that scales with the number of neurons and would poorly fit on a bi-dimensional array. A complex system of FIFOs and memories sustains the required input/output streams for the systolic array. The control module is a complete SISD system. Its tasks are (1) to control the SIMD module by dispatching instructions, (2) to manage data input and output, (3) to communicate with the external world, and (4) to perform data pre- and post-processing. The SIMD instructions are of the very long instruction word (VLIW) type. Synchronization between the two modules is achieved by an instruction FIFO. The performance of the MANTRA I machine has been analyzed using the delta rule. Measurements show that the sustained performance is very close to its peak value, as long as the problem fits in the memory banks connected to the GENES IV array. Experiments have also been run to investigate the impact of the constraints imposed by the hardware on the convergence of algorithms. Finally, the use of systolic arrays as neural accelerators is discussed in the light of the experience acquired with the GENES IV array and the MANTRA I machine. The weaknesses of the machine are analyzed, and several solutions are proposed to avoid them in a future design. A general discussion of the future of neural computers concludes this thesis.

In this paper, an automatic determination algorithm for nuclear magnetic resonance (NMR) spectra of the metabolites in the living body by magnetic resonance spectroscopy (MRS) without human intervention or complicated calculations is presented. In such method, the problem of NMR spectrum determination is transformed into the determination of the parameters of a mathematical model of the NMR signal. To calculate these parameters efficiently, a new model called Hopfield neural network is designed. The main achievement of this paper over the work in literature [30] is that the speed of the Hopfield neural network is accelerated. This is done by applying cross correlation in the frequency domain between the input values and the input weights. The Hopfield neural network can accomplish complex dignals perfectly with out any additinal computation steps. This is a valuable advantage as NMR signals are complex-valued. In addition, a technique called modified sequential extension of section (MSES) that takes into account the damping rate of the NMR signal is developed to be faster than that presented in [30]. Simulation results show that the calculation precision of the spectrum improves when MSES is used along with the neural network. Furthermore, MSES is found to reduce the local minimum problem in Hopfield neural networks. Moreover, the performance of the proposed method is evaluated and there is no effect on the performance of calculations when using the Hopfield neural networks.

Historical database training method for neural networks

DTOF-ANN: An Artificial Neural Network phishing detection model based on Decision Tree and Optimal Features

Distributed memory and localized memory in neural networks

We have developed a novel framework for combining physics-based forward models and neural networks to advance seismic processing and inversion algorithms. Migration is an effective tool in seismic data processing and imaging. Over the years, the scope of these algorithms has broadened; today, migration is a central step in the seismic data processing workflow. However, no single migration technique is suitable for all kinds of data and all styles of acquisition. There is always a compromise on the accuracy, cost, and flexibility of these algorithms. On the other hand, machine-learning algorithms and artificial intelligence methods have been found immensely successful in applications in which big data are available. The applicability of these algorithms is being extensively investigated in scientific disciplines such as exploration geophysics with the goal of reducing exploration and development costs. In this context, we have used a special kind of unsupervised recurrent neural network and its variants, Hopfield neural networks and the Boltzmann machine, to solve the problems of Kirchhoff and reverse time migrations. We use the network to migrate seismic data in a least-squares sense using simulated annealing to globally optimize the cost function of the neural network. The weights and biases of the neural network are derived from the physics-based forward models that are used to generate seismic data. The optimal configuration of the neural network after training corresponds to the minimum energy of the network and thus gives the reflectivity solution of the migration problem. Using synthetic examples, we determine that (1) Hopfield neural networks are fast and efficient and (2) they provide reflectivity images with mitigated migration artifacts and improved spatial resolution. Specifically, the presented approach minimizes the artifacts that arise from limited aperture, low subsurface illumination, coarse sampling, and gaps in the data.

This work presents a new approach to solve the location management problem by using the location areas approach. Hopfield neural network is used in this work to find the optimal configuration of location areas in a mobile network. Toward this end, the location areas configuration of the network is modeled so that the mobility management cost of the system is related to the energy value of this artificial neural network. Since a pure Hopfield neural network was not efficient enough to be used in solving this problem, some modifications applied to it to make it a reasonable approach. Basically, these modifications deal with randomness of selection in manipulating the cells during the solving process. Simulation results are very promising and they lead to network configurations that are unexpected.

Passwords are a very important sensitive aspect in computer security. Password is the important identitiy in authenticating any individual in different domains. It plays a vital role in accessing any device or any software. Users around the world are consistently told about the necessity of a strong password to protect their access with unauthorized users. In this paper, we propose a comparative analysis of soft computing techniques such as Back propagation neural network (BPN), Logistic regression (LR), Hopfield neural network (HNN), Brain state in a box (BSB) and associative memory network (BAM), Convolution neural network (CNN) system for strong password classification. The dataset contains only a single attribute of passwords with 3 classes as weak, medium and strong. The experimental results analysis shows that for such a datset, the simple logistic regression model yields better output when compared with Convolution Neural networks, Back propagation neural network, Logistic regression, Hopfield neural network, Brain state in a box and Bidirectional Associative Memory. It can be concluded that logistic regression is adapted for such an application and also plays as a better prediction system for such a dataset.

In order to assist the designer in the design process, it is necessary to develop a new intelligent computational methodology that involves the integration of design, analysis and evaluation, and optimisation. In this paper, a hybrid cross-mapping neural network model (HCMNN) is proposed to support design, analysis and evaluation, and optimisation in the design process, which integrates a back-propagation neural network (BPNN) and a Hopfield neural network (HNN). The BPNN is used for representing design patterns, training classification boundaries, and outputting the net weight values to the HNN, and then the HNN uses the calculated weight values to evaluate and modify or redesign the design patterns. The developed system provides a unified computational intelligent design framework. The system has self-modifying or self-learning functions. Within the system, only one network is needed to train for accomplishing design analysis and evaluation, rectification/modification and optimisation tasks in the design process. Finally, two case studies are provided to verify the proposed model.