Time Variant Change Analysis in Satellite Images

Concrete pavement joint evaluation involves a number of assessment criteria, such as deflection near the joint, load transfer efficiency of the dowels and severity of voids under the slab. Although there are well defined thresholds for each of these parameters, often there arises a situation where each of the considered parameters lends contradictory assessment that leads to a considerable subjectivity in the evaluation process. A Self-Organizing Map (SOM), an unsupervised learning procedure in artificial neural network, is utilised for the first time to map the joint condition of concrete pavements from Falling Weight Deflectometer (FWD) deflection bowls. A novel methodology is proposed for labelling the network, whereby pavement engineering expertise can be directly used in a SOM for consistent deflection data classification in joint evaluation. The effectiveness of the trained network is demonstrated by using joint assessment parameters; namely, load transfer efficiency (LTE), void intercepts and absolute deflection. The joints were classified as good, marginal or poor. For the three parameters based SOM classification, an accuracy of 65-70% was obtained; this improves to 87.5% when the SOM was trained with 2-parameters (LTE and absolute deflection). However, when the SOM was tested with the data classified as 'good', accuracy improves to around 90%. Therefore, a SOM can be a powerful supplementary tool for a consistent and non-subjective evaluation of concrete pavement joints.

A new method based on artificial neural networks is developed for the identification of white blood corpuscle images. It uses a two-dimensional Self Organizing Feature Map (SOFM) and a single-layer perceptron. The SOFM algorithm imitates the ordering of sensory pathways and the high level of organization created during learning in the human brain. It is used to reduce the dimension of the input vector corresponding to the digitized image. This reduced dimensional 'feature' vector is applied to a single-layer perceptron.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In recent times Artificial Neural Networks (ANNs) are more and more widely applied. The ANN is an information processing system consisting of numerous simple processing units (neurons) that are arranged in layers and have weighted connections to each other. In the present study the possible application of an unsupervised neural network model, the self-organizing map (SOM), for the delineation of excess water areas have been examined. By means of the self-organizing map high-dimensional data of large databases could be mapped to a low-dimensional data space. Within a data set, it is able to develop homogeneous clusters, thus it can be effectively applied for the classification of multispectral satellite images. The classification was carried out for an area of 88 km2 to the south of Hódmezővásárhely situated in the south-eastern part of Hungary, which is frequently inundated by excess water. As input data, the intensity values of the pixels measured in six bands of a Landsat ETM image taken on 23rd April 2000 were used. To perform the classification, three different sized neural network models were created, which classified the pixels of the satellite image to 9, 12 and 16 clusters. By using the gained clusters three thematic maps were created, on which different types of excess water areas were delineated. During the validation of the results it was concluded that the applied neural network model is suitable for the delimitation of excess water areas and it could be an alternative to the traditional classification methods.

Independent component analysis (ICA) is a powerful tool for redundancy reduction and nongaussian data analysis. And, artificial neural network (ANN), especially the self-organizing map (SOM) based on unsupervised learning is a kind of excellent method for pattern clustering and recognition. By combining ICA with ANN, a novel compound neural network for fault sources recognition was proposed. First, neural ICA algorithm was applied to fusion of multi-channel measurements by sensors. Moreover, further feature extraction was made. Thus, statistical features higher than second order were captured from the measurements. Second, a typical neural classifier such as the back-propagation (BP), the radial basis function (RBF) or the SOM network was trained for the final fault sources recognition. The results from contrast experiments in fault diagnosis of rotating machines show that the proposed compound neural network with ICA based feature extraction can recognize various fault sources at considerable accuracy, and be constructed in simpler way, both of which imply its great potential in fault diagnosis.

Ecological Applications of Fuzzy Logic.- Ecological Applications of Qualitative Reasoning.- Ecological Applications of Non-supervised Artificial Neural Networks.- Ecological Applications of Genetic Algorithms.- Ecological Applications of Evolutionary Computation.- Ecological Applications of Adaptive Agents.- Bio-Inspired Design of Computer Hardware by Self-Replicating Cellular Automata.- Prediction and Elucidation of Stream Ecosystems.- Development and Application of Predictive River Ecosystem Models Based on Classification Trees and Artificial Neural Networks.- Modelling Ecological Interrelations in Running Water Ecosystems with Artificial Neural Networks.- Non-linear Approach to Grouping, Dynamics and Organizational Informatics of Benthic Macroinvertebrate Communities in Streams by Artificial Neural Networks.- Elucidation of Hypothetical Relationships between Habitat Conditions and Macroinvertebrate Assemblages in Freshwater Streams by Artificial Neural Networks.- Prediction and Elucidation of River Ecosystems.- Prediction and Elucidation of Population Dynamics of the Blue-green Algae Microcystis aeruginosa and the Diatom Stephanodiscus hantzschii in the Nakdong River-Reservoir System (South Korea) by a Recurrent Artificial Neural Network.- An Evaluation of Methods for the Selection of Inputs for an Artificial Neural Network Based River Model.- Utility of Sensitivity Analysis by Artificial Neural Network Models to Study Patterns of Endemic Fish Species.- Prediction and Elucidation of Lake and Marine Ecosystems.- A Comparison between Neural Network Based and Multiple Regression Models for Chlorophyll-a Estimation.- Artificial Neural Network Approach to Unravel and Forecast Algal Population Dynamics of Two Lakes Different in Morphometry and Eutrophication.- Hybrid Evolutionary Algorithm for Rule Set Discovery in Time-Series Data to Forecast and Explain Algal Population Dynamics in Two Lakes Different in Morphometry and Eutrophication.- Multivariate Time Series Prediction of Marine Zooplankton by Artificial Neural Networks.- Classification of Fish Stock-Recruitment Relationships in Different Environmental Regimes by Fuzzy Logic with Bootstrap Re-sampling Approach.- Computational Assemblage of Ordinary Differential Equations for Chlorophyll-a Using a Lake Process Equation Library and Measured Data of Lake Kasumigaura.- Classification of Ecological Images at Micro and Macro Scale.- Identification of Marine Microalgae by Neural Network Analysis of Simple Descriptors of Flow Cytometric Pulse Shapes.- Age Estimation of Fish Using a Probabilistic Neural Network.- Pattern Recognition and Classification of Remotely Sensed Images by Artificial Neural Networks.

An unsupervised learning network, such as the self organizing feature map (SOFM), has been applied successfully to color classification for image compression and pattern recognition. Like other vector quantization algorithms, the reconstruction quality and adaptation rate of the SOFM are sensitive to the neuron initialization. We propose an efficient new initialization method, whereby an excess number of neurons is defined and the neurons are adaptively pruned, merged and split within their lattice according to the spatial distribution of the input color pixels. Comparisons with conventional gray scale initialization using subsampling and butterfly jumping sequences show that the proposed method obtains good initial code vectors that can accelerate the convergence of the SOFM and improve the reconstructed image quality significantly.

Nonlinear independent component analysis (NICA) is a powerful method for analyzing nonlinear and nongaussian data. Artificial neural network (ANN), especially self-organizing map (SOM) based on unsupervised learning, is an excellent tool for pattern clustering and recognition. A novel multi-NICA network is proposed for feature extraction of different mechanical patterns, followed by a typical ANN that is one of Multi-Layer Perceptron (MLP), or Radial Basis Function Network (RBFN), or self-organizing map (SOM), which implements the final classification. Using NICA and appropriate strategies for further feature extraction, nonlinear and higher than second order features embedded in multi-channel vibration measurements can be captured effectively. Mechanical fault patterns can be recognized correctly. Results from the contrast classification experiments show that the new compound ICA-SOM classifier can be constructed in a simpler way and it can classify various fault patterns with high accuracy, both of which imply a great potential in health condition monitoring of machine systems.

Spiking Self-organizing Maps for Classification Problem

Summary Well-test data have been used traditionally for determining a variety of reservoir parameters, such as average permeability, storage capacity, reservoir damage, presence of faults and fractures, and reservoir mechanism. A number of techniques, both conventional methods, such as type-curve matching and numerical simulation, and artificial intelligence (AI) methods, have been used for identifying well-test models. These methods are laborious and time-consuming and at times give incorrect results. Artificial neural networks (ANN's) are recent developments in computer vision and image analysis. These networks are specialized computer software that generate a strategy to produce nonlinear mapping functions for complex problems. ANN's are commonly used as a tool for recognizing an object or predicting an event given an associated pattern. Only a limited number of applications of ANN for analyzing well-test data have been reported. These applications are mostly model-specific (developed for specific reservoir models) and, hence, are not general enough. This paper presents a new method based on ANN's that uses Ko-honen's self-organizing feature (SOF) mapping technique to identify well-test interpretation models. By grouping well-test data into distinct categories, the SOF algorithm produces a general mapping function. This method can help analyze well-test data from a large variety of reservoirs (including reservoirs with faults, fractures, boundaries, etc.) more efficiently and inexpensively than was feasible previously.

An intrusion detection system (IDS) is an important protection instrument for detecting complex network attacks. Various machine learning (ML) or deep learning (DL) algorithms have been proposed for implementing anomaly-based IDS (AIDS). Our review of the AIDS literature identifies some issues in related work, including the randomness of the selected algorithms, parameters, and testing criteria, the application of old datasets, or shallow analyses and validation of the results. This paper comprehensively reviews previous studies on AIDS by using a set of criteria with different datasets and types of attacks to set benchmarking outcomes that can reveal the suitable AIDS algorithms, parameters, and testing criteria. Specifically, this paper applies 10 popular supervised and unsupervised ML algorithms for identifying effective and efficient ML-AIDS of networks and computers. These supervised ML algorithms include the artificial neural network (ANN), decision tree (DT), k-nearest neighbor (k-NN), naive Bayes (NB), random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) algorithms, whereas the unsupervised ML algorithms include the expectation-maximization (EM), k-means, and self-organizing maps (SOM) algorithms. Several models of these algorithms are introduced, and the turning and training parameters of each algorithm are examined to achieve an optimal classifier evaluation. Unlike previous studies, this study evaluates the performance of AIDS by measuring the true positive and negative rates, accuracy, precision, recall, and F-Score of 31 ML-AIDS models. The training and testing time for ML-AIDS models are also considered in measuring their performance efficiency given that time complexity is an important factor in AIDSs. The ML-AIDS models are tested by using a recent and highly unbalanced multiclass CICIDS2017 dataset that involves real-world network attacks. In general, the k-NN-AIDS, DT-AIDS, and NB-AIDS models obtain the best results and show a greater capability in detecting web attacks compared with other models that demonstrate irregular and inferior results.

For many years, artificial neural networks (ANNs) have been studied and used to model information processing systems based on or inspired by biological neural structures. They not only can provide solutions with improved performance when compared with traditional problem-solving methods, but also give a deeper understanding of human cognitive abilities. Among various existing neural network architectures and learning algorithms, Kohonen’s selforganizing map (SOM) [46] is one of the most popular neural network models. Developed for an associative memory model, it is an unsupervised learning algorithm with a simple structure and computational form, and is motivated by the retina-cortex mapping. Self-organization in general is a fundamental pattern recognition process, in which intrinsic interand intra-pattern relationships among the stimuli and responses are learnt without the presence of a potentially biased or subjective external influence. The SOM can provide topologically preserved mapping from input to output spaces. Although the computational form of the SOM is very simple, numerous researchers have already examined the algorithm and many of its problems, nevertheless research in this area goes deeper and deeper – there are still many aspects to be exploited.

Research on artificial neural networks (ANNs) has been carried out for more than five decades. A renewed interest appeared in the 80's with the finding of powerful models like J. Hopfield's recurrent networks, T. Kohonen's self-organizing feature maps, and the back-propagation rule. At that time, there was no platform that was at the same time versatile enough for any ANN model to be implemented and fast enough to solve large problems. Super-computers were the sole exception to this rule, but were prohibitively expensive for most applications. However, both research scientists and application engineers clearly identified the need for such a computing power. This triggered many projects in the field. In parallel, research on multi-processor systems started during the 60's. Systolic arrays have been proposed in 1979 as a means to fully exploit the possibilities of VLSI. Two previous theses, by F. Blayo and C. Lehmann, have studied the use of bi-dimensional systolic arrays for neural computation. At first, the presented system, called GENES, has been designed for the Hopfield model. Extensions to other ANNs have also been proposed. The goal of the present thesis is to study, design, build, and analyze an efficient accelerator for neural computation. In a first step, the GENES architecture has been extended towards generality and efficiency. This includes a thorough analysis of ANN models, of other neural computers, and of previous GENES implementations. The result of this work is the GENES IV integrated circuit, whose architecture has been co-designed by P. Ienne and the author. The main part of this thesis discusses the architecture, the design and the analysis of the MANTRA I machine, a neural computer based on a GENES IV array with up to 40 × 40 processing elements (PEs). The delta rule (and hence the Perceptron and Adaline rules), the back-propagation rule, the Hopfield model, and the Kohonen model can be implemented on this system. Although not a generic system, such a machine may be regarded as a multi-model neural computer. A prototype has been running for a year and is used daily by software designers. Several novel features distinguish the MANTRA I machine from other neural systems. First, it belongs to the few existing neural computers, contrary to the majority of implementations, which are specific to an application or to an algorithm. The machine does not hard-wire any algorithm, but provides the necessary primitives to implement the target models. This is a key feature for research, since several algorithms or versions of an algorithm can be tested on a problem. It is an important aspect for applications as well, because different ANN models are often cascaded to solve a problem. The GENES IV array — that is, the computing core of the MANTRA I machine — features synapse-level parallelism (i.e., one real or virtual PE is allocated per synapse or neural connection), while most other systems exploit only neuron-level parallelism (i.e., one PE per neuron). Hence, this system aims at a much finer parallelism grain and is well suited for massively parallel architectures. The problem size that can be computed by a neural accelerator should not be limited by the hardware (except for memory size). Therefore, it is essential to support time-sharing of PEs. On the MANTRA I machine, this is achieved by the concept of virtual arrays. Matrices are divided into sub-matrices that can be mapped onto the physical array, which is then time-shared among them. An efficient mechanism has been implemented to swap sub-matrices in background, while some other computation is performed. Since systolic arrays are pipelined systems, it is important to avoid emptying and re-filling them too often, in order to keep the hardware utilization rate high. Therefore, a systolic instruction flow has been implemented, so that each instruction follows the data for which it has been issued. Like any SIMD system, the MANTRA I machine is composed of a parallel or SIMD module and a control module. The SIMD module consists of the GENES IV array and a set of dedicated units designed for the computation that scales with the number of neurons and would poorly fit on a bi-dimensional array. A complex system of FIFOs and memories sustains the required input/output streams for the systolic array. The control module is a complete SISD system. Its tasks are (1) to control the SIMD module by dispatching instructions, (2) to manage data input and output, (3) to communicate with the external world, and (4) to perform data pre- and post-processing. The SIMD instructions are of the very long instruction word (VLIW) type. Synchronization between the two modules is achieved by an instruction FIFO. The performance of the MANTRA I machine has been analyzed using the delta rule. Measurements show that the sustained performance is very close to its peak value, as long as the problem fits in the memory banks connected to the GENES IV array. Experiments have also been run to investigate the impact of the constraints imposed by the hardware on the convergence of algorithms. Finally, the use of systolic arrays as neural accelerators is discussed in the light of the experience acquired with the GENES IV array and the MANTRA I machine. The weaknesses of the machine are analyzed, and several solutions are proposed to avoid them in a future design. A general discussion of the future of neural computers concludes this thesis.

Bringing artificial intelligence to embedded devices has become a central research topic in many scientific domains (environment, agriculture, sociology, health…). For Human Activity Recognition, Artificial Neural Networks (ANNs) have shown their capability to provide better performance compared to other machine learning methods. However, ANNs suffer from two major limitations. First, ANNs are often trained using supervised learning requiring labelled databases, which are often difficult to build in real applications. Then, those algorithms are usually very expensive in terms of computing power. For that reason, their integration into low-power microcontrollers has been so far only evaluated to a limited extent. In this paper, we propose to evaluate quantitatively and qualitatively the embedded implementation of different neural networks for human activity recognition. First, supervised learning approaches are presented, followed by an exploratory study of unsupervised learning approaches using Self-Organizing Maps. Finally, some aspects of embedded unsupervised online learning are investigated to improve classification results using subject-specific data over a more general training. Each neural network is tested on a Human Activity Recognition dataset acquired from a smartphone using accelerometer and gyroscope sensing information (UCI HAR) and deployed on the SparkFun Edge board. This board hosts a low-power ARM Cortex-M4F-based microcontroller.

Artificial neural networks assessing adolescent idiopathic scoliosis: comparison with Lenke classification

Artificial Neural Networks (ANN) are a classic pattern classifier and widely applicable to various problems and are relatively easy to use. Three of the most popular ANNs are Multilayer Perceptron (MLP) with Backpropagation learning algorithm, Self Organizing Map (SOM) and Recurrent Neural Network (RNN). Support Vector Machines (SVM) have gained great interest in the last few years in pattern recognition. Thus, this research compares the recognition performance of text and non-text images (text, table, figure and graph) from technical document images based on the pixel intensity of various zones between BPNN, SOM, RNN and SVM. Symmetrical and non-symmetrical zoning algorithms were compared as input. 400 different datasets have been tested and the experiments indicate that SVM classification is superior to the other three classifiers. The experiments also indicate that the combination of symmetrical and non-symmetrical zoning design is better than non-symmetrical or symmetrical zoning only.