Document zone content classification for technical document images using Artificial Neural Networks and Support Vector Machines

The Sixth International Symposium on Neural Networks (ISNN 2009)

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

Medical imaging aids in the analysis of interior parts of the human body such as the functioning of the organs or tissues for early treatment of diseases. Many different types of medical imaging technologies exist, for example, X-ray radiography, magnetic resonance imaging, endoscopy, positron emission tomography, CT scan (computed tomography), and many more. A tumor is an abnormal tissue in the brain which causes damage to the functioning of the cell. Therefore, brain tumor detection is an incredibly tricky task. Manual detection of a tumor is quite risky as it involves the insertion of a needle in the brain. Thus, there is a need for automated brain tumor detection systems. The well-timed detection of the tumor can add to accurate treatment and can increase the survival rate of patients. From machine learning techniques, namely K-nearest neighbor, support vector machine, and more to soft computing techniques, namely artificial neural network, self-organizing map, and others hold a significant stand in detection and categorization of brain tumor. Various methods including deep learning-based classifiers such as convolutional neural network, recurrent neural network, deep belief network (DBN), and others are used to make it easier to detect the tumor. Hybrid classifiers were also used for classification systems such as combining the machine learning approach with soft computing. This study is to summarize and compare the work of various authors on automatic brain tumor detection using medical imaging. Based on the accuracy, specificity, and sensitivity parameters, the results of different techniques are analyzed and compared graphically.

Arabic language can be used by native and non- native speakers; due to Arabic is the language of the holy book of Muslims. In this paper, Arabic phoneme recognition system is proposed based on Malay speakers. This system consists of three main stages. The first stage is noise reduction and it aims to enhance the phoneme signals by excluding the unvoiced signals and keep only the voiced signal. Wiener filter is adapted to accomplish this task. The second stage is based on Mel-Frequency Cepstral Coefficients method to extract a vector of features to represent each phoneme signal. Eventually, pattern recognition neural network is designed as recognizer. The proposed system produces sufficient outcomes with 20 hidden neurons. Keywords—Arabic; Malay; pattern recognition; Wiener; Mel- Frequency Cepstral Coefficients I. INTRODUCTION It is well known the wide range of Artificial Neural Networks (ANN) application in speech recognition, financial, telecommunications, electronics and medical. As for speech recognition system for Arabic language, (1) & (2) have addressed the implementation of recognition system based on Arabic native speakers. In general, neural networks (NNs) for phonemes recognition are divided into several categories for instance Very Large Scale Integration (VLSI) NN that can be divided into digital and analogue with digital NNs being more compatible with feed-forward neural networks while analogue NNs are found to be more successful with recurrent NNs (3). From Automatic Speech System (ASR) point of view, NNs for recognition purpose can be divided into two main category namely conventional neural networks such as Multilayer Perceptron (MLP) and Radial Basis Function (RBF) and secondly Recurrent Neural Networks (RNN). The first category of NNs was implemented as pattern classifiers and proven capable for recognition purpose but not at par as compared to Hidden Markov Model (HMM) performance (2). At present, HMM has been proven to be the most successful approach in ASR research area until recently the possibility to hybrid both HMM and ANN (4) & (5). In general, speech recognition systems consist of three stages specifically noise reduction, feature extraction and classification. The aim of noise reduction is to reduce the level of noise in the speech signal and make it more compatible for the ASR. As for feature extraction, this process attempted to extract the most relevant features related to the phoneme signal. The extracted features acted as input features of the NN during training and testing. Finally, the purpose of classification process is to arrange the most similar and related features in categories based on the training phase. Basically, networks training database are being observed and updated during training session of the networks, involving the weights and biases arguments. Networks with lower errors can be considered as better networks (6). Hence, this paper deems to explore further the ability of NN for Arabic phonemes recognition spoken by Malay individuals. This paper is arranged as following; related work is discussed in section II followed by section III that discussed method to be implemented based on Wiener filter, Mel-Frequency Cepstral Coefficients and pattern recognition neural network. Section IV elaborated experimental analysis and results and followed by Conclusion section.

Using information technology such as search engine for collecting and monitoring of network public opinion is a practical and effective method. This paper puts forward an improved algorithm of SVM classifier based on incremental learning online, and then implements a topic crawler system for network public opinion and to grab the public opinion.

Nowadays, the Structural Building Health Damage Monitoring System (SBHDMS) is a crucial technology for predicting the civil building structures' health. SBHDMS contains abnormal changes in the buildings in terms of damage levels. Natural Disasters like Earthquakes, Floods, and cyclones affect the unusual changes in the buildings. If the building undergoes any natural disaster, the sensors capture the vibration data or change the buildings' structure. Due to the vibration data, these unusual changes can be analyzed. Here sensors or Machine Learning based Building Damage Prediction (MLBDP) are used for capturing and collecting the vibration data. This paper proposes a Novel Rough Set based Artificial Neural Network with Support Vector Machine (RAS) metaheuristic method. RAS method is used to predict the damaged building's vibration data levels captured by the sensors. For the feature reduction subset, we use one of the essential pre-processing method called the Rough set theory (RST) strategy. RAS has two contributions. The first one is the Support Vector Machine (SVM) classification method used for identifying the structures of the buildings. The artificial Neural Network (ANN) method used to predict the buildings' damage levels is the second contribution. The proposed method (RAS) is accurately predicting the conditions of the construction building structure and predicting the damage levels, without human intervention. Comparing the results states that the proposed method accuracy is better than SVM's classification methods, ANN. The prediction analysis depicts that the RAS method can effectively detect the damage levels.

In this work, a Recurrent Neural Network (RNN) is trained using cepstral features and a set of difference cepstral feature (DCF) vectors on a frame to frame basis. The DCF vector is formulated to capture the temporal patterns of fricative sounds or phonemes of Assamese language. A hybrid algorithm is developed to recognize these fricative phonemes from certain words containing them. To preserve the temporal information of the speech segment, we here consider a frame-based hybrid approach to recognize fricatives from Assamese speech. A hybrid feature set is developed where simple frame-based feature is combined with differential frame-based feature. Investigation of feature extraction techniques like Linear Predictive Cepstral Coefficient (LPCC) and Mel-Frequency Cepstral Coefficient (MFCC) have been carried out and their performances have been evaluated in comparison to that obtained from the DCF set for Assamese fricative phoneme recognition. Here, speech segment is divided into 20 millisecond frames with overlap of 10 millisecond to extract features. Also, difference of a current frame with its preceding and succeeding frame is considered for forming a more accurate dynamic approach for fricative recognition. The differential processing enables to reduce correlation and retain only the most relevant portion of the input. After obtaining the feature vectors, Self Organizing Map (SOM) has been used to categorize the related features into different classes and remove repeating data. Thus the features obtained from the phoneme signal has been reduced into different sized cluster centres provided by SOM. The reduced feature vector is next applied to the RNN based hybrid classifier for learning the pattern and recognizing any unknown fricative segment.

Estimating impervious surfaces from medium spatial resolution imagery using the self-organizing map and multi-layer perceptron neural networks

In the electric industry, electricity loads forecasting has become more and more important, because electricity demand quantity is a major determinant factor in electricity supply strategy. Furthermore, accurate regional electricity loads forecasting is one of the principal factors for the electric industry to improve the energy management performance. Recently, statistical methods and artificial intelligence have been developed for electricity loads forecasting. However, there are two drawbacks in the past electricity loads forecasting models: (1) conventional statistical methods, such as regression models are unable to deal with the nonlinear relationships well, because electricity loads forecasting are known to be the nonlinear relationships; and (2) artificial intelligence technologies (i.e., support vector machines (SVM) and convention artificial neural networks (ANNs)) do not take into account time series causality for regional electricity loads forecasting. Based on these reasons above, this study proposes a new electricity loads forecasting model, which incorporates one step-ahead concept into recurrent neural network (RNN) to build a hybrid RNN model. The time series method considered from RNN model, which can be fitted to time series electricity loads datasets, and the neural network in hybrid RNN model can deal with the nonlinear relationships. For evaluating electricity loads forecasting performance, six different electricity loads forecasting models (RSVMG, ANN, Regression, HEFST, ANFIS and AR(2)) are used as comparison models. The research results indicate that the proposed electricity loads forecasting model is superior to the listing comparison models in terms of mean absolute percentage errors (MAPE).

36 - Time-Series Prediction

Ability to organize data spatially while conserving the topological relation between data features makes the Self Organizing Map (SOM) a very useful tool for analysis and visualization of high dimensional data such as a power transformer's Dissolved Gas Analysis (DGA). Past SOM application required large historical data for its training and has limited fault detection sensitivity. In this paper, the effects of input features and data normalization are studied to enhance SOM's clustering. SOM is trained using DGA results extracted from actual faulted transformers. Combination of input features and data normalization methods are tested on SOM before the best SOM is identified. Validation is conducted using several datasets i.e. the IEC Technical Committee 10 database. Compared with past SOM applications, the proposed SOM required lesser training data, improved SOM's sensitivity in incipient fault detection and has good diagnosis accuracy. The proposed SOM is also compared with other AI-based DGA interpretation method i.e. Support Vector Machine (SVM) for benchmarking.

In this work, we propose a solution for the recognition of protein folds using Self-Organizing Map (SOM) neural network and present a comparison between few approaches. We use SOM, Fisher’s Linear Discriminant Analysis (FLD), K-Nearest Neighbor (KNN), Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP) methods for the recognition of three SCOP folds with six attributes (amino acid composition, predicted secondary structure, hydrophobicity, normalized van der Waals volume, polarity and polarizability). Then we classify the most common 27 SCOP folds using 125 dimensional data formed by the six attributes. This paper has a novelty in the way of applying SOM to these six attributes, and also portrays the capabilities of SOM among the other methods in protein fold classification. Firstly for the threeclass problem, the methods are tested on 120 proteins by applying 10-fold cross-validation technique and 93.33% classification performance is obtained with SOM. Secondly for the 27-class problem SOM is tested on 694 proteins by applying one-versus-others technique and 73.37% classification performance is obtained. Keywords: Protein fold recognition, protein fold classification, neural networks, self-organizing map.

The prediction of meteorological time series plays very important role in several fields. In this paper, an application of least squares support vector machine (LS-SVM) for short-term prediction of meteorological time series (e.g. solar irradiation, air temperature, relative humidity, wind speed, wind direction and pressure) is presented. In order to check the generalization capability of the LS-SVM approach, a K-fold cross-validation and Kolmogorov–Smirnov test have been carried out. A comparison between LS-SVM and different artificial neural network (ANN) architectures (recurrent neural network, multi-layered perceptron, radial basis function and probabilistic neural network) is presented and discussed. The comparison showed that the LS-SVM produced significantly better results than ANN architectures. It also indicates that LS-SVM provides promising results for short-term prediction of meteorological data.

An accurate vertical wind speed (WS) data estimation is required to determine the potential for wind farm installation. In general, the vertical extrapolation of WS at different heights must consider different parameters from different locations, such as wind shear coefficient, roughness length, and atmospheric conditions. The novelty presented in this article is the introduction of two steps optimization for the Recurrent Neural Networks (RNN) model to estimate WS at different heights using measurements from lower heights. The first optimization of the RNN is performed to minimize a differentiable cost function, namely, mean squared error (MSE), using the Broyden-Fletcher-Goldfarb-Shanno algorithm. Secondly, the RNN is optimized to reduce a non-differentiable cost function using simulated annealing (RNN-SA), namely mean absolute error (MAE). Estimation of WS vertically at 50 m height is done by training RNN-SA with the actual WS data a 10–40 m heights. The estimated WS at height of 50 m and the measured WS at 10–40 heights are further used to train RNN-SA to obtain WS at 60 m height. This procedure is repeated continuously until the WS is estimated at a height of 180 m. The RNN-SA performance is compared with the standard RNN, Multilayer Perceptron (MLP), Support Vector Machine (SVM), and state of the art methods like convolutional neural networks (CNN) and long short-term memory (LSTM) networks to extrapolate the WS vertically. The estimated values are also compared with real WS dataset acquired using LiDAR and tested using four error metrics namely, mean squared error (MSE), mean absolute percentage error (MAPE), mean bias error (MBE), and coefficient of determination (). The numerical experimental results show that the MSE values between the estimated and actual WS at 180 m height for the RNN-SA, RNN, MLP, and SVM methods are found to be 2.09, 2.12, 2.37, and 2.63, respectively.

Concrete pavement joint evaluation involves a number of assessment criteria, such as deflection near the joint, load transfer efficiency of the dowels and severity of voids under the slab. Although there are well defined thresholds for each of these parameters, often there arises a situation where each of the considered parameters lends contradictory assessment that leads to a considerable subjectivity in the evaluation process. A Self-Organizing Map (SOM), an unsupervised learning procedure in artificial neural network, is utilised for the first time to map the joint condition of concrete pavements from Falling Weight Deflectometer (FWD) deflection bowls. A novel methodology is proposed for labelling the network, whereby pavement engineering expertise can be directly used in a SOM for consistent deflection data classification in joint evaluation. The effectiveness of the trained network is demonstrated by using joint assessment parameters; namely, load transfer efficiency (LTE), void intercepts and absolute deflection. The joints were classified as good, marginal or poor. For the three parameters based SOM classification, an accuracy of 65-70% was obtained; this improves to 87.5% when the SOM was trained with 2-parameters (LTE and absolute deflection). However, when the SOM was tested with the data classified as 'good', accuracy improves to around 90%. Therefore, a SOM can be a powerful supplementary tool for a consistent and non-subjective evaluation of concrete pavement joints.