A Comparison of Machine Learning Techniques to Extract Human Settlements from High Resolution Imagery

Palm tree detection using remote sensing images has received increasing attention in recent years, concerning the issues of sustainability, productivity and profitability. There has been significant progress in the research of using machine learning techniques, especially convolutional neural networks (CNNs) for automatic palm tree detection. However, whether CNNs can actually outperform traditional human-engineered approaches in terms of classification accuracy and detection speed is yet unknown. In the present study, we have compared human-engineered features namely histogram of oriented gradients (HOG), local binary pattern (LBP) and scale-invariant feature transform (SIFT) with features extracted using pre-trained AlexNet model for detecting palm trees in high resolution images obtained via unmanned aerial vehicle (UAV). Support vector machines (SVM) with linear and non-linear kernels were used to classify feature vectors obtained by different feature extractors. Haar-like features used in Viola-Jones framework was also tested in the study. Results showed that features extracted from the fifth convolutional layer of AlexNet achieved the highest accuracy of 96.1% using SVM with RBF kernel as classifier, which surpassed the accuracy obtained by fully-connected CNN, namely 95.6%. The results also suggest that SVM classifier with radial basis function (RBF) kernel using LBP as features is the optimal combination as it achieved comparable accuracy but higher detection speed than CNN approaches.

Landsat-like moderate resolution remote sensing images are widely used in land use and land cover (LULC) classification. Limited by coarser resolutions, most of the traditional LULC classifications that are based on moderate resolution remote sensing images focus on the spectral features of a single pixel. Inspired by the spatial evaluation methods in landscape ecology, this study proposed a new method to extract neighborhood characteristics around a pixel for moderate resolution images. 3 landscape-metric-like indexes, i.e., mean index, standard deviation index, and distance weighted value index, were defined as adjacent region features to include the surrounding environmental characteristics. The effects of the adjacent region features and the different feature set configurations on improving the LULC classification were evaluated by a series of well-controlled LULC classification experiments using K nearest neighbor (KNN) and support vector machine (SVM) classifiers on a Landsat 8 Operational Land Imager (OLI) image. When the adjacent region features were added, the overall accuracies of both the classifiers were higher than when only spectral features were used. For the KNN and SVM classifiers that used only spectral features, the overall accuracies of the LULC classification were 85.45% and 88.87%, respectively, and the accuracies were improved to 94.52% and 96.97%. The classification accuracies of all the LULC types improved. Highly heterogeneous LULC types that are easily misclassified achieved greater improvements. As comparisons, the grey-level co-occurrence matrix (GLCM) and convolutional neural network (CNN) approaches were also implemented on the same dataset. The results revealed that the new method outperformed GLCM and CNN approaches and can significantly improve the classification performance that is based on moderate resolution data.

Semantic segmentation of remote sensing images enables in particular land-cover map generation for a given set of classes. Very recent literature has shown the superior performance of deep convolutional neural networks (DCNN) for many tasks, from object recognition to semantic labelling, including the classification of Very High Resolution (VHR) satellite images. However, while plethora of works aim at improving object delineation on geographically restricted areas, few tend to solve this classification task at very large scales. New issues occur such as intra-class class variability, diachrony between surveys, and the appearance of new classes in a specific area, that do not exist in the predefined set of labels. Therefore, this work intends to (i) perform large scale classification and to (ii) expand a set of land-cover classes, using the off-the-shelf model learnt in a specific area of interest and adapting it to unseen areas.

Studying the motion of some roundworms types such as Caenorhabditis elegans (CE) is important to identify the actions and reactions and their effects of worm’s life. In this study, the time series of CE motion represented by the angles of wave-motion between 1 to 177 degrees will be the case study. Each observation of this time series is a recorded frame (0.5 second) of 2.5 hours video of CE motion. A convolutional neural network (CNN) as one of deep learning techniques will be used to classify CE motion as dependent variable in binary cases based on the images of the angles of wave-motion as explanatory variable. The images of motion angles are imagined and designed by two dimensions image corresponding to every observation. These images combined into 4-d image (four dimensions matrix) to represent univariate explanatory variable. Support vector machine (SVM) will be also used to classify the angles of CE. In these types of data, the nonlinearity and uncertainty will be the most probably problems as reasons for in accurate classifications. CNN and SVM used with this type of dataset to improve the classification results. The results of comparisons explain that CNN approach outperforms SVM absolutely. In conclusion, CNN approach can be used to classify this type of time series with accurate results.

Multiple myeloma is a type of bone marrow cancer. Patient’s blood samples are analysed from protein gel strips and densitometer graphs which are then interpreted by a pathologist to diagnose multiple myeloma. This manual process of diagnosis is slow which is problematic as patients need to be diagnosed as soon as possible in order to prevent the condition from worsening, hence the need to automate this process. Given the success of machine learning in diagnosing diseases from images, this study investigates the use of machine learning approaches for the diagnosis of multiple myeloma. This is the first study investigating the automation of this process and hence presents a novel application of machine learning. The study compares machine learning approaches, artificial neural networks, convolutional neural networks, random forests and support vector machines, for the diagnosis of multiple myeloma from gel strips and densitometer graphs. The study has revealed that convolutional neural networks, specifically VGG16, is the most suitable approach for the detection of multiple myeloma.

Pluripotent stem cells can potentially be used in clinical applications as a model for studying disease progress. This tracking of disease-causing events in cells requires constant assessment of the quality of stem cells. Existing approaches are inadequate for robust and automated differentiation of stem cell colonies. In this study, we developed a new model of vector–based convolutional neural network (V-CNN) with respect to extracted features of the induced pluripotent stem cell (iPSC) colony for distinguishing colony characteristics. A transfer function from the feature vectors to the virtual image was generated at the front of the CNN in order for classification of feature vectors of healthy and unhealthy colonies. The robustness of the proposed V-CNN model in distinguishing colonies was compared with that of the competitive support vector machine (SVM) classifier based on morphological, textural, and combined features. Additionally, five-fold cross-validation was used to investigate the performance of the V-CNN model. The precision, recall, and F-measure values of the V-CNN model were comparatively higher than those of the SVM classifier, with a range of 87–93%, indicating fewer false positives and false negative rates. Furthermore, for determining the quality of colonies, the V-CNN model showed higher accuracy values based on morphological (95.5%), textural (91.0%), and combined (93.2%) features than those estimated with the SVM classifier (86.7, 83.3, and 83.4%, respectively). Similarly, the accuracy of the feature sets using five-fold cross-validation was above 90% for the V-CNN model, whereas that yielded by the SVM model was in the range of 75–77%. We thus concluded that the proposed V-CNN model outperforms the conventional SVM classifier, which strongly suggests that it as a reliable framework for robust colony classification of iPSCs. It can also serve as a cost-effective quality recognition tool during culture and other experimental procedures.

One of the most powerful ideas in deep learning is the transfer learning technique. Transfer learning can be utilized to take a knowledge from what a deep neural network has learned from a particular task and apply that knowledge to a different task. Transfer learning is very useful when the size of the training samples of interest is small to train a neural network from scratch. This research focuses on the concept of transfer learning where the Convolutional Neural Network (CNN) power can be utilized as a features extractor to help with classifying benign from malignant breast cancer images. In addition, Support Vector Machine (SVM) classifier based on Radial Basis Function (RBF) was adapted for its flexibility in fitting the data dimension space adequately by tuning the kernel width. The hybridization between CNN and RBF-Based SVM showed robust results for both the dataset and the application task of this research. The contribution of this paper can be summarized in three major parts. First, a CNN was implemented from scratch on a large number of available spine images to classify images of two different spine views (sagittal and axial) in order to transfer the learning process to the CNN of breast images. Second, retrain the spine CNN on the images of breast cancer to classify between benign and malignant cases by fine-tuning. Finally, the features were extracted from the retrained CNN and fed to RBF-Based SVM to classify benign from malignant breast mammogram images.

As the acquisition of very high resolution (VHR) satellite images becomes easier owing to technological advancements, ever more stringent requirements are being imposed on automatic image interpretation. Moreover, per-pixel classification has become the focus of research interests in this regard. However, the efficient and effective processing and the interpretation of VHR satellite images remain a critical task. Convolutional neural networks (CNNs) have recently been applied to VHR satellite images with considerable success. However, the prevalent CNN models accept input data of fixed sizes and train the classifier using features extracted directly from the convolutional stages or the fully connected layers, which cannot yield pixel-to-pixel classifications. Moreover, training a CNN model requires large amounts of labeled reference data. These are challenging to obtain because per-pixel labeled VHR satellite images are not open access. In this paper, we propose a framework called the unsupervised-restricted deconvolutional neural network (URDNN). It can solve these problems by learning an end-to-end and pixel-to-pixel classification and handling a VHR classification using a fully convolutional network and a small number of labeled pixels. In URDNN, supervised learning is always under the restriction of unsupervised learning, which serves to constrain and aid supervised training in learning more generalized and abstract feature. To some degree, it will try to reduce the problems of overfitting and undertraining, which arise from the scarcity of labeled training data, and to gain better classification results using fewer training samples. It improves the generality of the classification model. We tested the proposed URDNN on images from the Geoeye and Quickbird sensors and obtained satisfactory results with the highest overall accuracy (OA) achieved as 0.977 and 0.989, respectively. Experiments showed that the combined effects of additional kernels and stages may have produced better results, and two-stage URDNN consistently produced a more stable result. We compared URDNN with four methods and found that with a small ratio of selected labeled data items, it yielded the highest and most stable results, whereas the accuracy values of the other methods quickly decreased. For some categories with fewer training pixels, accuracy for categories from other methods was considerably worse than that in URDNN, with the largest difference reaching almost 10%. Hence, the proposed URDNN can successfully handle the VHR image classification using a small number of labeled pixels. Furthermore, it is more effective than state-of-the-art methods.

Identified mainly by memory loss and social inability, dementia may result from several different diseases. In the world with ever growing elderly population, the problem of dementia is rising. Despite being one of the prevalent mental health conditions in the community, it is not timely identified, reported and even understood completely. With the massive improvement in the computational power, researchers have developed machine learning (ML) techniques to diagnose and detect neurodegenerative diseases. This current work reports a comparative study of performance of several ML techniques, including support vector machine, logistic regression, artificial neural network, Naive Bayes, decision tree, random forest and K-nearest neighbor, when they are employed in identifying dementia from clinical datasets. It has been found that support vector machine and random forest perform better on datasets coming from open access repositories such as open access series of imaging studies, Alzheimer’s disease neuroimaging initiative and dementia bank datasets.

<p class="Normal1">Medicinal plants have been found and utilized in traditional medical practices from ancient times. Many medicinal plants play a vital role in curating many life threatening diseases. Very few of the medicinal herbs are commercially cultivated. Many plant diseases are there which destroys these medicinal plants. Early detection of plant diseases can prevent the huge loss of these medicinal plants. Here, we presented a hybrid model that makes use of SVM along with the traditional convolutional neural network (CNN) for predicting <em>Basil</em> plants leaves diseases. We transformed the conventional CNN model by adding a classification layer Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) after feature extraction and this approach tends to perform better than traditional CNN as we make the dataset balanced by data augmentation and SVN and KNN tend to perform better in case of balanced samples. CNN is used for training, SVM/KNN is used for classification. The advantages of CNN and SVM are used in proposed the CNN and SVM and KNN model. It is assumed that such a combined model would incorporate the benefits of CNN and SVM. Here, we identified the four types of diseases that affect <em>basil</em> plant leaves as Leaf spot, Downy mildew, Fusarium wilt, Fungal, and Healthy. Since there isn’t a standard dataset for <em>basil</em> leaves, we created our own 803 picture data set and used various machine learning techniques to train and evaluate the model. However, over other existing algorithms, our hybrid model i.e., CNN+SVM has produced more accurate results. For five classes of <em>basil</em> plant leaves, the proposed model produced 95.02% accuracy of for leaf diseases.</p>

Urban land cover and land use mapping plays an important role in urban planning and management. In this paper, novel multi-scale deep learning models, namely ASPP-Unet and ResASPP-Unet are proposed for urban land cover classification based on very high resolution (VHR) satellite imagery. The proposed ASPP-Unet model consists of a contracting path which extracts the high-level features, and an expansive path, which up-samples the features to create a high-resolution output. The atrous spatial pyramid pooling (ASPP) technique is utilized in the bottom layer in order to incorporate multi-scale deep features into a discriminative feature. The ResASPP-Unet model further improves the architecture by replacing each layer with residual unit. The models were trained and tested based on WorldView-2 (WV2) and WorldView-3 (WV3) imageries over the city of Beijing. Model parameters including layer depth and the number of initial feature maps (IFMs) as well as the input image bands were evaluated in terms of their impact on the model performances. It is shown that the ResASPP-Unet model with 11 layers and 64 IFMs based on 8-band WV2 imagery produced the highest classification accuracy (87.1% for WV2 imagery and 84.0% for WV3 imagery). The ASPP-Unet model with the same parameter setting produced slightly lower accuracy, with overall accuracy of 85.2% for WV2 imagery and 83.2% for WV3 imagery. Overall, the proposed models outperformed the state-of-the-art models, e.g., U-Net, convolutional neural network (CNN) and Support Vector Machine (SVM) model over both WV2 and WV3 images, and yielded robust and efficient urban land cover classification results.

A novel CS-NET architecture based on the unification of CNN, SVM and super-resolution spectrogram to monitor and classify blood pressure using photoplethysmography

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

The availability of the data has increased tremendously due to the excess usage of social media platforms like Twitter and Facebook. Due to the abundant availability of data, scientists, businesses, educationalists and other people working under different roles have started using Sentiment Analysis (SA) to get in-depth knowledge about the sentiments of the people regarding any topic of interest. There are many techniques to implement SA, and one of them is Machine Learning (ML). This study is focused on the comparison of ancient ML methods such as Naïve Bayes (NB), Decision Tree (DT), Support Vector Machine (SVM), and a modern method, i.e., Deep Learning (DL). The ML techniques are applied to a single dataset to compare their performance in terms of accuracy to understand how they perform against each other. The study found that DL performed the best with 96.41% accuracy followed by NB and SVM with 87.18% and 82.05% respectively. DT performed the poorest with 68.21% accuracy.

The aim of Information Lifecycle Management (ILM) is to govern data throughout its lifecycle as efficiently as possible and effectively from technical points of view. A core aspect is the question, where the data should be stored, since different costs and access times are entailed. For this purpose data have to be classified, which presently is either done manually in an elaborate way, or with recourse to only a few data attributes, in particular access frequency. In the context of Data-Warehouse-Systems this article introduces an automated and therefore speedy and cost-effective data classification for ILM. Machine learning techniques, in particular an artificial neural network (multilayer perceptron), a support vector machine and a decision tree approach are compared on an SAP-based real-world data set from the automotive industry. This data classification considers a large number of data attributes and thus attains similar results akin to human experts. In this comparison of machine learning techniques, besides the accuracy of classification, also the types of misclassification that appear, are included, since this is important in ILM.