Large-scale image retrieval using transductive support vector machines

Classifying the heterogeneous classes present in the hyper spectral image is one of the recent research issues in the field of remote sensing. The classification accuracy can be improved if and only if both the feature extraction and classifier selection are proper. As the classes present in the hyper spectral image are having different textures, textural classification is entertained. Wavelet based textural feature extraction is entailed. As hyper spectral images are having dozen numbers of bands, few bands are selected and wavelet transform is applied. For each of the sub band Gray Level Co-occurrence Matrix (GLCM) are calculated. From GLCMs co-occurrence features are derived for individual pixels. Apart from Co-occurrence features, statistical features are also calculated. Addition of statistical and co-occurrence features of individual pixels at other individual bands form new features. By the process of adding these new features of approximation band and individual sub-bands at the pixel level, Combined Features are derived. These Combined Features are used for classification. Support Vector Machines with Binary Hierarchical Tree (BHT) classifier is developed to classify the data by One Against All(OAA) methodology. Airborne Visible Infra Red Imaging Sensor (AVIRIS) image of Cuprite - Nevada field is inducted for the experiment and the results are compared with the ground truth and with the maximum likelihood classifier output which available in HIAT toolbox.

To solve the problem that traditional multi-label support vector machine (SVM) classification algorithm adopting nonlinear kernel has been severely restricted from being used on large-scale data sets, we propose fast multi-label low-rank-linearized SVM classification algorithm based on approximate extreme points (AEML-LLSVM). First, it adopts the approximate extreme points' method to obtain representative sets from the training data set. Then, the approximate extreme points' low-rank-linearized SVM (AELLSVM) is trained on the representative sets. The AELLSVM integrates the advantages of approximate extreme points' method and LLSVM. Experimental results on three large-scale multi-label data sets have proven that the training and the testing speed of AEML-LLSVM classification algorithm are greatly improved under the premise that its classification performance is similar to that of ML-LIBSVM classification algorithm and superior to that of other fast multi-label SVM classification algorithms.

In conventional distributed machine learning methods, distributed support vector machines (SVM) algorithms are trained over pre-configured intranet/internet environments to find out an optimal classifier. These methods are very complicated and costly for large datasets. Hence, we propose a method that is referred as the Cloud SVM training mechanism (CloudSVM) in a cloud computing environment with MapReduce technique for distributed machine learning applications. Accordingly, (i) SVM algorithm is trained in distributed cloud storage servers that work concurrently; (ii) merge all support vectors in every trained cloud node; and (iii) iterate these two steps until the SVM converges to the optimal classifier function. Single computer is incapable to train SVM algorithm with large scale data sets. The results of this study are important for training of large scale data sets for machine learning applications. We provided that iterative training of splitted data set in cloud computing environment using SVM will converge to a global optimal classifier in finite iteration size.

This aimed at the problem of insufficient samples of training datasets for deep convolutional neural networks in computer vision tasks, which may easily lead to over-fitting, a Gaussian Adjusted Local Dynamic Transform (GA-LDT) data enhancement algorithm based on Gaussian adjustment was proposed. The distribution characteristics of the function perform a local dynamic transformation on the image, enhance the difference and diversity of image information, and alleviate the problem of overfitting. To verify the algorithm's effectiveness, comparative experiments on ResNet, Yolox, and Espnet networks suitable for different tasks are carried out on CIFAR-100, CIFAR-10, Pascal VOC, and Cityscapes data sets, respectively. After using the GA-LDT algorithm, the accuracy of the ResNet network on CIFAR-100 and CIFAR-10 data sets has increased by 3.86% and 7.61%, respectively, year-on-year. Yolox network uses GA-LDT algorithm, and the mAP on Pascal VOC dataset is increased by 1.86% year-on-year, while the mloU on Cityscapes dataset of Espnet network using GA-LDT algorithm is increased by 0.51% year-on-year.

PurposeThe purpose of this study is to explore the potential of trainable activation functions to enhance the performance of deep neural networks, specifically ResNet architectures, in the task of image classification. By introducing activation functions that adapt during training, the authors aim to determine whether such flexibility can lead to improved learning outcomes and generalization capabilities compared to static activation functions like ReLU. This research seeks to provide insights into how dynamic nonlinearities might influence deep learning models' efficiency and accuracy in handling complex image data sets.Design/methodology/approachThis research integrates three novel trainable activation functions – CosLU, DELU and ReLUN – into various ResNet-n architectures, where “n” denotes the number of convolutional layers. Using CIFAR-10 and CIFAR-100 data sets, the authors conducted a comparative study to assess the impact of these functions on image classification accuracy. The approach included modifying the traditional ResNet models by replacing their static activation functions with the trainable variants, allowing for dynamic adaptation during training. The performance was evaluated based on accuracy metrics and loss profiles across different network depths.FindingsThe findings indicate that trainable activation functions, particularly CosLU, can significantly enhance the performance of deep learning models, outperforming the traditional ReLU in deeper network configurations on the CIFAR-10 data set. CosLU showed the highest improvement in accuracy, whereas DELU and ReLUN offered varying levels of performance enhancements. These functions also demonstrated potential in reducing overfitting and improving model generalization across more complex data sets like CIFAR-100, suggesting that the adaptability of activation functions plays a crucial role in the training dynamics of deep neural networks.Originality/valueThis study contributes to the field of deep learning by introducing and evaluating the impact of three novel trainable activation functions within widely used ResNet architectures. Unlike previous works that primarily focused on static activation functions, this research demonstrates that incorporating trainable nonlinearities can lead to significant improvements in model performance and adaptability. The introduction of CosLU, DELU and ReLUN provides a new pathway for enhancing the flexibility and efficiency of neural networks, potentially setting a new standard for future deep learning applications in image classification and beyond.

It is very crucial for large-scale image retrieval tasks to extract effective hash feature representations. Encouraged by the recent advances in convolutional neural networks (CNNs), this paper presents a novel cascaded deep hashing (CDH) method to generate compact hash codes for highly efficient image retrieval tasks on given large-scale datasets. Specifically, we ingeniously utilize three CNN models to learn robust image feature representations on a given dataset, which solves the issue that categories with poor feature representation have a fairly low retrieval precision. Experimental results indicate that CDH outperforms some state-of-the-art hashing algorithms on both CIFAR-10 and MNIST datasets.

This thesis presents a critical appraisal of several differing approaches to the design and testing of fault detection (FD) algorithms monitoring the instrumentation used in the alternated aerobic/anoxic cycles (AC) process for nitrogen removal. Several features are selected as the basis of the FD, involving the slope and the timing of the process measurements of the various nitrogen compounds. Two separate FD algorithms have been developed for the anoxic and the aerobic phases, requiring a separate tuning but sharing the same principles: first some low-level checks are performed on the raw signals, discriminating gross malfunctions like missing data and spikes, then more sophisticated methods are used to investigate the presence of more subtle anomalies that were not detected by the previous screening. The FD problem is treated either in terms of classification problem, testing different algorithms such as binary trees, support vector machines (SVM) and principal component analysis (PCA), or as a forecasting one, using the Bayesian theory to predict the faulty or normal state of the process based on the previous records. An operational data set obtained from a municipal plant was used to first train the algorithm. However, due to the fairly limited information which could be extracted, a more comprehensive data set was created building an AC model based on the standard Benchmark simulation model with improved nitrogen kinetics and seasonal temperature variations. Detailed sensor models were also included, so that the occurrence of faults could be totally controlled, both in kind and timing. The performances of the various methods on either the operational and the synthetic datasets have been assessed comparing the anomalies detected by the methods with those actually observed. While the great majority of the gross faults is successfully detected by the preliminary screening, differing performances of the subsequent finer detection are obtained, depending on both the quality of data set and the detection method used: poorer results are observed using the plant data, in part due to an insufficient characterization of the fault events and in part due to the limited number of signals monitored. The higher availability of measurements provided by the numerical model, instead, enhances the discrimination capabilities of the tested methods, especially the nonlinear SVM, while the PCA-based approach and the Bayesian predictor results less affected by a change in the combination of diagnostic parameters used.

Cancer survival classification using integrated data sets and intermediate information.

Content-based image retrieval (CBIR) methods search for points with the most similar content to query features from within a large dataset. The most notable approach for this purpose is an approximate nearest neighbor (ANN) searching. The main properties expected from a retrieval system can be listed as follows; low storage requirement, high retrieve speed, and high average precision. Hashing, which can generate discriminative and low-dimensional binary codes, is one of today’s most effective ANN searching methods. Although there are various hashing approaches in the literature, almost all hashing approaches consist of low-dimension feature representation and binarization sections. This study focuses on the low-dimension feature representation. Hand-crafted or deep learning based approaches are used for feature extraction in hashing methods. These features are the main components that affect the performance in creating binary codes. Contrastive loss is often used in the literature to update the learnable parameters of these feature extraction algorithms. The distance parameter of these data points critical to calculating contrastive loss. In this study, contrastive loss performance is tested using five different distance methods (Euclidean, Manhattan, Cosine, Minkowski, Chebyshev) for more effective feature representation. Retrieval performance is tested using low-dimensional feature vectors produced by these methods in MNIST and CIFAR-10 datasets. It is thought that the information obtained from this study is very useful for new researchers.

Purpose: To investigate an approach of automatically recognizing anatomical sites and imaging views (the orientation of the image acquisition) in 2D X-ray images. Methods: A hierarchical (binary tree) multiclass recognition model was developed to recognize the treatment sites and views in x-ray images. From top to bottom of the tree, the treatment sites are grouped hierarchically from more general to more specific. Each node in the hierarchical model was designed to assign images to one of two categories of anatomical sites. The binary image classification function of each node in the hierarchical model is implemented by using a PCA transformation and a support vector machine (SVM) model. The optimal PCA transformation matrices and SVM models are obtained by learning from a set of sample images. Alternatives of the hierarchical model were developed to support three scenarios of site recognition that may happen in radiotherapy clinics, including two or one X-ray images with or without view information. The performance of the approach was tested with images of 120 patients from six treatment sites – brain, head-neck, breast, lung, abdomen and pelvis – with 20 patients per site and two views (AP and RT) per patient. Results: Given two images in known orthogonal views (AP and RT), the hierarchical model achieved a 99% average F1 score to recognize the six sites. Site specific view recognition models have 100 percent accuracy. The computation time to process a new patient case (preprocessing, site and view recognition) is 0.02 seconds. Conclusion: The proposed hierarchical model of site and view recognition is effective and computationally efficient. It could be useful to automatically and independently confirm the treatment sites and views in daily setup x-ray 2D images. It could also be applied to guide subsequent image processing tasks, e.g. site and view dependent contrast enhancement and image registration. The senior author received research grants from ViewRay Inc. and Varian Medical System.

The scanning N-tuple classifier (SNT) is a fast and accurate method for classifying sequences. Applications include both on-line and off-line hand-written character recognition. SNTs have conventionally been trained using maximum likelihood parameter estimation. This paper describes a discriminative training rule that can be applied to ensembles of SNTs. Results demonstrate a significant improvement for the discriminative ensemble method. For comparison purposes we also implemented a support vector machine (SVM) operating in the sequence domain. We tested each method on a chain-coded version of the MNIST hand-written digit dataset. The SNT is not quite as accurate as the SVM, but is much faster both in training and recognition.

In the field of fast and large-scale image retrieval, the binary hash code based on convolutional features enjoys powerful advantages. CNN model's leading recognition ability shows that its high-level features distribute intensively and have stable between-class distances. On the basis of the distribution characteristic of convolutional features, this paper presents a new method of improving retrieval performance by means of using the category centers of high-level features to enhance the common semantic content of high-level features of the query image and making the consequent binary hash code gain better retrieval results. Experimental results indicate that (i) the category centers of CNN feature offer comparable discriminability with the initial model when identifying the semantic information of images; (ii) before calculating binary hash code, the integration of the CNN feature of a query image with category centers weighted by predicted probabilities can boost performance in similar image retrieval. State-of-the-art retrieval performance is obtained on the Cifar-10 dataset in this paper among all hash methods.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

In large-scale image retrieval, deep features extracted by Convolutional Neural Network (CNN) can effectively express more image information than those extracted by traditional manual methods. However, the deep feature dimensions obtained by Deep Convolutional Neural Network (DCNN) are too high and redundant, which leads to low retrieval efficiency. We propose a novel image retrieval method, which combines deep features selection with improved DCNN and hash transform based on high-dimension features reduction to gain lowdimension deep features and realizes efficient image retrieval. Firstly, the improved network is based on the existing deep model to build a more profound and broader network by adding multiple groups of different branches. Therefore, it is named DFS-Net (Deep Feature Selection Network). The adaptive learning deep features of the Network can effectively alleviate the influence of over-fitting and improve the feature expression of image content. Secondly, the information gain rate method is used to filter the extracted deep features to reduce the feature dimension and ensure the information loss is small. The last step of the method, hash Transform, sparsifies and binarizes this representation to reduce the computation and storage pressure while maintaining the retrieval accuracy. Finally, the scheme is based on the distinguished ResNet50, InceptionV3, and MobileNetV2 models, and studied and evaluated deeply on the CIFAR10 and Caltech256 datasets. The experimental results show that the novel method can train the deep features with stronger recognition ability on limited training samples, and improve the accuracy and efficiency of image retrieval effectively.

Support Vector Machine (SVM) is mainly used to classify the data into two categories. To solve the multi-category problems using SVM, researchers used two approaches. The first approach based on solving multiple SVM binary classifiers, whereas another approach based on solving a single optimization problem. In this paper, we have used the first approach and proposed an Efficient Multiclass Support Vector Machine (ESVM) algorithm using a skewed binary tree. To construct the skewed binary tree, no extra efforts are required as compared to the binary tree approach. The algorithm is tested on the benchmark data sets, and the results are compared with both the multiclass approaches of SVM. The ESVM’s results are compared with five techniques of solving multiple binary SVM classifiers and four techniques of solving a single optimization problem. The comparative experiments prove the efficiency of the ESVM in terms of its accuracy as compared to other contemporary algorithms. Further, ESVM is successfully applied for classification of the email dataset into positive, negative and neutral sentiments.