Overlooked pitfalls in multi-class machine learning classification in radiation oncology and how to avoid them

A combinatorial optimization approach for multi-label associative classification

Convolutional Neural Networks (CNNs) have proven to be very effective in image classification due to their status as a powerful feature learning algorithm. Traditional approaches have considered the problem of multiclass classification, where the goal is to classify a set of objects at once. However, co-occurrence can make the discriminative features of the target less salient and may lead to overfitting of the model, resulting in lower performance. To address this, we propose a multi-label classification ensemble model including a Vision Transformer (ViT) and CNN for directly detecting one or multiple objects in an image. First, we improve the MobileNetV2 and DenseNet201 models using extra convolutional layers to strengthen image classification. In detail, three convolution layers are applied in parallel at the end of both models. ViT can learn dependencies among distant positions and local detail, making it an effective tool for multi-label classification. Finally, an ensemble learning algorithm is used to combine the classification predictions of the ViT, the modified MobileNetV2, and DenseNet201 bands for increased image classification accuracy using a voting system. The performance of the proposed model is examined on four benchmark datasets, achieving accuracies of 98.24%, 98.89%, 99.91%, and 96.69% on ASCAL VOC 2007, PASCAL VOC 2012, MS-COCO, and NUS-WIDE 318, respectively, showing that our framework can enhance current state-of-the-art methods.

If their assumptions are not met, classifiers may fail. In this paper, the possibility of combining classifiers in multi-class problems is investigated. Multi-class classification problems are split into two class problems. For each of the latter problems an optimal classifier is determined. The results of applying the optimal classifiers on the two class problems can be combined using the Pairwise Couplingalgorithm by Hastie and Tibshirani (1998). In this paper exemplary situations are investigated where the respective assumptions of Naive Bayes or the classical Linear Discriminant Analysis (LDA, Fisher, 1936) fail. It is investigated at which degree of violations of the assumptions it may be advantageous to use single methods or a classifier combination by Pairwise Coupling.

In many real-world applications, it is common to have uneven number of examples among multiple classes. The data imbalance, however, usually complicates the learning process, especially for the minority classes, and results in deteriorated performance. Boosting methods were proposed to handle the imbalance problem. These methods need elongated training time and require diversity among the classifiers of the ensemble to achieve improved performance. Additionally, extending the boosting method to handle multi-class data sets is not straightforward. Examples of applications that suffer from imbalanced multi-class data can be found in face recognition, where tens of classes exist, and in capsule endoscopy, which suffers massive imbalance between the classes. This dissertation introduces RegBoost, a new boosting framework to address the imbalanced, multi-class problems. This method applies a weighted stratified sampling technique and incorporates a regularization term that accommodates multi-class data sets and automatically determines the error bound of each base classifier. The regularization parameter penalizes the classifier when it misclassifies instances that were correctly classified in the previous iteration. The parameter additionally reduces the bias towards majority classes. Experiments are conducted using 12 diverse data sets with moderate to high imbalance ratios. The results demonstrate superior performance of the proposed method compared to several state-of-the-art algorithms for imbalanced, multi-class classification problems. More importantly, the sensitivity improvement of the minority classes using RegBoost is accompanied with the improvement of the overall accuracy for all classes. With unpredictability regularization, a diverse group of classifiers are created and the maximum accuracy improvement reaches above 24%. Using stratified undersampling, RegBoost exhibits the best efficiency. The reduction in computational cost is significant reaching above 50%. As the volume of training data increase, the gain of efficiency with the proposed method becomes more significant.

Error Correcting Output Codes reveal an efficient strategy in dealing with multi-class classification problems. According to this technique, a multi-class problem is decomposed into several binary ones. On these created sub-problems we apply binary classifiers and then, by combining the acquired solutions, we are able to solve the initial multi-class problem. In this paper we consider the optimization of the Linear Discriminant Error Correcting Output Codes framework using Particle Swarm Optimization. In particular, we apply the Particle Swarm Optimization algorithm in order to optimally select the free parameters that control the split of the initial problem’s classes into sub-classes. Moreover, by using the Support Vector Machine as classifier we can additionally apply the Particle Swarm Optimization algorithm to tune its free parameters. Our experimental results show that by applying Particle Swarm Optimization on the Sub-class Linear Discriminant Error Correcting Output Codes framework we get a significant improvement in the classification performance.

In multiclass classification problems we face the challenge of having many binary classifiers. Consulting this large number of classifiers might be confusing and time consuming. In this paper, we propose a new framework for training and prediction in multiclass problems. In this framework, we perform traditional training. Next we map training examples to prediction models. Finally we produce the Example Classifier (EC). In prediction a new example is passed through the EC to determine the appropriate classifier which in turn makes the last prediction decision. We conduct experiments comparing our framework with one-VS-one and Directed Acyclic Graph (DAG) using Support Vector Machines. Additionally, we compare our model with well-known ensemble models, namely, AdaBoost and Bagging, Our results indicate that prediction accuracy is comparable to other methodologies with the advantage of consuming less prediction time.

Improved multiclass feature selection via list combination

Support vector machine (SVM) learns the maximum margin to separate training examples that belong to two classes and has been widely used in many pattern recognition tasks due to its high effectiveness. However, conventional SVM suffers from the following deficiencies: (1) SVM cannot take full advantage of multiple modalities in a dataset if they are available, and (2) SVM trains the marginal hyper-plane by solving the quadratic programming problem, and thus costs too much computational overheads. In this paper, we propose a correlation maximization machine (CMM) model to overcome the aforementioned deficiencies by integrating two modalities in a dataset to boost the classification performance and utilizing randomized nonlinear features to output labels from multiple classes. In particular, CMM reveals the nonlinear relationships among both modalities by generating randomized nonlinear features for each modality. CMM learns to project these features into a common subspace with a constraint that their coefficients are highly correlated, and narrows the gap between the coefficients of both modalities and the class indicator of each training example to deal with multiclass classification problem. At the classification stage, CMM indicates the classes of a test example by using the summation of its coefficients of both modalities. Since the objective function of CMM is non-convex, it is quite difficult to obtain the global minimum. In this paper, we developed a block coordinate descent-based algorithm to optimize CMM and theoretically proved its convergence to a local minimum. Experimental results of face recognition on three popular face image datasets and experimental results of image retrieval on CIFAR-10, NUS-Wide, and Wikipedia datasets demonstrate that CMM outperforms the representative methods.

Grid computing is employed to unravel massive computational problems by using large numbers of heterogeneous computers connected to the computing network. Job scheduling is an important part of the grid computing environment, which is employed to extend the throughput and reduce the turnaround and reaction time. This paper proposed a new scheduling algorithm called "Feed forward neural network in the grid computing (FFNNGC) system," which is used to solve some real-life problems related to the pattern classification. In the proposed method, we have used a feed-forward algorithm to find the output in the grid computing network, and the network training is done until the system converges to a minimum error solution. The pattern classification problem consists of 13 real-life, and artificial dataset problems, including two class and multiclass problems. Experiments were performed under these real-life problems, and the results indicated that the proposed method is helpful in such types of problems.

In this paper, we investigate the robust single-label multi-class classification problems in machine learning using the proposed linearly scored categorical cross-entropy for training data with wrong class labels. Deep neural networks are constructed and trained using different loss functions with various noise levels. CIFAR10 and CIFAR100 image datasets are used to calculate the estimated classification accuracy performances via 10-fold cross validation. From the simulation results, the proposed loss function may actually provide a promising alternative for robust multi-class classification problems.

Deep learning has shown promising results in various text-based classification tasks. However, deep learning performance is affected by the number of data, i.e., when the number of data is small, deep learning algorithms do not perform well, and vice versa. Classical machine learning algorithms commonly work well for a few data, and their performance reaches an optimal value and does not increase with the increase in sample data. Therefore, this study aimed to compare the performance of classical machine learning and deep learning methods to detect temperament based on Indonesian Twitter. In this study, the proposed Indonesian Linguistic Inquiry and Word Count were employed to analyze the context of Twitter. The classical machine learning methods implemented were support vector machine and K-nearest neighbor, whereas the deep learning method employed was a convolutional neural network (CNN) with three different architectures. Both learning methods were implemented using multiclass classification and one versus all (OVA) multiclass classification. The highest average f-measure was 58.73%, obtained by CNN OVA with a pool size of 3, a dropout value of 0.7, and a learning rate value of 0.0007.

In this work we address the problem of multi-class classification in machine learning. In particular, we consider the coding approach which converts a multi-class problem to several binary classification problems by mapping the binary labeled space into several partitioned binary labeled spaces through binary channel codes. By modeling this learning problem as a communication channel, these codes are meant to have error correcting capabilities and thus performance improvement in classification. However, we argue that conventional coding schemes designed for communication systems do not treat the space partitioning problem optimally, because they are heedless of the partitioning behavior of underlying binary classifiers. We discuss an approach which is optimal in terms of space partitioning and advise it as a powerful tool towards multi-class classification. We then review the LDA, a known method for multi-class case and compare its performance with the proposed method. We run the experiments on synthetic data in several scenarios and then on a real database for face identification.KeywordsMulti-Class ClassificationError Correcting Output CodesSupport Vector MachinesLinear Discriminant Analysis

In this paper, link adaptation (LA) schemes based on adaptive modulation (AM), where the transmit parameters can be adapted to the varying channel conditions, are investigated for spatial modulation (SM). We propose machine learning (ML) based adaptive spatial modulation (ASM) as the optimal technique for modulation selection (MS) compared with the exhaustive search based ASM scheme. Specifically, two supervised machine learning classifiers (MLCs), K-nearest neighbors (KNN) and support vector machine (SVM), are proposed to design the MS in ASM and make the tradeoff between the system complexity and the bit error rate (BER) performance. The proposed classifiers are employed to solve the MS problem in ASM as a multi-class classification problem and obtain the best modulation selection candidate (MSC) that achieves the maximum minimum Euclidean distance. Simulation results show that, under the same spectral efficiency (SE), the proposed schemes provide considerable system complexity reduction and obtain a sub-optimal BER performance compared with the typical ASM scheme.

Reinforced knowledge distillation: Multi-class imbalanced classifier based on policy gradient reinforcement learning

The present study aimed to assess the use of technical-tactical variables and machine learning (ML) classifiers in the automatic classification of the passing difficulty (DP) level in soccer matches and to illustrate the use of the model with the best performance to distinguish the best passing players. We compared eight ML classifiers according to their accuracy performance in classifying passing events using 35 technical-tactical variables based on spatiotemporal data. The Support Vector Machine (SVM) algorithm achieved a balanced accuracy of 0.70 ± 0.04%, considering a multi-class classification. Next, we illustrate the use of the best-performing classifier in the assessment of players. In our study, 2,522 pass actions were classified by the SVM algorithm as low (53.9%), medium (23.6%), and high difficulty passes (22.5%). Furthermore, we used successful rates in low-DP, medium-DP, and high-DP as inputs for principal component analysis (PCA). The first principal component (PC1) showed a higher correlation with high-DP (0.80), followed by medium-DP (0.73), and low-DP accuracy (0.24). The PC1 scores were used to rank the best passing players. This information can be a very rich performance indication by ranking the best passing players and teams and can be applied in offensive sequences analysis and talent identification.