EXACT: How to train your accuracy

A novel method for tackling the problem of imbalanced data in medical image segmentation is proposed in this work. In Balanced Cross Entropy (BCE) loss, which is a type of weighted CE loss, the weight assigned to each class is the inverse of the class frequency. These balancing weights are expected to equalize the effect of each class on the overall loss and prevent the model from being biased towards the majority class. But, as it has been shown in previous studies, this method degrades the performance by a large margin. Therefore, Balanced CE is not a popular loss in medical segmentation tasks, and usually a region-based loss, like the Dice loss, is used to address the class imbalance problem. Existing weighting CE loss approaches, like Balanced CE, are usually adapted from classification tasks. In this study we propose a weighting method which is designed for segmentation and takes advantage of spatial and geometrical properties of the labels. In the proposed method, the weighting of cross entropy loss for each class is based on a dilated area of each class mask, and balancing weights are assigned to each class together with its surrounding pixels. The goal of this study is to show that the performance of Balanced CE loss can be greatly improved my modifying its weighting strategy. Experiments on different datasets show that the proposed Dilated Balanced CE (DBCE) loss outperforms the Balanced CE loss by a large margin and produces superior results compared to CE loss, and its performance is similar or better compared to the performance of the combination of Dice and CE loss. Also, the performance of different combination losses can be improved by replacing the CE loss with the proposed DBCE loss. This study shows that a weighted cross entropy loss with the right weighing strategy can be as effective as a region-based loss in handling the class imbalance problem.

Huberized one-class support vector machine with truncated loss function in the primal space

The optimal design of gas generator was conducted. Turbo pump system with fuel rich combustion in gas generator was the system considered in this study. The objective function is the specific impulse of main thrust chamber operating with RP-11LOX propellants. Sensitivity analyses to the objective function were used to measure the relative dependence of design variables the objective function. Also optimization method was adopted with two different algorithms; GBM (Gradient based method) and non-GBM. GBM includes SLP, MMFD, and SQP algorithms and the result of GBM were compared with non-GBM result; SA (Simulated Annealing) method. As a result, it was confirmed that SA could yield a global optimization regardless of initial variations. Whereas GBM showed the results were dependent on the initial conditions. And the global optimization could provide a better configuration of gas generator in the design stage. The present study aims at the optimization of gas generator design. Thus optimization methodology was tested with variety of optimization technique applied to the gas generator design with various constraints. Also, the present study compares the results from GBM algorithms with that of SA in order to check the validity of global solution

Recently, deep-learning-based approaches have been proposed for the classification of neuroimaging data related to Alzheimer’s disease (AD), and significant progress has been made. However, end-to-end learning that is capable of maximizing the impact of deep learning has yet to receive much attention due to the endemic challenge of neuroimaging caused by the scarcity of data. Thus, this study presents an approach meant to encourage the end-to-end learning of a volumetric convolutional neural network (CNN) model for four binary classification tasks (AD vs. normal control (NC), progressive mild cognitive impairment (pMCI) vs. NC, stable mild cognitive impairment (sMCI) vs. NC and pMCI vs. sMCI) based on magnetic resonance imaging (MRI) and visualizes its outcomes in terms of the decision of the CNNs without any human intervention. In the proposed approach, we use convolutional autoencoder (CAE)-based unsupervised learning for the AD vs. NC classification task, and supervised transfer learning is applied to solve the pMCI vs. sMCI classification task. To detect the most important biomarkers related to AD and pMCI, a gradient-based visualization method that approximates the spatial influence of the CNN model’s decision was applied. To validate the contributions of this study, we conducted experiments on the ADNI database, and the results demonstrated that the proposed approach achieved the accuracies of 86.60% and 73.95% for the AD and pMCI classification tasks respectively, outperforming other network models. In the visualization results, the temporal and parietal lobes were identified as key regions for classification.

The growth of the Internet of Things (IoT) and smart systems has paved the way for the integration of machine learning (ML) and optimization techniques to enhance their efficiency, security, and scalability. This survey provides a comprehensive overview of recent advancements in applying machine learning algorithms and optimization strategies to IoT and smart systems. It highlights key areas where these technologies intersect, such as energy efficiency, intelligent decision-making, resource management, and anomaly detection. The paper discusses various machine learning approaches, including deep learning, reinforcement learning, and supervised/unsupervised models, that have been applied to optimize system performance in real-time applications. Furthermore, the survey examines optimization techniques like metaheuristics, evolutionary algorithms, and gradient-based methods that improve IoT systems' reliability and resource utilization. The integration of these techniques offers significant advantages, including reduced latency, enhanced security, and increased adaptability in smart systems. The findings emphasize the importance of adopting ML and optimization methods to address the challenges posed by the growing complexity of IoT environments, fostering future innovations in connected and autonomous systems. This survey serves as a resource for researchers and practitioners seeking to explore the synergies between IoT, ML, and optimization techniques.

Neural network model has undergone many modifications from the original model through the development of network architecture and optimization methods. By default, the gradient-based optimization method is a method used to find network weights. But some of the weaknesses and limitations of these methods inspired many researchers to try to use other methods. Non-gradient based heuristic method is a reasonable choice considering the learning algorithm in artificial neural networks is inspired by the characteristics of creatures so that optimization methods that also mimic life patterns in nature will be appropriate. One disadvantage of the heurisic method is the length of the iteration process. In this paper, a method which combines heuristic optimization methods and gradient-based methods is applied, namely particle swarm optimization (PSO) and Levenberg Marquardt. The weight obtained from the PSO method becomes the initial weight for the Levenberg Marquardt method. The proposed procedure is applied to the rainfall data in Cokrotulung Klaten. The results showed that this procedure succeeded in providing better predictions than the Levenberg Marquardt method.

In this paper, a general Maximum K-Min approach for classification is proposed. With the physical meaning of optimizing the classification confidence of the K worst instances, Maximum K-Min Gain/Minimum K-Max Loss (MKM) criterion is introduced. To make the original optimization problem with combinational constraints computationally tractable, the optimization techniques are adopted and a general compact representation lemma for MKM Criterion is summarized. Based on the lemma, a Nonlinear Maximum K-Min (NMKM) classifier and a Semi-supervised Maximum K-Min (SMKM) classifier are presented for traditional classification task and semi-supervised classification task respectively. Based on the experiment results of publicly available datasets, our Maximum K-Min methods have achieved competitive performance when comparing against Hinge Loss classifiers.

Nuclei segmentation and classification provide an essential basis for tumor immune microenvironment analysis. The previous nuclei segmentation and classification models require splitting large images into smaller patches for training, leading to two significant issues. First, nuclei at the borders of adjacent patches often misalign during inference. Second, this patch-based approach significantly increases the model's training and inference time. Recently, Mamba has garnered attention for its ability to model large-scale images with linear time complexity and low memory consumption. It offers a promising solution for training nuclei segmentation and classification models on full-sized images. However, the Mamba orientation-based scanning method lacks account for category-specific features, resulting in suboptimal performance in scenarios with imbalanced class distributions. To address these challenges, this paper introduces a novel scanning strategy based on category probability sorting, which independently ranks and scans features for each category according to confidence from high to low. This approach enhances the feature representation of uncertain samples and mitigates the issues caused by imbalanced distributions. Extensive experiments conducted on four public datasets demonstrate that our method outperforms state-of-the-art approaches, delivering superior performance in nuclei segmentation and classification tasks.

Classification and Regression are commonly used techniques in machine learning. The Loss functions are foundational to machine learning, serving as quantifiable measures of prediction error though which model optimization is lead. This study presents a comprehensive mathematical analysis of prominent loss functions utilized across classification and regression tasks. We present formal definitions and investigate inherent mathematical properties such as convexity and differentiability. Their implications for the learning process, specifically within gradient-based optimization frameworks. The analysis begins with fundamental regression losses such as Mean Squared Error and Mean Absolute Error. The analysis further extends to robust alternatives such as Huber and Quantile Loss. For classification, we explore binary and categorical Cross-Entropy, Hinge Loss, Exponential Loss by clarifying their characteristics. Also, we discuss practical considerations in selecting appropriate loss functions, importance of regularization, custom loss functions design, and the critical distinction between optimization objectives and evaluations metrics. The work aims to create understanding for readers with deeper theoretical understanding of how different loss functions shape model behaviour and performance in diverse machine learning applications.

Chapter 7 - Optimal Power Flow Using Recent Optimization Techniques

To construct a visual intelligent recognition model for Oncomelania hupensis robertsoni in Yunnan Province based on the EfficientNet-B4 model, and to evaluate the impact of data augmentation methods and model hyperparameters on the recognition of O. hupensis robertsoni. A total of 400 O. hupensis robertsoni and 400 Tricula snails were collected from Yongsheng County, Yunnan Province in June 2024, and snail images were captured following identification and classification of 300 O. hupensis robertsoni and 300 Tricula snails. A total of 925 O. hupensis robertsoni images and 1 062 Tricula snail images were collected as a dataset and divided into a training set and a validation set at a ratio of 8:2, while 352 images captured from the remaining 100 O. hupensis robertsoni and 354 images from the remaining 100 Tricula snails served as an external test set. All acquired images were subjected to preprocessing, including cropping and resizing. Three data augmentation approaches were employed, including baseline, Mixup and Gaussian blurring, and model hyperparameters included two optimization algorithms of adaptive moment estimation (Adam) and stochastic gradient descent (SGD), two loss functions of focal loss and cross entropy loss, and two learning rate decay strategies of cosine annealing and multi-step. The intelligent recognition models of O. hupensis robertsoni and Tricula snails were constructed based on the EfficientNet-B4 model, and 7 training strategy groups were generated by combinations of different data augmentation approaches and hyperparameters. The performance of intelligent recognition models was tested with external test sets, and evaluated with accuracy, precision, recall, F1 score, loss, Youden's index, and the area under the receiver operating characteristic curve (AUC) under different training strategies. The variation of loss values was comparable among intelligent recognition models with different data augmentation approaches. The Group 4 model constructed with Mixup and Gaussian blurring data augmentation approaches showed the optimal performance, with an accuracy of 90.38%, precision of 90.07%, F1 score of 89.44%, Youden's index of 0.81 and AUC of 0.961 in the external test set. The accuracy of models using the SGD optimizer reduced by 29.16% as compared to those using the Adam optimizer (χ2 = 81.325, P < 0.001), and the accuracy of models using the cross entropy loss function reduced by 0.80% as compared to the Group 4 model (χ2 = 3.147, P > 0.05), while the accuracy of models using the multi-step learning rate decay strategy increased by 0.65% as compared to the Group 4 model (χ2 = 0.208, P > 0.05). In addition, the model with the baseline + Mixup + Gaussianblurring data augmentation approach and hyperparameters of Adam optimizer, focal loss function and multi-step learning rate decay strategy showed the highest performance, with an accuracy of 91.03%, precision of 91.97%, recall of 88.11%, F1 score of 90.00%, Youden's index of 0.82 and AUC values of 0.969 in external test set, respectively. The intelligent recognition model of O. hupensis robertsoni based on EfficientNet-B4 model is accurate for identification of O. hupensis robertsoni and Tricula snails in Yunnan Province.

BackgroundImplicit shape-based reconstruction method in fluorescence molecular tomography (FMT) is capable of achieving higher image clarity than image-based reconstruction method. However, the implicit shape method suffers from a low convergence speed and performs unstably due to the utilization of gradient-based optimization methods. Moreover, the implicit shape method requires priori information about the number of targets.MethodsA shape-based reconstruction scheme of FMT with a cosinoidal level set method is proposed in this paper. The Heaviside function in the classical implicit shape method is replaced with a cosine function, and then the reconstruction can be accomplished with the Levenberg–Marquardt method rather than gradient-based methods. As a result, the priori information about the number of targets is not required anymore and the choice of step length is avoided.ResultsNumerical simulations and phantom experiments were carried out to validate the proposed method. Results of the proposed method show higher contrast to noise ratios and Pearson correlations than the implicit shape method and image-based reconstruction method. Moreover, the number of iterations required in the proposed method is much less than the implicit shape method.ConclusionsThe proposed method performs more stably, provides a faster convergence speed than the implicit shape method, and achieves higher image clarity than the image-based reconstruction method.

Solution of implicitly formulated inverse heat transfer problems with hybrid methods

In the task of music genre classification, feature extraction and classifier modeling are the two key parts that directly affect the classification accuracy. In the traditional classification method, the feature extraction and classification processes are designed separately. First, the features are extracted manually from the original music signal, and then a reasonable classifier is selected to build a model and classify the extracted features. Although the traditional methods have achieved good results in many classification tasks, the feature extraction process is complex and difficult to achieve, and the features required for different classification tasks need to be specially designed, and the extracted features lack of universality. With the successful application and continuous development of deep learning models in other fields, more and more studies begin to use music spectra as the input of deep learning models to classify music genres. However, so far, the accuracy of existing classification methods based on deep learning is not ideal, so this paper mainly studies a classification method based on deep learning to improve the classification accuracy of music genre classification model. In this paper, a parallel structured deep attention classification model is proposed. By training BRNN, it can automatically learn music features from samples. The linear attention model calculates the attention probability distribution on this feature according to the learned features, and reassigns it to the feature representation. Finally, the classification is realized according to the eigenvectors with different weights. Besides the linear attention model with simple structure, a CNN attention model with stronger learning ability is also designed. In order to verify the feasibility and validity of the model, validation experiments were conducted on two standard datasets, GTZAN and Extended Ballroom. The experimental results show that the classification model based on deep parallel attention mechanism has good classification robustness, and the accuracy of classification based on BRNN and parallel CNN attention model reaches 92.7% on Extended Ballroom data set, which is better than the existing classification method based on deep learning. The validity and feasibility of the classification model are proved.

Various methods have been proposed to conduct optimization via discrete-event simulation. Prevalent among these are gradient-based algorithms, and more recently, so-called retrospective approaches that deterministically optimize over simulated outcomes. In this paper we compare and contrast the performance of these two approaches for the optimization of a periodic review (s,S) inventory system. In practice, gradient-based methods often exhibit undesirably slow convergence, whereas the retrospective approach can be computationally burdensome. We propose a hybrid algorithm that combines aspects of both approaches to mitigate these shortcomings.