Resource-Constrained Federated Edge Learning With Heterogeneous Data: Formulation and Analysis

This paper explores an interesting way to solve a multi-label classification problem using an ensemble learning with natural language processing approach. The multi-label classification problem selected is that of movie genre prediction where each movie can possibly belong to multiple genres which essentially differentiates it from a much more classical multi-class classification problem. In the current machine learning environment, there are a host of different classifiers, some which are more suited to handle more than binary classes at once such as random forests and decision trees while some which are modelled around solving an inherently binary classification problem. As we are using textual data in the form of synopsis of various movies, we use Word2Vec to vectorize the textual data in order to fit the classifiers. In this project, we use classifiers belonging to both these types, namely Naive Bayes, random forest classifier and XGBoost classifier to individually predict the multiple labels for each movie, and then we construct an ensemble model using a voting classier in the purview of ensemble learning by adjusting the weights of the combining classifiers and show that the ensemble model performs much better than each of the individual classifiers. The final micro-f-score achieved is 0.6738 which is commendable with using non-neural traditional machine learning techniques and is an improvement from the referred work for this paper.

Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

In this paper, a neuro-fuzzy based classification technique is adopted to efficiently deal with the classification problem of the heterogeneous medical data sets. The Proposed classification technique is based on the neuro-fuzzy classification (NE-FCLASS) system. However, several improvements are introduced to the origin NEFCLASS. The motivation of this work is triggered by the fact that most conventional classification techniques are capable of handling the numeric data sets but not the heterogenous ones. This can be seen in the classification of the medical data sets. This paper tackles the data classification problem of two medical diseases. The first data set, which is a numeric data set, is related to the Wisconsin breast cancer diagnosis. The second is a heterogeneous data set and is the Wisconsin heart disease diagnosis. Experimental results demonstrate that the proposed technique can effectively improve the classification performance of heterogenous data sets.

Federated learning (FL) embraces the concepts of targeted data gathering and training, and it can reduce many of the systemic privacy costs and hazards associated with traditional machine learning frameworks. However, with the low latency requirements of the sixth generation (6G) wireless communication networks and the Internet of Things (IoT) networks, the convergence delay of FL dramatically influences the overall system performance. In order to solve this urgent and challenging problem, in this paper, a joint client scheduling and wireless resource allocation algorithm is proposed, named SCSBA, which considers system heterogeneity, client heterogeneity, and the fairness of client participation to reduce the latency resulting from the heterogeneous communication conditions and computation capabilities among clients with the non identically independently distributed (Non-IID) data distributions. Specifically, the Stackelberg leader-follower game is first formulated in which the server decides the price of the single quota of participating in the FL process every communication round and the clients decide whether to participate in FL. Then the equilibrium solution of the game is derived and proved. In addition, a bandwidth allocation algorithm based on the covariance matrix adaptation evolutionary strategy (CMA-ES) is designed to minimize the time delay of each communication round. The simulation results verify the effectiveness of the proposed strategy for reducing the time latency of FL processes with heterogeneous clients, i.e., FedAvg and FedOpt.

With the outbreak of the COVID-19, people are eager to develop potential drugs for specific diseases through efficient technological means. Alzheimer's disease (AD) has become one of the top ten causes of death in the world and is a typical neurodegenerative disease. When acetylcholinesterase (AChE) is inhibited, it improves the transmission of cholinergic neurotransmitters in patients and restores cognitive function, so acetylcholinesterase inhibitors (AChEIs) are often considered by researchers as potential drugs for the treatment of AD. Machine learning algorithms and data mining techniques can accelerate drug development and reduce the cost of biological experiments, so it is of great significance to develop models that can accurately predict AChEIs. However, few studies have applied efficient and mature ensemble learning methods to the problem of predicting potential inhibitors of AChE. In this study, we constructed a dataset from a publicly available biological experiment database, and for the first time established an ensemble learning model based on CatBoost and XGBoost to predict potential AChEIs. We demonstrate the advantages of ensemble learning models in building AChEIs predictor based on imbalanced, heterogeneous data through a comprehensive evaluation. Afterwards, we also combined the best-performing models into a blending model AChEI-EL for case studies, and obtained the top-ranked potential inhibitors that have been shown to have the potential to inhibit the AChE. These results suggest that our method has a promising application in the field of AD. Finally, we developed a WEB online prediction platform based on the best model for the use and reference of researchers.

Machine Learning (ML) and Deep Learning (DL) are derivatives of Artificial Intelligence (AI) that have already demonstrated their effectiveness in a variety of domains, including healthcare, where they are now routinely integrated into patients’ daily activities. On the other hand, data heterogeneity has long been a key obstacle in AI, ML and DL. Here, Multimodal Machine Learning (Multimodal ML) has emerged as a method that enables the training of complex ML and DL models that use heterogeneous data in their learning process. In addition, Multimodal ML enables the integration of multiple models in the search for a single, comprehensive solution to a complex problem. In this review, the technical aspects of Multimodal ML are discussed, including a definition of the technology and its technical underpinnings, especially data fusion. It also outlines the differences between this technology and others, such as Ensemble Learning, as well as the various workflows that can be followed in Multimodal ML. In addition, this article examines in depth the use of Multimodal ML in the detection and prediction of Cardiovascular Diseases, highlighting the results obtained so far and the possible starting points for improving its use in the aforementioned field. Finally, a number of the most common problems hindering the development of this technology and potential solutions that could be pursued in future studies are outlined.

Accurate fault diagnosis for large machines is very important for its economic meaning. In essence, fault diagnosis is a pattern classification and recognition problem of judging whether the operation status of the system is normal or not. Support vector machines (SVMs), well motivated theoretically, have been introduced as effective methods for solving classification problems. However, the generalization performance of SVMs is sometimes far from the expected level due to their limitations in practical applications. Ensemble learning of SVMs provides a promising way for such cases. In this paper, a new ensemble learning method based on genetic algorithm is presented. Genetic algorithm is introduced into ensemble learning so as to search for accurate and diverse classifiers to construct a good ensemble. The presented method works on a higher level and is more direct than other search based ensemble learning methods. Different strategies of constructing diverse base classifiers are also studied. Experimental results on a steam turbine fault diagnosis problem show that the presented method can achieve much better generalization performance than other methods including single SVM, Bagging and Adaboost.M1 if the strategies used are appropriate for the practical problem.

Failure mode classification and bearing capacity prediction for reinforced concrete columns based on ensemble machine learning algorithm

Currently smartphone and other mobile devices have become incredibly significant in every facet of our life. They are practically offering same abilities as desktop workstations and are also powerful in terms of CPU, storage space and installing several applications. Therefore, security is deliberated as a vital factor in wireless communication technologies. Furthermore, based on growing range of mobile applications within variety of platforms, security is regarded as one of the most valuable and substantial debate in terms of issues, trustees, consistencies and accurateness. The worst security threat is that when your device has been lost or stolen. This paper aims to introduce an offline android application wherein the mobile phones which get lost by mistake or stolen by thief, then the application will help to get the device back by performing various operations on remote mobile devices. The approach will be to send offline messages from the genuine user with help of android devices to the one which is stolen. This developed application will be installed on any android device. The user of the app will set the verification code on his device. This code is the key which the user need to type along with the contact name in order to get recognized. The app always runs in background waiting for the incoming message.

Image classification is an important research branch in the field of computer vision. Ordinary image classification focuses on the discrimination of object differences, while fine-grained image classification focuses on the distinction between different sub-categories. Considering the small differences between sub-categories and the larger differences within sub-categories due to factors such as shooting angle, background, and posture, fine-grained image recognition becomes a challenging task. In order to solve the problem of fine-grained object classification, this paper proposes a finegrained object classification method based on block diagonal feature and ensemble learning. The Gabor feature of the object image is block diagonalized by using low-rank recovery technology, and the low-rank feature representation of the object is constructed by introducing the block diagonal sparse regular term to increase the discriminability of features between object sub-categories. On this basis, the stacking ensemble learning method is used to classify objects, and a fine-grained real animal image and non-real animal image (toy animal image) classification database is constructed. And the algorithm design and experiment are completed. Through the proposed algorithm, accuracy of fine-grained classification of objects is improved

Traffic crash prediction (TCP) is a fundamental problem for intelligent transportation systems in smart cities. Improving the accuracy of traffic crash prediction is important for road safety and effective traffic management. Owing to recent advances in artificial neural networks, several new deep-learning models have been proposed for TCP. However, these works mainly focus on accidents in regions, which are typically pre-determined using a grid map. We argue that TCP for roads, especially for crashes at or near road intersections which account for more than 50% of the fatal or injury crashes based on the Federal Highway Administration, has a significant practical and research value and thus deserves more research. In this paper, we formulate TCP at Road Intersections as a classification problem and propose a three-phase data-driven deep learning model, called Road Intersection Traffic Crash Prediction (RoadInTCP), to predict traffic crashes at intersections by exploiting publicly available heterogeneous big data. In Phase I we extract discriminative latent features called topological-relational features (tr-features), of intersections using a neural network model by exploiting topological information of the road network and various relationships amongst nearby intersections. In Phase II, in addition to tr-features which capture some inherent properties of the road network, we also explore additional thematic information in terms of environmental, traffic, weather, risk, and calendar features associated with intersections. In order to incorporate the potential correlation in nearby intersections, we utilize a Graph Convolution Network (GCN) to aggregate features from neighboring intersections based on a message-passing paradigm for TCP. While Phase II serves well as a TCP model, we further explore the signals embedded in the sequential feature changes over time for TCP in Phase III, by exploring RNN or 1DCNN which have known success on sequential data. Additionally, to address the serious issues of imbalanced classes in TCP and large-scale heterogeneous big data, we propose an effective data sampling approach in data preparation to facilitate model training. We evaluate the proposed RoadInTCP model via extensive experiments on a real-world New York City traffic dataset. The experimental results show that the proposed RoadInTCP robustly outperforms existing methods.

A new feasible descent algorithm combining SQP with generalized projection for optimization problems without strict complementarity

This paper explores parallel and distributed implementation of the Learning Classifier System (LCS) technology. Specifically, the adaptation of supervised LCS to the grid data mining requisites, using the agent paradigm, is studied. The paper also examines the competitive data mining model induction possibility with homogeneous and heterogeneous data. A distributed framework is proposed using the grid computing architecture to make Supervised Classifier Systems (UCS) a more efficient distributed environment. This framework allows each local site (or agent) to employ a completely independent UCS. Local models (or UCS's populations) are transmitted occasionally to the global model for combination. The global model then represents a complete knowledge base of the overall classification problem.

Pathogenic microorganisms exploit host cellular mechanisms and evade host defense mechanisms through molecular pathogen-host interactions (PHIs). Therefore, comprehensive analysis of these PHI networks should be an initial step for developing effective therapeutics against infectious diseases. Computational prediction of PHI data is gaining increasing demand because of scarcity of experimental data. Prediction of protein-protein interactions (PPIs) within PHI systems can be formulated as a classification problem, which requires the knowledge of non-interacting protein pairs. This is a restricting requirement since we lack datasets that report non-interacting protein pairs. In this study, we formulated the "computational prediction of PHI data" problem using kernel embedding of heterogeneous data. This eliminates the abovementioned requirement and enables us to predict new interactions without randomly labeling protein pairs as non-interacting. Domain-domain associations are used to filter the predicted results leading to 175 novel PHIs between 170 human proteins and 105 viral proteins. To compare our results with the state-of-the-art studies that use a binary classification formulation, we modified our settings to consider the same formulation. Detailed evaluations are conducted and our results provide more than 10 percent improvements for accuracy and AUC (area under the receiving operating curve) results in comparison with state-of-the-art methods.

Vehicle insurance claim severity modeling requires accurate and interpretable methods that can handle skewed and heterogeneous loss data. This study provides a structured empirical comparison between classical parametric regression models and tree-based ensemble learning approaches for predicting claim size conditional on claim occurrence. The analysis is conducted within a cross-sectional conditional severity framework using real-world motor insurance data. We implement and compare ordinary least squares (OLS), a Tweedie generalized linear model (GLM), and three ensemble methods: bagging, random forests (RFs), and gradient boosting. Model performance is evaluated using out-of-sample root mean square error (RMSE), and variable importance measures assess the relative contribution of predictors. The results indicate that tree-based ensemble methods achieve modest improvements in predictive accuracy relative to classical parametric models. The Tweedie GLM remains a competitive, flexible parametric benchmark for skewed positive claim amounts. Variable importance analysis consistently identifies premium and insured value as key determinants of claim severity. Overall, the findings suggest that ensemble learning methods can complement traditional actuarial models, offering additional flexibility in capturing non-linear effects while maintaining comparable predictive performance in moderate-complexity severity data.