Comparison of Deep Learning With Multiple Machine Learning Methods and Metrics Using Diverse Drug Discovery Data Sets.

Deep learning has become widely used in image analysis. Transfer learning can make use of information from other datasets for the analysis of the chosen dataset. When there is a small number of images at hand, transfer learning using pre-trained models with coefficients already estimated from other datasets is recommended. This is in contrast to deep learning, where most model parameters are re-estimated. Deep transfer learning uses pre-trained models with fixed weight parameters in the lower layers; thus, deep learning can be viewed as a two-stage approach: (1) feature extraction from a lower neural network layer and (2) estimation of a neural network using the extracted features as inputs. Since deep transfer learning is feature extraction, we can extend the two-stage approach to a more general two-stage framework: (1) feature extraction using multiple methods and (2) machine learning methods taking extracted features as inputs. We evaluate the performance of methods with different Stage 1 and Stage 2 approaches in predicting the phenotype leaf numbers based on a multi-view plant imaging dataset. This paper contains an evaluation of different two-stage machine learning methods for multi-view image data in plant image phenotyping.

Deep learning (DL) and machine learning (ML) methods have recently contributed to the advancement of models in the various aspects of prediction, planning, and uncertainty analysis of smart cities and urban development. This paper presents the state of the art of DL and ML methods used in this realm. Through a novel taxonomy, the advances in model development and new application domains in urban sustainability and smart cities are presented. Findings reveal that five DL and ML methods have been most applied to address the different aspects of smart cities. These are artificial neural networks; support vector machines; decision trees; ensembles, Bayesians, hybrids, and neuro-fuzzy; and deep learning. It is also disclosed that energy, health, and urban transport are the main domains of smart cities that DL and ML methods contributed in to address their problems.

Shear viscosity, though being a fundamental property of all fluids, is computationally expensive to calculate from equilibrium molecular dynamics simulations. Recently, machine learning (ML) methods have been used to augment molecular simulations in many contexts, thus showing promise to estimate viscosity too in a relatively inexpensive manner. However, ML methods face significant challenges—such as overfitting, when the size of the data set is small, as is the case with viscosity. In this work, we train seven ML models to predict the shear viscosity of a Lennard–Jones fluid, with particular emphasis on addressing issues arising from a small data set. Specifically, the issues related to model selection, performance estimation and uncertainty quantification were investigated. First, we show that the widely used performance estimation procedure of using a single unseen data set shows a wide variability—in estimating the errors on—small data sets. In this context, the common practice of using cross validation (CV) to select the hyperparameters (model selection) can be adapted to estimate the generalization error (performance estimation) as well. We compare two simple CV procedures for their ability to do both model selection and performance estimation, and find that k-fold CV based procedure shows a lower variance of error estimates. Also, these CV procedures naturally lead to an ensemble of trained ML models. We discuss the role of performance metrics in training and evaluation and propose a method to rank the ML models based on multiple metrics. Finally, two methods for uncertainty quantification—Gaussian process regression (GPR) and ensemble method—were used to estimate the uncertainty on individual predictions. The uncertainty estimates from GPR were also used to construct an applicability domain using which the ML models provided even more reliable predictions on an independent viscosity data set generated in this work. Overall, the procedures prescribed in this work, together, lead to robust ML models for small data sets.

Industrial robots are widely used in critical tasks such as assembly, welding, and material handling as core components of modern manufacturing systems. For the reliable operation of these systems, early and accurate detection of execution failures is crucial. In this study, a comprehensive comparison of machine learning and deep learning methods is conducted for the classification of robot execution failures using data acquired from force-torque sensors. Three different feature engineering approaches are proposed. The first is a Baseline approach that includes 90 raw time-series features. The second is the Domain-6 approach, which consists of 6 basic statistical features per sensor (36 in total). The third is the Domain-12 approach, which comprises 12 comprehensive statistical features per sensor (72 in total). The domain features include the mean, standard deviation, minimum, maximum, range, slope, median, skewness, kurtosis, RMS, energy, and IQR. In total, ten classification algorithms are evaluated, including eight machine learning methods and two deep learning models: Support Vector Machines (SVM), Random Forest (RF), k-Nearest Neighbors (KNN), Artificial Neural Network (ANN), Naive Bayes (NB), Decision Trees (DT), eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM-LGBM), as well as a One-Dimensional Convolutional Neural Network (CNN-1D) and Long Short-Term Memory (LSTM). For traditional machine learning algorithms, 5 × 5 nested cross-validation is used, whereas for deep learning models, 5-fold cross-validation with a 20% validation split is employed. To ensure statistical reliability, all experiments are repeated over 30 independent runs. The experimental results demonstrate that feature engineering has a decisive impact on classification performance. In addition, regardless of the feature set, the highest accuracy (93.85% ± 0.90) is achieved by the Naive Bayes classifier using the Baseline features. The Domain-12 feature set provides consistent improvements across many algorithms, with substantial performance gains. The results are reported using accuracy, precision, recall, and F1-score metrics and are supported by confusion matrices. Finally, permutation feature importance analysis indicates that the skewness features of the Fx and Fy sensors are the most critical variables for failure detection. Overall, these findings show that time-domain statistical features offer an effective approach for robot failure classification.

Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then, the field has rapidly progressed congruently with the wide adoption of machine learning (ML) methods in the environmental sciences. Here, we present a scoping review of ML applications in wildfire science and management. Our overall objective is to improve awareness of ML methods among wildfire researchers and managers, as well as illustrate the diverse and challenging range of problems in wildfire science available to ML data scientists. To that end, we first present an overview of popular ML approaches used in wildfire science to date and then review the use of ML in wildfire science as broadly categorized into six problem domains, including (i) fuels characterization, fire detection, and mapping; (ii) fire weather and climate change; (iii) fire occurrence, susceptibility, and risk; (iv) fire behavior prediction; (v) fire effects; and (vi) fire management. Furthermore, we discuss the advantages and limitations of various ML approaches relating to data size, computational requirements, generalizability, and interpretability, as well as identify opportunities for future advances in the science and management of wildfires within a data science context. In total, to the end of 2019, we identified 300 relevant publications in which the most frequently used ML methods across problem domains included random forests, MaxEnt, artificial neural networks, decision trees, support vector machines, and genetic algorithms. As such, there exists opportunities to apply more current ML methods — including deep learning and agent-based learning — in the wildfire sciences, especially in instances involving very large multivariate datasets. We must recognize, however, that despite the ability of ML models to learn on their own, expertise in wildfire science is necessary to ensure realistic modelling of fire processes across multiple scales, while the complexity of some ML methods such as deep learning requires a dedicated and sophisticated knowledge of their application. Finally, we stress that the wildfire research and management communities play an active role in providing relevant, high-quality, and freely available wildfire data for use by practitioners of ML methods.

Two-centers machine learning analysis for predicting acid-fast bacilli results in tuberculosis sputum tests.

Ransomware is malware that encrypts a victim's files and demands payment, typically in cryptocurrencies, to restore access. The global spread of ransomware attacks underscores the need for automatic detection systems. Signature‐based antivirus tools cannot identify novel ransomware strains, while machine learning (ML) and deep learning (DL) models offer promise because they learn data patterns and generalize to new unseen samples. However, the choice of model architecture remains an open question: do classical ML methods suffice or do DL methods bring a considerable performance increase for static malware analysis? In this paper, we benchmark three classic algorithms (Logistic Regression, Random Forest and XGBoost) and two deep architectures (Multi‑Layer Perceptron and TabNet) on the Windows Portable Executable (PE) ransomware detection dataset from Kaggle. We perform comprehensive data preprocessing, 5‑fold stratified cross‑validation, and compute performance metrics like accuracy, precision, recall, F1 score, area under the ROC curve (AUC), and Matthews correlation coefficient (MCC). We further examine the feature importance, plot receiver operating characteristic (ROC) curves, and discuss the computational expense and interpretability of each method. Our experiments show that tree ensemble methods (Random Forest and XGBoost) achieve near‑perfect detection with very low false positives and negatives, outperforming Logistic Regression and MLP. Deep models offer no significant advantages for this tabular data set but with higher computational expense. We discuss the implications of these results for ransomware defense in practice and propose directions for future research.

Power system fault type classification and location prediction is critical in assessing the reliability of the power system, and later restoring it to a stable operating point followed by a fault. State of the art methods include sequence component, impedance measurement from the origin of the fault, and traveling wave based methods for fault type detection and classification problem. It is important to identify and classify the fault as quickly as possible for restoring power system stability to normal operation. Machine Learning and Deep Learning methods allow the analysis of large data of fault voltages and currents by using fast and efficient algorithms. These methods require large amount of data, however, with the recent advances in the field of power system, data acquisition using smart meters and Phasor Measurement Units (PMU), huge amount of system-wide data can be made available to analyze the problem of fault type classification and location prediction. This paper presents a comparative study of Stochastic Gradient Descent (SGD) based Deep Neural Network (DNN) and Machine Learning (ML) applied to power system fault type and location prediction problem. DNN architecture uses 10 hidden layers, each layer having 60 units with hyperbolic tangent as activation function, and a combination of Support Vector Machine (SVM) and Principal Component Analysis (PCA) method are considered. Comparative results in terms of time taken to run the algorithms, and accuracy of the results obtained are presented for a 3 machine 9 bus system. Results indicate that SVM method is an optimal choice for fault classification with high accuracy for location prediction and low computational requirements compared with DNN.

BackgroundThe brain tumor is the growth of abnormal cells inside the brain. These cells can be grown into malignant or benign tumors. Segmentation of tumor from MRI images using image processing techniques started decades back. Image processing based brain tumor segmentation can be divided in to three categories conventional image processing methods, Machine Learning methods and Deep Learning methods. Conventional methods lacks the accuracy in segmentation due to complex spatial variation of tumor. Machine Learning methods stand as a good alternative to conventional methods. Methods like SVM, KNN, Fuzzy and a combination of either of these provide good accuracy with reasonable processing speed. The difficulty in processing the various feature extraction methods and maintain accuracy as per the medical standards still exist as a limitation for machine learning methods. In Deep Learning features are extracted automatically in various stages of the network and maintain accuracy as per the medical standards. Huge database requirement and high computational time is still poses a problem for deep learning. To overcome the limitations specified above we propose an unsupervised dual autoencoder with latent space optimization here. The model require only normal MRI images for its training thus reducing the huge tumor database requirement. With a set of normal class data, an autoencoder can reproduce the feature vector into an output layer. This trained autoencoder works well with normal data while it fails to reproduce an anomaly to the output layer. But a classical autoencoder suffer due to poor latent space optimization. The Latent space loss of classical autoencoder is reduced using an auxiliary encoder along with the feature optimization based on singular value decomposition (SVD). The patches used for training are not traditional square patches but we took both horizontal and vertical patches to keep both local and global appearance features on the training set. An Autoencoder is applied separately for learning both horizontal and vertical patches. While training a logistic sigmoid transfer function is used for both encoder and decoder parts. SGD optimizer is used for optimization with an initial learning rate of .001 and the maximum epochs used are 4000. The network is trained in MATLAB 2018a with a processor capacity of 3.7 GHz with NVIDIA GPU and 16 GB of RAM.ResultsThe results are obtained using a patch size of 16 × 64, 64 × 16 for horizontal and vertical patches respectively. In Glioma images tumor is not grown from a point rather it spreads randomly. Region filling and connectivity operations are performed to get the final tumor segmentation. Overall the method segments Meningioma better than Gliomas. Three evaluation metrics are considered to measure the performance of the proposed system such as Dice Similarity Coefficient, Positive Predictive Value, and Sensitivity.ConclusionAn unsupervised method for the segmentation of brain tumor from MRI images is proposed here. The proposed dual autoencoder with SVD based feature optimization reduce the latent space loss in the classical autoencoder. The proposed method have advantages in computational efficiency, no need of huge database requirement and better accuracy than machine learning methods. The method is compared Machine Learning methods Like SVM, KNN and supervised deep learning methods like CNN and commentable results are obtained.

Landslide inventories are crucial to studying the dynamics, associated risks, and effects of these geomorphological processes on the evolution of mountainous landscapes. The production of landslide maps is mainly based on manual visual interpretation methods of aerial and satellite images combined with field surveys. In recent times, advances in machine learning methods have made it possible to explore new semi-automated landslide detection methodologies using remotely detected images. In this sense, developing new artificial intelligence models based on Deep Learning (DL) opens up an excellent opportunity to automate this arduous process. Although the Andes mountain range is one of the most geomorphologically active areas on the planet, the few investigations that use DL mainly focus on mountain ranges in Europe and Asia. One of the main reasons is the low density of landslide data available in the Andean areas, making it difficult to experiment with DL models requiring large data volumes. In this work, we seek to narrow the existing gap in the availability of landslide inventories in the area of the Patagonian Andes. In addition, the feasibility and efficiency of DL techniques are studied to develop landslide detection models in the Andes from the generated datasets. To achieve this goal, we generated in a manual process a datasets of 10,000 landslides for northern Chilean Patagonia (42–45°S), being the largest freely accessible landslide datasets in this region. We implement a machine learning model, through DL, to detect landslides in optical images of the Sentinel-2 constellation using a model based on the DeepLabv3+ architecture, a state-of-the-art deep learning network for semantic segmentation. Our results indicate that the algorithm detects landslides with an accuracy of 0.75 at the object level. For its part, the segmentation reaches a precision of 0.86, a recall of 0.74, and an F1-score of 0.79. The correlation of the segmentation measured through the Matthews correlation coefficient shows a value of 0.59, and the geometric similarity of the correctly detected landslides measured through the Jaccard score reaches 0.70. Although the model shows a good response in the testing area, errors are generated that can be explained by geometric and spectral relationships, which should be solved through new training approaches and data sets.

In recent years, medical data analysis has become paramount in delivering accurate diagnoses for various diseases. The plethora of medical data sources, encompassing disease types, disease-related proteins, ligands for proteins, and molecular drug components, necessitates adopting effective disease analysis and diagnosis methods. Soft computing techniques, including swarm algorithms and machine learning (ML) methods, have emerged as superior approaches. While ML techniques such as classification and clustering have gained prominence, feature selection methods are crucial in extracting optimal features and reducing data dimensions. This review paper presents a comprehensive overview of soft computing techniques for tackling medical data problems through classifying and analyzing medical data. The focus lies mainly on the classification of medical data resources. A detailed examination of various techniques developed for classifying numerous diseases is provided. The review encompasses an in-depth exploration of multiple ML methods designed explicitly for disease detection and classification. Additionally, the review paper offers insights into the underlying biological disease mechanisms and highlights several medical and chemical databases that facilitate research in this field. Furthermore, the review paper outlines emerging trends and identifies the key challenges in biomedical data analysis. It sheds light on this research domain’s exciting possibilities and future directions. The enhanced understanding of soft computing techniques and their practical applications and limitations will contribute to advancing biomedical data analysis and support healthcare professionals in making accurate diagnoses.

Domain name generation algorithm (DGA) classification is an essential but challenging problem. Both feature-extracting machine learning (ML) methods and deep learning (DL) models such as convolutional neural networks and long short-term memory have been developed. However, the performance of these approaches varies with different types of DGAs. Most features in the ML methods can characterize random-looking DGAs better than word-looking DGAs. To improve the classification performance on word-looking DGAs, subword tokenization is employed for the DL models. Our experimental results proved that the subword tokenization can provide excellent classification performance on the word-looking DGAs. We then propose an integrated scheme that chooses an appropriate method for DGA classification depending on the nature of the DGAs. Results show that the integrated scheme outperformed existing ML and DL methods, and also the subword DL methods.

Cardiovascular diseases (CVDs) remain a leading cause of global mortality, underscoring the urgent need for early and accurate diagnosis. Electrocardiograms (ECGs) are a widely used, noninvasive tool for detecting cardiac abnormalities, but manual interpretation can be time- consuming and prone to error. This study explores advanced machine learning (ML) and deep learning (DL) methods to improve the detection of CVDs from ECG images, leveraging their ability to analyze complex patterns with high precision. Models such as Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and ensemble frameworks are utilized, alongside traditional ML models like XGBoost, CatBoost, and LightGBM. The performance of these models is evaluated based on precision, recall, F1-score, and accuracy. Results show that DL models, particularly an ensemble of CNN and ViT, achieve superior accuracy , outperforming standalone ML methods. Furthermore, an ensemble of XGBoost, CatBoost, and LightGBM achieves competitive results, while Optuna-tuned XGBoost further enhances ML performance. The findings highlight the potential of combining ML and DL techniques for robust and reliable CVD detection in ECG images. This research demonstrates the promise of AI-driven solutions in advancing medical diagnostics and supports the integration of automated systems for early detection of cardiovascular diseases.

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

The attack that occurred recently involved the utilization of malicious software, commonly referred to as malware, along with advanced techniques such as machine learning, specifically deep learning, code transformation, and polymorphism. This makes it harder for cyber experts to detect malware using traditional analysis methods. In view of the low accuracy and high false positive rate of traditional malware detection methods, this research proposes a fine-tuned deep learning model with a novel dataset that is compared with ANN, Support Vector Machine (SVM), random forest (RF), K-Nearest Neighbourhood (KNN) classifiers. Converting a malware code into an image could allow users to effectively identify the presence of malware, even if the original code is modified by the creator. This is due to the fact that the attributes of images remain unchanged, allowing for reliable identification. So, researchers used deep learning technology to detect malware, like detecting malaria from red blood cells. The deep learning model found that a more detailed analysis of malware data sets, focusing on RGB and greyscale images, is needed. These data sets currently rely on publicly available data, but the accuracy of the traditional model could be a lot higher. It also produces many false positive results. The main goal is to create a new data set and model using malware images to identify and categorize malware using deep learning without relying on existing image detection and transfer learning models. The researcher adjusted different hyper parameters, like the number of neurons, filters, stride, hidden units, layers, learning rate, batch size, activation function, optimizer, and epochs. Identifying and correcting issues in models, improving their clarity, stability, and fairness, as well as debugging and monitoring them, is a challenging task. To eliminate this obstacle, assess the model's behaviour and performance by employing various methods such as logging, profiling, testing, visualization, and model clarity. To overcome the challenges posed by the electricity shutdown, we utilized Google's cloud-based GPU and Python 3.7 language to conduct the experiment and train the model. Kali Linux is an operating system that can automatically encrypt file systems and has a lower chance of crashing the system due to malware in a virtual sandbox. The researcher used techniques like early stopping and cross-validation to prevent over fitting and assess generalization in addition to monitoring and evaluating the model's behaviour and performance. The researcher used different methods like L1 and L2 regularization, dropout, and batch normalization to improve the model's performance and avoid over fitting. The binary portable executable (PE) malware dataset is collected from Kaggle, Malimg, Virusshare, Malvis, MS Big2015, and VX-underground and finally converted to greyscale and RGB images to create a novel dataset to fill the lack of image dataset. The raw dataset was then rescaled to a 128x128 greyscale and RGB (red, green, blue) image and flattened to 1024-byte vector images input into convolution neural network (CNN) interpolation to extract features for malware detection. The newly acquired dataset is utilized as an input for the innovative DL algorithms in order to create a tailor-made model capable of accurately predicting malware. The findings in this customized CNN model by fine-tuning hyper-parameters with the three-channel RGB dataset outperformed a greyscale dataset with an accuracy of 98.7% and error rate of 1.3% in current malware compared with other artificial neural network (ANN), ML algorithms such as Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and also Random Forest (RF) models. The proposed approach is compatible with all operating systems (Windows, Linux, and Mac) and can identify different types of malwares, such as packed, polymorphic, obfuscated, metamorphic, or variations of a malware family. There are some limitations to the shortage of malware image datasets and computational costs for fine-tuning hyper parameters in the whole model. Furthermore, the proposed approach detects malware and groups it into families without using common techniques like disassembly, decompilation, de-obfuscation, or running malicious code in a virtual environment.