On Automating Hyperparameter Optimization for Deep Learning Applications

A comprehensive review on detection of plant disease using machine learning and deep learning approaches

The Internet of Things (IoT) has transformed many aspects of modern life, from healthcare and transportation to home automation and industrial control systems. However, the increasing number of connected devices has also led to an increase in security threats, particularly from botnets. To mitigate these threats, various machine learning (ML) and deep learning (DL) techniques have been proposed for IoT botnet attack detection. This systematic review aims to identify the most effective ML and DL techniques for detecting IoT botnets by delving into benchmark datasets, evaluation metrics, and data pre-processing techniques in detail. A comprehensive search was conducted in multiple databases for primary studies published between 2018 and 2023. Studies were included if they reported the use of ML or DL techniques for IoT botnet detection. After screening 1,567 records, 25 studies were included in the final review. The findings suggest that ML and DL techniques show promising results in detecting IoT botnet attacks, outperforming traditional signature-based methods. However, the effectiveness of the techniques varied depending on the dataset, features, and evaluation metrics used. Based on the synthesis of the findings, this review proposes a taxonomy for ML and DL techniques in IoT botnet attack detection and provides recommendations for future research in this area. This review illuminates the considerable potential of ML and DL approaches in bolstering the detection of IoT botnet attacks, thereby offering valuable insights to researchers involved in the domain of IoT security.

Network Intrusion Detection Systems (NIDSs) are an increasingly important tool for the prevention and mitigation of cyber attacks. Over the past years, a lot of research efforts have aimed at leveraging the increasingly powerful models of Machine Learning (ML) for this purpose. A number of labelled synthetic datasets have been generated and made publicly available by researchers, and they have become the benchmarks via which new ML-based NIDS classifiers are being evaluated. Recently published results show excellent classification performance with these datasets, increasingly approaching 100 percent performance across key evaluation metrics such as Accuracy, F1 score, AUC, etc. Unfortunately, we have not yet seen these excellent academic research results translated into practical NIDS systems with such near-perfect performance. This motivated our research presented in this paper, where we analyse the statistical properties of the benign traffic in three of the more recent and relevant NIDS datasets, (CIC_IDS, UNSW_NB15, TON_IOT), by converting them into a common flow format. As a comparison, we consider two datasets obtained from real-world production networks, one from a university network and one from a medium size Internet Service Provider (ISP). Our results show that the two real-world datasets are quite similar among themselves in regards to most of the considered statistical features. Equally, the three synthetic datasets are also relatively similar within their group. However, and most importantly, our results show a distinct difference of most of the considered statistical features between the three synthetic datasets and the two real-world datasets. Since ML relies on the basic assumption of training and test datasets being sampled from the same distribution, this raises the question of how well the performance results of ML-classifiers trained on the considered synthetic datasets can translate and generalise to real-world networks. We believe this is an interesting and relevant question which provides motivation for further research in this space.

PurposeThe first transmission of coronavirus to humans started in Wuhan city of China, took the shape of a pandemic called Corona Virus Disease 2019 (COVID-19), and posed a principal threat to the entire world. The researchers are trying to inculcate artificial intelligence (Machine learning or deep learning models) for the efficient detection of COVID-19. This research explores all the existing machine learning (ML) or deep learning (DL) models, used for COVID-19 detection which may help the researcher to explore in different directions. The main purpose of this review article is to present a compact overview of the application of artificial intelligence to the research experts, helping them to explore the future scopes of improvement.MethodsThe researchers have used various machine learning, deep learning, and a combination of machine and deep learning models for extracting significant features and classifying various health conditions in COVID-19 patients. For this purpose, the researchers have utilized different image modalities such as CT-Scan, X-Ray, etc. This study has collected over 200 research papers from various repositories like Google Scholar, PubMed, Web of Science, etc. These research papers were passed through various levels of scrutiny and finally, 50 research articles were selected.ResultsIn those listed articles, the ML / DL models showed an accuracy of 99% and above while performing the classification of COVID-19. This study has also presented various clinical applications of various research. This study specifies the importance of various machine and deep learning models in the field of medical diagnosis and research.ConclusionIn conclusion, it is evident that ML/DL models have made significant progress in recent years, but there are still limitations that need to be addressed. Overfitting is one such limitation that can lead to incorrect predictions and overburdening of the models. The research community must continue to work towards finding ways to overcome these limitations and make machine and deep learning models even more effective and efficient. Through this ongoing research and development, we can expect even greater advances in the future.

Money laundering has been a global issue for decades, which is one of the major threat for economy and society. Government, regulatory and financial institutions are combating it together in their respective capacity, however still billions of dollars in fines by authorities make the headlines in the news. High-speed internet services have enabled financial institutions to deliver better customer experience through multi-channel engagements, which has led to exponential growth in transactions and new avenues for laundering the money for fraudsters. Literature shows the usage of statistical methods, data mining and Machine Learning (ML) techniques for money laundering detection, but limited research on Deep Learning (DL) techniques, primarily due to lack of model interpretability and explainability of the decisions made. Several studies are conducted on application of ML for Anti-Money Laundering (AML), and Explainable Artificial Intelligence (XAI) techniques in general, but lacks the study on usage of DL techniques together with XAI. This paper aims to review the current state-of-the-art literature on DL together with XAI for identifying suspicious money laundering transactions and identify future research areas. Key findings of the review are, researchers have preferred variants of Convolutional Neural Networks, and AutoEncoder; graph deep learning together with natural language processing is emerging as an important technology for AML; XAI use is not seen in AML domain; 51% ML methods used in AML are non-interpretable, 58% studies used sample of old real data; key challenges for researchers are access to recent real transaction data and scarcity of labelled training data; and data being highly imbalanced. Future research directions are, application of XAI techniques to bring-out explainability, graph deep learning using natural language processing (NLP), unsupervised and reinforcement learning to handle lack of labelled data; and joint research programs between research community and industry to benefit from domain knowledge and controlled access to data.

This article explains how deep learning and machine learning techniques are being utilized to replace humans in a variety of activities, like item classification and sound addition. It emphasizes handwritten text recognition (HTR), which involves computers interpreting handwritten content from sources like paper documents and touchscreens. For recognizing handwritten digits, the study analyzes three algorithms: support vector machines (SVM), multilayer perception (MLP), and convolutional neural networks (CNN). This comparison aims to evaluate the correctness and execution times of different approaches. The evaluation is based on parameters like the dataset used, the number of epochs, algorithm complexity, accuracy, and hardware specifications. The study compares the effectiveness of CNN, Multilayer Perception (MLP), and Support Vector Machines using handwritten digits from the MNIST dataset. It evaluates these algorithms based on criteria such as runtime, complexity, rate of precision, the count of epochs, and the quantity of hidden layers specifically within deep learning techniques. SVM had the highest training data accuracy, but CNN outperformed others on the test dataset. The article’s primary goal is to identify the most precise model for recognizing handwritten digits. It achieves this by assessing the performance of different algorithms and comparing them based on various characteristics.

Agriculture has been a vital sector for the majority of people, especially in countries like India. However, the increasing need for food production has led to intensive farming practices that have resulted in the deterioration of soil quality. This deterioration in soil quality poses significant challenges to both agricultural productivity and environmental sustainability. To address these challenges, advanced soil nutrient prediction systems that utilize machine learning and deep learning techniques are being developed. These advanced soil nutrient prediction systems utilize various sources of data, such as soil parameters, plant diseases, pests, fertilizer usage, and changes in weather patterns. By mapping and analyzing these data sources, machine learning algorithms can accurately predict the distribution of soil nutrients and other properties essential for precise agricultural practices. A previous study compared machine learning algorithms like SVM and Random Forest with deep learning algorithms CNN and LSTM for predicting crop yields. The most appropriate model is a significant challenge, but several studies have evaluated recommendation system models using deep machine learning techniques. Deep learning models attain accuracy above 90%, while many ML models achieve rates between 90% and 93%. Furthermore, the research seeks to propose specific crop suggestions grounded in soil nutrients for precision agriculture to enhance crop productivity.

Electric vehicles (EVs) are a promising zero‐emission technology in the automobile industry, but they face several challenges in terms of performance, reliability, and safety. Batteries are the heart of the EV system which helps to run the vehicle with reliability. Batteries during the process of running undergo various changes that need to be addressed. On the other hand, real‐time data analysis and online access to information are necessary conditions in the modern world. Machine learning and deep learning algorithms mimic humans by focusing on statistical data and algorithms on a real‐time basis. Therefore, in today’s research, machine learning and deep learning algorithms are used in EV technologies to obtain a more efficient and capable system. The battery management system (BMS) is the main part that is often in need of data processing of battery parameters and diagnosis of the problem. This paper explores the comprehensive literature review on machine learning and deep learning approaches for BMS in EVs. The state of charge (SOC) estimation, charge equalization and cell balancing, fault detection and diagnosis, and thermal management systems using various combined machine learning and deep learning techniques are discussed. By synthesizing insights from various studies, this article presents improved parameters and valuable inferences. This article aims to highlight the pivotal role of artificial intelligence (AI) and deep learning in improving the functionality of the BMS, ultimately contributing to the performance and longevity of EVs.

Deep learning (DL) is now generally acknowledged as the benchmark and evolution in machine learning (ML) fields. Further more, it has steadily suited the most extensively in computational techniques for ML, delivering excellent outcomes in several challenging intellectual works that have equal or still outperformed human ability. DL has the advantage of learning from enormous amounts of data. Here, the DL approach has been applied to classify thunderstorms and non-thunderstorms with a meteorological dataset. Daily observational hourly data sets from 2016 to 2021 have been used to classify the incidence of thunderstorms and non-thunderstorms. Three DL approaches are used: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) to classify thunderstorm incidences. These DL techniques are compared with conventional ML techniques, showing that DL approaches outperformed ML. The CNN method provides the highest performance with 92.01% accuracy among all DL and ML approaches.

Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together.

Coastal aquifers are vital for sustaining ecosystems and providing freshwater for agriculture and domestic use. However, rising groundwater salinity in these regions demands innovative approaches for accurate modeling and prediction. In recent decades, machine learning (ML) and deep learning (DL) techniques have been extensively used across various water resource management fields, often yielding promising outcomes. This review examines the growing usage of ML and DL techniques in modeling groundwater salinity in coastal aquifers. The analysis is based on 104 peer-reviewed journal articles including the review articles indexed in PubMed, ScienceDirect, Google Scholar, Scopus, and SpringerLink using keywords like "saltwater intrusion", "machine learning", “deep learning”, and "groundwater salinity". The review discusses recent advancements, challenges, and future directions for improving ML model accuracy and applicability. Studies reveal an increasing reliance on ML techniques to predict groundwater salinity and manage seawater intrusion across various coastal geological settings. Researchers have employed a range of ML models, from traditional methods like Artificial Neural Networks and Adaptive Neuro Fuzzy Inference Systems to advanced DL-based models like Convolutional Neural Networks and Long Short-Term Memory Networks, often using ensemble methods to enhance accuracy. Key findings include the top-tier output of ensemble and optimized models, including Random Forest and Grasshopper Optimization Algorithm-XGBoost. These studies highlight the importance of model selection, input variable ranking, and computational efficiency, demonstrating ML's global relevance in environmental management. The review highlights the prospects of ML, DL, and ensemble models to redefine groundwater management in coastal aquifers, offering more effective and sustainable solutions to the challenges posed by groundwater salinity and saltwater intrusion. Future advancements will focus on further integrating DL, explainable artificial intelligence, and ensemble approaches, along with other innovative techniques, to explore sparsely studied variables, model new or unique study areas, and apply ML methods for managing groundwater salinity.

IntroductionAccurate classification of brain tumors from MRI scans is a critical task for improving patient outcomes. Machine learning (ML) and deep learning (DL) methods have shown promise in this domain, but their relative performance remains unclear.MethodsThis study evaluates several ML and DL techniques using the BraTS 2024 dataset. The models assessed include traditional algorithms such as Random Forest and advanced deep learning architectures including Simple CNN, VGG16, VGG19, ResNet50, Inception-ResNetV2, and EfficientNet. Preprocessing strategies were applied to optimize model performance.ResultsThe Random Forest classifier achieved the highest accuracy of 87%, outperforming all deep learning models, which achieved accuracy in the range of 47% to 70%. This indicates that traditional ML approaches can sometimes surpass state-of-the-art DL methods in tumor classification tasks.DiscussionThe findings highlight the importance of model selection and parameter tuning in automated brain tumor diagnosis. While deep learning models are generally considered standard for image analysis, Random Forest demonstrated superior performance in this context. This underscores the need for fine-grained consideration of dataset characteristics, computational resources, and diagnostic requirements.ConclusionThe study shows that carefully selected and optimized ML approaches can improve tumor classification and support more accurate and efficient diagnostic systems for brain tumor patients.

In an open source software development environment, it is hard to decide the number of group members required for resolving software issues. Developers generally reply to issues based totally on their domain knowledge and interest, and there are no predetermined groups. The developers openly collaborate on resolving the issues based on many factors, such as their interest, domain expertise, and availability. This study compares eight different algorithms employing machine learning and deep learning, namely—Convolutional Neural Network, Multilayer Perceptron, Classification and Regression Trees, Generalized Linear Model, Bayesian Additive Regression Trees, Gaussian Process, Random Forest and Conditional Inference Tree for predicting group size in five open source software projects developed and managed using an open source development framework GitHub. The social information foraging model has also been extended to predict group size in software issues, and its results compared to those obtained using machine learning and deep learning algorithms. The prediction results suggest that deep learning and machine learning models predict better than the extended social information foraging model, while the best-ranked model is a deep multilayer perceptron((R.M.S.E. sequelize—1.21, opencv—1.17, bitcoin—1.05, aseprite—1.01, electron—1.16). Also it was observed that issue labels helped improve the prediction performance of the machine learning and deep learning models. The prediction results of these models have been used to build an Issue Group Recommendation System as an Internet of Things application that recommends and alerts additional developers to help resolve an open issue.

The detection of lung and colon cancer is a critical challenge in medical diagnosis, and machine learning (ML) and deep learning (DL) techniques are increasingly being used to enhance accuracy and efficiency. This review focuses on the integration of ML and DL methods for the combined detection of lung and colon cancer, emphasizing their strengths, limitations, and future potential. The motivation behind this study is to address the growing demand for accurate and early detection of these cancers, which significantly impacts treatment outcomes. Current methods often struggle with feature complexity, image variability, and computational intensity, which limit their real-world applicability. The aim is to consolidate various ML and DL techniques that have been employed for this purpose, highlighting how hybrid models can improve detection rates. The objective of this review is to provide a comprehensive analysis of different methodologies, their datasets, pre-processing techniques, feature extraction methods, and evaluation parameters. This review also explores recent advancements, such as transfer learning combined with fine-tuning techniques, which can further optimize performance in cancer detection. The findings suggest that while current methods show promise, further improvements in model generalization, interpretability, and computational efficiency are required to overcome existing limitations and expand clinical use.

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.