Enhancing CNN-based network intrusion detection through hyperparameter optimization

The network intrusion detection system is a core technology of network security that detects packets for malicious activities occurring in the network and is an essential element for stable services in extended network environments such as big data and IoT. These network intrusion detection systems have been studied together with machine learning and deep learning, but performance is not guaranteed in the actual environment or the class balance problem has not been solved. Therefore, in this study, we investigate the performance of a discretization preprocessing method with a CNN-based classifier on the class imbalance problem of network traffic data. The preprocessing method adds a discretization algorithm for continuous variables in the commonly used conventional preprocessing method and converts 1D network packet vectors into 2D image vectors to improve relational analysis and generalization performance. Since the convolution neural network has immutability to the input data, it improves statistical efficiency in learning network packets converted into images. To evaluate the proposed model, we compared the computational complexity and generalization performance using NSL-KDD and CSE-CIC-IDS 2018, which is a representative network packet data. As a result of the experiment, it was confirmed that in the case of computational complexity, training time and parameters were reduced compared to the model designed similarly to the proposed model, and accuracy and F1 score improved in generalization performance.

Automatic prediction of citrus fruit diseases requires the selection of a suitable classification model among various Convolutional Neural Network (CNN) models. The CNN models are the most effective for image classification. However, more than one CNN model occasionally performs closer to each other. Choosing the best classification model requires considerable effort in this situation. In this study, 17 CNN models were used for citrus fruit disease identification. Citrus fruit diseases include anthracnose, blackspot, canker, scab, and healthy. In this research, the performance of CNN models was analysed, and a two-phase statistical analysis was carried out to determine which CNN model provided the most accurate predictions of citrus crop disease. The 17 CNN models used were AlexNet, Darknet, Densenet, efficientnet, nasnetlarge, googlenet, inceptionresnetv2, inceptionv3, mobilenetv2, nasnetmobile, Xception, squeezenet, resnet18, resnet50, resnet101, vgg16, and vgg19. The two-phase statistical analysis was Duncan's multiple range test, followed by the Wilcoxon signed-rank test is the second one. Again, in order to do a statistical analysis, nine different indicators, including “accuracy, sensitivity, specificity, precision, FPR, F1 Score, MCC, Kappa, and computing time”, were taken into consideration. After the execution of 17 CNN models and twophase statistical analysis, it was revealed that resnet101 is superior among CNN models for citrus fruit disease prediction.

This paper compares the performance of five different supervised machine learning (SL) algorithms to identify the practical model for tracking copper using solution cathode glow discharge optical emission spectroscopy (SCGD-OES). A total of 4500 SCGD-OES spectra were collected, corresponding to 15 different copper concentrations ranging from 10.3 mg l−1 to 59.6 mg l−1, for training and validating five models: K-nearest neighbors (KNN), random forest (RF), artificial neural network (ANN), convolutional neural network (CNN), and recurrent neural network (RNN). Then, performance of these models were evaluated based on accuracy, precision of predictions and training time. In terms of accuracy, the relative error (RE) was 3% for the CNN model and 4% for the ANN model. Conversely, the RE increased from 21% and 43% for the RF and RNN models to 103% for the KNN model due to the unpredictability of Cu concentration outside the training range. Regarding precision, the relative standard deviation (RSD) values were similar across all models, with the lowest RSD being 3% for the CNN model and the highest being 5% for the RF model. The training time indicated that the KNN and RF models provided the best learning speed, while the CNN model had the slowest learning speed, taking 5 times longer than the ANN model. The accuracy of the ANN and CNN models is in agreement with inductively coupled plasma-optical emission spectroscopy (ICP-OES), with a average difference not exceeding 6% when tracking six different Cu concentrations, both inside and outside the training range. Additionally, this research investigated the weight distribution in the hidden layers of the ANN, CNN, and RNN models and feature importances in the RF model to enhance understanding of the internal structure of models and assist analysts in selecting the practical model for tracking heavy metals based on SCGD-OES.

Understanding the learning mechanism of convolutional neural networks in spectral analysis

Using smart sensors and data analytics tools, predictive maintenance (PdM) aims to reduce downtime by forecasting potential faults; however, accurately distinguishing between failure and normal conditions remains a significant challenge. To address this challenge, we propose a fault classification system that transforms 1D PdM sensor signals into 2D images using a novel Predictive Maintenance Image Transformation (PdM-IT) technique, followed by feature extraction with pre-trained convolutional neural networks (CNNs). The extracted features are then combined across all CNN models. We additionally optimize the combined feature set using three nature-inspired optimization algorithms, including Ant Colony Optimization (ACO), Particle Swarm Optimization (PSO), and Grey Wolf Optimization (GWO), and then evaluate four different classifier such as Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), Naive Bayes (NB), and Decision Trees (DT) on the obtained data using cross-validation. In our experimental evaluation, the best-performing model, SVM with features selected via ACO, achieved 98.22% accuracy, 98.39% F1-score, 98.51% precision, and 98.26% recall. These results show the effectiveness of the proposed hybrid image-based fault classification approach and its potential to support more reliable fault diagnosis and operational planning in PdM systems.

This study comparatively evaluates the performance of Convolutional Neural Networks (CNN), Vision Transformer (ViT), and Swin Transformer architectures for diagnosing plant leaf diseases. The experimental framework was evaluated using model effectiveness, F1 scores, and confusion matrices on the PlantVillage dataset, which contains 55,448 images from 39 different classes. The results show that the Swin Transformer achieved the highest diagnostic performance with a 100% F1 score, outperforming both CNN and ViT models. These findings confirm the high potential of transformer-based architectures for automated disease detection in agriculture.

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

The subject matter of this article is a HybridAttention mechanism integrated into a deep neural architecture for Network Intrusion Detection Systems (NIDS). This study aims to develop and study a HybridAttention mechanism based on a combination of global (self-attention) and local (dynamic local attention) models to improve the quality of traffic classification in real-time NIDS. The tasks to be solved are as follows: analyzing the applicability of existing attention mechanisms in network intrusion detection; integrating various attention types into a CNN-BiGRU architecture; developing a HybridAttention mechanism based on dynamic window alignment; optimizing the model using Optuna; and experimentally evaluating its performance on benchmark datasets using standard classification metrics. The methods used are: deep learning modeling with CNN-BiGRU architecture, integration of various attention mechanisms, including a novel HybridAttention, hyperparameter optimization using Optuna, and performance evaluation based on standard classification metrics. The results of this study show that the proposed HybridAttention mechanism demonstrates superiority over individual types of attention in all key metrics. The model achieved up to 99.85% accuracy on the NSL-KDD dataset training data and demonstrated strong generalization on the UNSW-NB15 dataset, achieving up to 98.06% accuracy in multi-class classification and up to 99.20% in binary classification. The proposed model also outperformed state-of-the-art approaches for processing unbalanced data and detecting various types of attacks. Conclusions. The scientific novelty of the results obtained is as follows: a HybridAttention mechanism combining self-attention and dynamic local attention was developed to enhance sequential pattern recognition in network traffic; the CNN-BiGRU architecture was improved by integrating multiple attention modules; systematic hyperparameter optimization using Optuna improved generalization on imbalanced data; and the proposed model outperformed existing approaches on benchmark datasets in detecting both known and novel cyberattacks.

Distinguishing pericarpium citri reticulatae of different origins using terahertz time-domain spectroscopy combined with convolutional neural networks

This study leveraged Convolutional Neural Network (CNN) models to estimate canopy height in Southeast Australian forests before and after the 2019–2020 bushfire event, using inputs from Sentinel-1, Sentinel-2, spectral indices, and terrain features and GEDI canopy height (rh99) as target variables. Our research consisted of three primary objectives: (1) an evaluation of GEDI rh99 canopy height in relation to an nDSM derived from airborne LiDAR, (2) the development of two CNN models for pre- and post-fire scenarios, and (3) utilizing the trained CNN models to generate pre- and post-fire canopy height maps and canopy height change maps across the continuous landscape. GEDI rh99 accuracy analysis revealed R 2 = 0.85, RMSE = 7.25 m and nRMSE = 0.23 for pre-fire GEDI and R 2 = 0.78, RMSE = 10.1 m and nRMSE = 0.3 for post-fire GEDI. The pre-fire CNN model achieved R 2 = 0.75, RMSE = 7.29 m and nRMSE = 0.22, and post-fire CNN model achieved R 2 = 0.62, RMSE = 11.67 m, and nRMSE = 0.34 when validated against nDSM. Bias analysis was conducted, revealing that GEDI rh99 underestimated nDSM across all canopy height ranges, and slope has no significant impact on GEDI accuracy. Notably, post-fire GEDI rh99 exhibited significant underestimation in high to extreme fire severity. The CNN models’ prediction displayed overestimation in short forests < 20 m and increasing underestimation in taller forests > 20 m and underestimation in high and extreme fire severity. Subsequent investigation indicated that canopy layer thinning in high fire severity conditions resulted in weak GEDI waveform signals and consequent bias. Additionally, the GEDI rh99 bias was propagated to the CNN models, together with insensitivity of spectral band values to canopy height in dense and burnt forests potentially contributed to models’ bias. Finally, we demonstrated CNN models’ ability to monitor forest recovery by generating canopy height maps spanning East Gippsland from 2019 to 2023.

Hybrid convolution neural network model for a quicker detection of infested maize plants with fall armyworms using UAV-based images

The uncertainty of wind power as the main obstacle of its integration into the power grid can be addressed by an accurate and efficient wind power forecast. Among the various wind power forecasting methods, machine learning (ML) algorithms, are recognized as a powerful wind power forecasting tool, however, their performance is highly dependent on the proper tuning of their hyperparameters. Common hyperparameter tuning methods such as grid search or random search are time-consuming, computationally expensive, and unreliable for complex models such as deep learning neural networks. Therefore, there is an urgent need for automatic methods to discover optimal hyperparameters for higher accuracy and efficiency of prediction models. In this study, a novel investigation is contributed to the field of wind power forecasting by a comprehensive comparison of three advanced techniques – Scikit-opt, Optuna, and Hyperopt – for hyperparameter optimization of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) models, a facet that, to our knowledge, has not been systematically explored in existing literature. The impact of these optimization techniques on the accuracy and efficiency of the CNN and LSTM models are assessed by comparing the root mean square error (RMSE) of the predictions and the required time to tune the models. The results show that the Optuna algorithm, using a Tree-structured Parzen Estimator (TPE) search method and Expected Improvement (EI) acquisition function, has the best efficiency for both CNN and LSTM models. In terms of accuracy, it is demonstrated that while for the CNN model all the optimization methods achieve similar performances, the LSTM model optimized by the Hyperopt algorithm, based on the annealing search method, results in the highest accuracy. In addition, for the first time in this research, the impact of the random initialization features on the performance of the forecasting models with neural networks is investigated. The proposed structures for deep learning models were examined to determine the most robust structure with the minimal sensitivity to the randomness. What we have discovered from the comparison of advanced hyperparameter optimization methods can be used by researchers to tune the time series-based forecasting models.

Accurately identifying epileptogenic zone (EZ) using high-frequency oscillations (HFOs) is a challenge that must be mastered to transfer HFOs into clinical use. We analyzed the ability of a convolutional neural network (CNN) model to distinguish EZ and non-EZ HFOs. Nineteen medically intractable epilepsy patients with good surgical outcomes 2 years after surgery were studied. Five-minute interictal intracranial electroencephalogram epochs of slow-wave sleep were selected randomly. Then 5 s segments of ripples (80–200 Hz) and fast ripples (FRs, 200–500 Hz) were detected automatically. The EZs and non-EZs were identified using the surgery resection range. We innovatively converted all epochs into four types of images using two scales: original waveforms, filtered waveforms, wavelet spectrum images, and smoothed pseudo Wigner–Ville distribution (SPWVD) spectrum images. Two scales were fixed and fitted scales. We then used a CNN model to classify the HFOs into EZ and non-EZ categories. As a result, 7,000 epochs of ripples and 2,000 epochs of FRs were randomly selected from the EZ and non-EZ data for analysis. Our CNN model can distinguish EZ and non-EZ HFOs successfully. Except for original ripple waveforms, the results from CNN models that are trained using fixed-scale images are significantly better than those from models trained using fitted-scale images (p < 0.05). Of the four fixed-scale transformations, the CNN based on the adjusted SPWVD (ASPWVD) produced the best accuracies (80.89 ± 1.43% and 77.85 ± 1.61% for ripples and FRs, respectively, p < 0.05). The CNN using ASPWVD transformation images is an effective deep learning method that can be used to classify EZ and non-EZ HFOs.

To better understand and improve the prediction of the seasonal surface temperature (TS) across the United States, we employed convolutional neural network (CNN) models trained on the Community Earth System Model Version 2 Large Ensemble (CESM2 LENS). We used lagged sea surface temperatures (SST) over the tropical Pacific region, containing the information of the El Niño Southern Oscillation (ENSO), as input for the CNN models. ENSO is the principal driver of variability in seasonal US surface temperatures (TSUS) and employing CNN models allows for spatiotemporal aspects of ENSO to be analyzed to make seasonal TSUS predictions. For predicting TSUS, the CNN models exhibited significantly improved skill over standard statistical multilinear regression (MLR) models and dynamical forecasts across most regions in the US, for lead times ranging from 1 to 6 months. Furthermore, we employed the CNN models to predict seasonal TSUS during extreme ENSO events. For these events, the CNN models outperformed the MLR models in predicting the effects on seasonal TSUS, suggesting that the CNN models are able to capture the ENSO‐TSUS teleconnection more effectively. Results from a heatmap analysis demonstrate that the CNN models utilize spatial features of ENSO rather than solely the magnitude of the ENSO events, indicating that the improved skill of seasonal TSUS is due to analyzing spatial variation in ENSO events. The proposed CNN model demonstrates a promising improvement in prediction skill compared to existing methods, suggesting a potential path forward for enhancing TSUS forecast skill from subseasonal to seasonal timescales.

BACKGROUND AND AIM: Outdoor ultrafine particles (UFP; 100 nm) and black carbon (BC) vary greatly within cities and may have adverse impacts on human health such as cardiovascular mortality and brain tumour incidence. Traditional regression methods for estimating spatial distributions of ambient air pollution depend on extensive, curated geospatial information system databases. This has resulted in a disparity wherein little is known about local air pollution in the data-sparse settings of low- and middle- income countries. We used a hybrid approach to develop new models to estimate within-city spatial variations in outdoor UFP and BC concentrations across Bucaramanga, Colombia. METHODS: We conducted a large-scale mobile monitoring campaign over twenty days in 2019. Land use regression (LUR) models were developed using land use parameters from curated and open-source databases. Convolutional Neural Network (CNN) models were developed using Google Maps satellite view and street view images. Combined models were developed by combining predictions from the LUR and CNN models. After training in the training and validation sets, predictions were generated in the test set and compared to measured values. RESULTS:The combined UFP model (R2=0.54) outperformed the CNN (R2=0.47) and land use regression (LUR) models (R2=0.47) on their own. Similarly, the combined BC model also outperformed the CNN and LUR BC models (R2 = 0.51 vs 0.43 and 0.45 respectively). Spatial variations in model performance were more stable for the CNN and combined models compared to the LUR models suggesting that the combined approach may be less likely to contribute to differential exposure measurement error in epidemiological studies. CONCLUSIONS:Estimates from these models can then be applied to population-based cohorts in order to evaluate population health risks. Additionally, our findings demonstrated that satellite and street-level images can be combined with a traditional LUR modelling approach to improve predictions of within-city spatial variations in outdoor UFP and BC concentrations. KEYWORDS: Ultrafine particles, Black Carbon, Deep learning, Images, Land use regression