Intrusion Detection Model Based on Improved Transformer

Accurate intrusion detection is necessary to preserve network security. However, developing efficient intrusion detection system is a complex problem due to the nonlinear nature of the intrusion attempts, the unpredictable behaviour of network traffic, and the large number features in the problem space. Hence, selecting the most effective and discriminating feature is highly important. Additionally, eliminating irrelevant features can improve the detection accuracy as well as reduce the learning time of machine learning algorithms. However, feature reduction is an NPhard problem. Therefore, several metaheuristics have been employed to determine the most effective feature subset within reasonable time. In this paper, two intrusion detection models are built based on a modified version of the firefly algorithm to achieve the feature selection task. The first and, the second models have been used for binary and multiclass classification, respectively. The modified firefly algorithm employed a mutation operation to avoid trapping into local optima through enhancing the exploration capabilities of the original firefly. The significance of the selected features is evaluated using a Naïve Bayes classifier over a benchmark standard dataset, which contains different types of attacks. The obtained results revealed the superiority of the modified firefly algorithm against the original firefly algorithm in terms of the classification accuracy and the number of selected features under different scenarios. Additionally, the results assured the superiority of the proposed intrusion detection system against other recently proposed systems in both binary classification and multi-classification scenarios. The proposed system has 96.51% and 96.942% detection accuracy in binary classification and multi-classification, respectively. Moreover, the proposed system reduced the number of attributes from 41 to 9 for binary classification and to 10 for multi-classification.

The field of histopathological image analysis has evolved significantly with the advent of digital pathology, leading to the development of automated models capable of classifying tissues and structures within diverse pathological images. Artificial intelligence algorithms, such as convolutional neural networks, have shown remarkable capabilities in pathology image analysis tasks, including tumor identification, metastasis detection, and patient prognosis assessment. However, traditional manual analysis methods have generally shown low accuracy in diagnosing colorectal cancer using histopathological images. This study investigates the use of AI in image classification and image analytics using histopathological images using the histogram of oriented gradients method. The study develops an AI-based architecture for image classification using histopathological images, aiming to achieve high performance with less complexity through specific parameters and layers. In this study, we investigate the complicated state of histopathological image classification, explicitly focusing on categorizing nine distinct tissue types. Our research used open-source multi-centered image datasets that included records of 100.000 non-overlapping images from 86 patients for training and 7180 non-overlapping images from 50 patients for testing. The study compares two distinct approaches, training artificial intelligence-based algorithms and manual machine learning models, to automate tissue classification. This research comprises two primary classification tasks: binary classification, distinguishing between normal and tumor tissues, and multi-classification, encompassing nine tissue types, including adipose, background, debris, stroma, lymphocytes, mucus, smooth muscle, normal colon mucosa, and tumor. Our findings show that artificial intelligence-based systems can achieve 0.91 and 0.97 accuracy in binary and multi-class classifications. In comparison, the histogram of directed gradient features and the Random Forest classifier achieved accuracy rates of 0.75 and 0.44 in binary and multi-class classifications, respectively. Our artificial intelligence-based methods are generalizable, allowing them to be integrated into histopathology diagnostics procedures and improve diagnostic accuracy and efficiency. The CNN model outperforms existing machine learning techniques, demonstrating its potential to improve the precision and effectiveness of histopathology image analysis. This research emphasizes the importance of maintaining data consistency and applying normalization methods during the data preparation stage for analysis. It particularly highlights the potential of artificial intelligence to assess histopathological images.

The rapid expansion of the Internet of Things (IoT) and Big Data (BD) has led to security challenges. Securing IoT-BD against cyberattacks is necessary. An increasing number of applications are being implemented on BD platforms due to the rapid proliferation of data on the Internet. As the volume of data increases, the possibility of intrusions on the platform correspondingly increases. Conventional Intrusion Detection Systems (IDS) are ineffective for managing the extensive volume of historical data and unable to fulfil the security demands of BD platforms. This research aims to propose a novel intrusion detection model using Binary Sand Cat Swarm Optimization and Spatiotemporal Transformer Neural Network (BSCSO-STNN) model to address these issues. The CIC-IoT-23 and Bot-IoT datasets are collected and applied to train the model for evaluation. The developed BSCSO-STNN model is deployed in an Apache Spark (APS) framework. The datasets are initially preprocessed in this framework with data cleaning, oversampling, label encoding, and normalization. After preprocessing, the data is applied to the BSCSO for feature selection. Using the selected features, the STNN model performs binary and multiclass classification for both datasets. The BSCSO-STNN model attained 99.08% accuracy, 98.78% detection rate, 99.02% precision, and 98.94% F1-score using the CIC-IoT-23 dataset. The model attained 99.04% accuracy, 98.81% detection rate, 98.97% precision, and 98.95% F1-score for the BoT-IoT dataset in multiclass classification. The developed model outperformed all the current models in this research and demonstrated its accuracy in detecting intrusions.

This study uses deep learning methods to explore the Internet of Things (IoT) network intrusion detection method based on the CIC-IoT-2023 dataset. This dataset contains extensive data on real-life IoT environments. Based on this, this study proposes an effective intrusion detection method. Apply seven deep learning models, including Transformer, to analyze network traffic characteristics and identify abnormal behavior and potential intrusions through binary and multivariate classifications. Compared with other papers, we not only use a Transformer model, but we also consider the model’s performance in the multi-class classification. Although the accuracy of the Transformer model used in the binary classification is lower than that of DNN and CNN + LSTM hybrid models, it achieves better results in the multi-class classification. The accuracy of binary classification of our model is 0.74% higher than that of papers that also use Transformer on TON-IOT. In the multi-class classification, our best-performing model combination is Transformer, which reaches 99.40% accuracy. Its accuracy is 3.8%, 0.65%, and 0.29% higher than the 95.60%, 98.75%, and 99.11% figures recorded in papers using the same dataset, respectively.

Alzheimer's Disease (AD), the most common type of dementia, is characterized by memory issues that worsen with time. When evaluating single modalities, there are specific efficient learning techniques that produce a poor identification rate. The clinical significance of Mild Cognitive Impairment (MCI) has clinical importance, which is crucial for identifying and categorizing AD patients. This study includes the utilization of 3D T1 weighted /Magnetic Resonance Imaging (3D T1WI MRI) images where the structural changes in the brain are identified for predicting early AD. The classification accuracy for AD, MCI, and Normal Control (NC) has also increased due to cognitive evaluations when comparing three one vs one classifications, i.e. AD vs. NC, MCI vs. NC, AD vs. MCI, and multi-class classification. The proposed approach’s novelty is a binary and multi-class classification of structural MRI images using a Siamese 3D Convolutional Neural Network (Siamese 3D-CNN). The Stochastic Gradient Descent (SGD) has been employed to modify the proposed network's weights, and 5-Fold Cross-Validation (CV) has been used for each binary classification task. Our study used the neuropsychological data of 417 people from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset to validate the suggested methodology. Experimental results demonstrate that accuracy, sensitivity, and specificity for three binary classifications produce the maximum percentage, in contrast to other relevant approaches found in the literature. In addition, the multi-class classification accuracy of 99.8% was attained.

Advancements in machine learning and artificial intelligence have been widely utilised in the security domain, including but not limited to intrusion detection techniques. With the large training datasets of modern traffic, intelligent algorithms and powerful machine learning tools, security researchers have been able to greatly improve on the intrusion detection models and enhance their ability to detect malicious traffic more accurately. Nonetheless, the problem of detecting completely unknown security attacks is still an open area of research. The enormous number of newly developed attacks constitutes an eccentric challenge for all types of intrusion detection systems. Additionally, the lack of a standard definition of what constitutes an unknown security attack in the literature and the industry alike adds to the problem. In this paper, the researchers reviewed the studies on detecting unknown attacks over the past 10 years and found that they tended to use inconsistent definitions. This formulates the need for a standard consistent definition to have comparable results. The researchers proposed a new categorisation of two types of unknown attacks, namely Type-A, which represents a completely new category of unknown attacks, and Type-B, which represents unknown attacks within already known categories of attacks. The researchers conducted several experiments and evaluated modern intrusion detection systems based on shallow and deep artificial neural network models and their ability to detect Type-A and Type-B attacks using two well-known benchmark datasets for network intrusion detection. The research problem was studied as both a binary and multi-class classification problem. The results showed that the evaluated models had poor overall generalisation error measures, where the classification error rate in detecting several types of unknown attacks from 92 experiments was 50.09%, which highlights the need for new approaches and techniques to address this problem.

Ultrawideband (UWB) radio frequency positioning technology has found extensive applications in indoor and outdoor localization due to its strong anti-interference capabilities, high penetration power, and precise measurement accuracy. However, its positioning accuracy significantly decreases under nonline-of-sight (NLOS) conditions, particularly in dynamic NLOS environments. Therefore, it is essential to identify NLOS propagation and mitigate its associated errors. To overcome these issues, this study proposes a novel enhanced domain-adversarial neural network for UWB signal occlusion recognition (EDANN-SOR). EDANN-SOR integrates hybrid manually extracted channel impulse response (CIR) features, effectively aligning the source and target domains and extracting discriminative and domain-invariant UWB CIR features in dynamic NLOS environments. Through adversarial learning, our method achieves high generalization with minimal CIR sample requirements, reducing the need for extensive data collection and significantly enhancing robustness in distance measurement and positioning under dynamic NLOS conditions. In contrast to state-of-the-art NLOS identification and transfer learning (TL) techniques, our method excels in both binary and multiclass NLOS classification. Specifically, it achieves an accuracy of over 97.36% in conventional LOS/NLOS binary classification across scenarios and 97.07% in multiclass classification. The integration of manually extracted features has been proved effective in improving the EDANN-SOR model’s ability to distinguish among LOS, human, and glass obstacles.

Breast cancer is widely prevalent in women. To diagnose it, pathologists evaluate breast histopathology images under a microscope in various magnifications. However, the number of pathologists per population is low in many countries, and a mistake is probable. Nowadays, deep learning has become a popular artificial intelligence research trends. Even though deep neural networks have obtained promising results in image processing, the need for extensive training data avoids its use in medical image processing. After the evaluation of various state-of-the-art deep learning methods and algorithms in medical data processing, this study proposes an effective deep transfer learning-based model, relying on pre-trained DCNN using an extensive data of ImageNet dataset which improves state-of the-art systems in both binary and multi-class classification. The weights of the pre-trained DesneNet121 on the Imagenet are transferred as initial weights first, and then the model is fine-tuned with a deep classifier along with data augmentation to distinguish various malignant and benign samples in the two categories of binary and multi-class classification. In the multi-class classification, the proposed model obtained up to 97% image-level accuracy. In binary classification, the model obtained up to 100% image-level accuracy. The achieved-results are outperforming previous studies accuracies in multiple performance metrics in breast cancer CAD systems. Moreover, the proposed method is both flexible and scalable, meaning it can be easily expanded to cover the detection of other types of diseases in the future and be integrated with more CNNs to increase its generalization capabilities.

A deep learning fusion network trained with clinical and high-frequency ultrasound images in the multi-classification of skin diseases in comparison with dermatologists: a prospective and multicenter study

As security threats become more complex, the need for effective intrusion detection systems (IDSs) has grown. Traditional machine learning methods are limited by the need for extensive feature engineering and data preprocessing. To overcome this, we propose two enhanced hybrid deep learning models, an autoencoder–convolutional neural network (Autoencoder–CNN) and a transformer–deep neural network (Transformer–DNN). The Autoencoder reshapes network traffic data, addressing class imbalance, and the CNN performs precise classification. The transformer component extracts contextual features, which the DNN uses for accurate classification. Our approach utilizes an enhanced hybrid adaptive synthetic sampling–synthetic minority oversampling technique (ADASYN-SMOTE) for binary classification and enhanced SMOTE for multi-class classification, along with edited nearest neighbors (ENN) for further class imbalance handling. The models were designed to minimize false positives and negatives, improve real-time detection, and identify zero-day attacks. Evaluations based on the CICIDS2017 dataset showed 99.90% accuracy for Autoencoder–CNN and 99.92% for Transformer–DNN in binary classification, and 99.95% and 99.96% in multi-class classification, respectively. On the NF-BoT-IoT-v2 dataset, the Autoencoder–CNN achieved 99.98% in binary classification and 97.95% in multi-class classification, while the Transformer–DNN reached 99.98% and 97.90%, respectively. These results demonstrate the superior performance of the proposed models compared with traditional methods for handling diverse network attacks.

BACKGROUND: Classification of action intention understanding is extremely important for human computer interaction. Many studies on the action intention understanding classification mainly focus on binary classification, while the classification accuracy is often unsatisfactory, not to mention multi-class classification. METHOD: To complete the multi-class classification task of action intention understanding brain signals effectively, we propose a novel feature extraction procedure based on thresholding graph metric. RESULTS: Both the alpha frequency band and full-band obtained considerable classification accuracies. Compared with other methods, the novel method has better classification accuracy. CONCLUSIONS: Brain activity of action intention understanding is closely related to the alpha band. The new feature extraction procedure is an effective method for the multi-class classification of action intention understanding brain signals.

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

In the recent era, the growth of deep learning is inevitable. Various models such as convolutional neural networks (CNNs) and transformers are used widely in images for high classification accuracy. Since the invention of transformers, researchers have widely used novel approaches using transformers to achieve an impressive accuracy. In spite of this, this paper proposes a novel custom lightweight CNN model called Attentive and Lightweight Network (ALNet). ALNet consists of three major blocks: stem, core, and head. The core part is the novel classifier built as an inspiration from various pre-trained models such as ResNet, SENet (Squeeze and Excitation Network), EfficientNet, SqueezeNet, and ShuffleNet. The main objective is to build a model that has a high classification accuracy while reducing the number of parameters. This reduces the size of the model and hence makes it easy to deploy on cloud platforms and use in edge devices. The model was evaluated using 5-fold cross-validation on three different datasets. The primary dataset was a grapevine dataset with an accuracy of 99.78 percent and 100 percent in multi-class and binary classification respectively. To test the robustness of the model, a multi-class classification using the apple dataset achieved an accuracy of 99.95 percent and a binary classification with the cherry dataset achieved an accuracy of 100 percent. ALNet uses only 0.17 million parameters which is 18 times less parameters than the lightest model (SqueezeNet) and it takes only 14 seconds to train each epoch while pretrained models take 17-31 seconds. ALNet requires only 151.98 MFLOPs with a model size of 677.20 KB, making it approximately 18 times smaller than SqueezeNet. On the whole, ALNet is a highly accurate, lightweight model for plant leaf diseases prediction.

The softmax and binary classifier are commonly preferred for image classification applications. However, as softmax is specifically designed for categorical classification, it assumes each image has just one class label. This limits its applicability for problems where the number of labels does not equal one, most notably zero- and multi-label problems. In these challenging settings, binary classifiers are, in theory, better suited. However, as they ignore the correlation between classes, they are not as accurate and scalable in practice. In this paper, we start from the observation that the only difference between binary and softmax classifiers is their normalization function. Specifically, while the binary classifier self-normalizes its score, the softmax classifier combines the scores from all classes before normalisation. On the basis of this observation we introduce a normalization function that is learnable, constant, and shared between classes and data points. By doing so, we arrive at a new type of binary classifier that we coin quasibinary classifier. We show in a variety of image classification settings, and on several datasets, that quasibinary classifiers are considerably better in classification settings where regular binary and softmax classifiers suffer, including zero-label and multi-label classification. What is more, we show that quasibinary classifiers yield well-calibrated probabilities allowing for direct and reliable comparisons, not only between classes but also between data points.