Malware Analysis of Imaged Binary Samples by Convolutional Neural Network with Attention Mechanism

Neural malware analysis with attention mechanism

Covid-19 first occurred in Wuhan, China in December 2019. Subsequently, the virus spread throughout the world and as of June 2020 the total number of confirmed cases are above 4.7 million with over 315,000 deaths. Machine learning algorithms built on radiography images can be used as a decision support mechanism to aid radiologists to speed up the diagnostic process. The aim of this work is to conduct a critical analysis to investigate the applicability of convolutional neural networks (CNNs) for the purpose of COVID-19 detection in chest X-ray images and highlight the issues of using CNN directly on the whole image. To accomplish this task, we use 12-off-the-shelf CNN architectures in transfer learning mode on 3 publicly available chest X-ray databases together with proposing a shallow CNN architecture in which we train it from scratch. Chest X-ray images are fed into CNN models without any preprocessing to replicate researches used chest X-rays in this manner. Then a qualitative investigation performed to inspect the decisions made by CNNs using a technique known as class activation maps (CAM). Using CAMs, one can map the activations contributed to the decision of CNNs back to the original image to visualize the most discriminating region(s) on the input image. We conclude that CNN decisions should not be taken into consideration, despite their high classification accuracy, until clinicians can visually inspect and approve the region(s) of the input image used by CNNs that lead to its prediction.

For breast cancer, a malignant tumor developing from breast epithelial tissue, the limitations of traditional diagnostic methods (e.g., diagnostic errors and invasiveness) create critical challenges, underscoring the urgent need for artificial intelligence-assisted technological development. This paper systematically reviews the applications of Convolutional Neural Networks (CNNs) across the entire workflow of breast cancer, including screening, diagnosis, and prognosis, with a focus on hybrid CNNs such as architectures combining Transformer/Long Short-Term Memory (LSTM) and CNN-Support Vector Machine (SVM) models and attention-based CNNs such as contour-enhanced attention and cross-attention mechanisms. It analyzes how these models automate feature extraction from medical images to achieve breast lesion detection, benign-malignant differentiation, and prognosis prediction. Results show that hybrid models like Fusion of Hybrid Deep Features (FHDF) achieve over 98% classification accuracy on datasets such as MIAS by fusing features from multiple CNNs, while attention-based models like Convolutional Block Attention Module (CBAM)-Xception attain an Area Under Curve (AUC) of 0.97 and an accuracy of 89.1% in differentiating benign and malignant lesions. However, challenges remain, including insufficient interpretability, cross-institutional data heterogeneity, and privacy risks. The study proposes integrating medical expert systems and applying transfer learning and domain adaptation techniques to enhance model reliability and generalizability, promoting the translation of CNN technologies into clinical practice and constructing a precise and trustworthy AI-driven breast cancer diagnosis and treatment system.

In order to realize high-accuracy recognition of aerobics actions, a highly applicable deep learning model and faster data processing methods are required. Therefore, it is a major difficulty in the field of research on aerobics action recognition. Based on this, this paper studies the application of the convolution neural network (CNN) model combined with the pyramid algorithm in aerobics action recognition. Firstly, the basic architecture of the convolution neural network model based on the pyramid algorithm is proposed. Combined with the application strategy of the common recognition model in aerobics action recognition, the traditional aerobics action capture information is processed. Through the characteristics of different aerobics actions, different accurate recognition is realized, and then, the error of the recognition model is evaluated. Secondly, the composite recognition function of the convolution neural network model in this application is constructed, and the common data layer effect recognition method is used in the optimization recognition. Aiming at the shortcomings of the composite recognition function, the pyramid algorithm is used to improve the convolution neural network recognition model by deep learning optimization. Finally, through the effectiveness comparison experiment, the results show that the convolution neural network model based on the pyramid algorithm is more efficient than the conventional recognition method in aerobics action recognition.

Convolutional neural networks (CNN) applications range from recognition and reasoning (such as handwriting recognition, facial expression recognition and video surveillance) to intelligent text applications such as semantic text analysis and natural language processing applications. Two key observations drive the design of a new architecture for CNN. First, CNN workloads exhibit a widely varying mix of three types of parallelism: parallelism within a convolution operation, intra-output parallelism where multiple input sources (features) are combined to create a single output, and inter-output parallelism where multiple, independent outputs (features) are computed simultaneously. Workloads differ significantly across different CNN applications, and across different layers of a CNN. Second, the number of processing elements in an architecture continues to scale (as per Moore's law) much faster than off-chip memory bandwidth (or pin-count) of chips. Based on these two observations, we show that for a given number of processing elements and off-chip memory bandwidth, a new CNN hardware architecture that dynamically configures the hardware on-the-fly to match the specific mix of parallelism in a given workload gives the best throughput performance. Our CNN compiler automatically translates high abstraction network specification into a parallel microprogram (a sequence of low-level VLIW instructions) that is mapped, scheduled and executed by the coprocessor. Compared to a 2.3 GHz quad-core, dual socket Intel Xeon, 1.35 GHz C870 GPU, and a 200 MHz FPGA implementation, our 120 MHz dynamically configurable architecture is 4x to 8x faster. This is the first CNN architecture to achieve real-time video stream processing (25 to 30 frames per second) on a wide range of object detection and recognition tasks.

Arabic vocabulary recognition is an important aspect of learning at MTs Darun - Najah, a school that emphasizes on Islamic religious education. This research proposes the application of Convolutional Neural Network (CNN) and EfficientNet B7 to create learning media for Arabic vocabulary recognition for students. This method is implemented in the form of a web-based application. The built application offers an innovative approach in learning by utilizing deep learning. The results of several trials conducted showed that the application of Convolutional Neural Network (CNN) and EfficientNet B7 achieved 90% accuracy with an average precision of 94.6%, recall 94.6%, and f1-score 94.6%. Tests using User Acceptence Testing (UAT) have a success accuracy rate of 87.2% which proves that users can accept quite well.

Abstract. Deep learning models have demonstrated remarkable success in fields such as language processing and computer vision, routinely employed for tasks like language translation, image classification, and anomaly detection. Recent advancements in ocean sciences, particularly in data assimilation (DA), suggest that machine learning can emulate dynamical models, replace traditional DA steps to expedite processes, or serve as hybrid surrogate models to enhance forecasts. However, these studies often rely on ocean models of intermediate complexity, which involve significant simplifications that present challenges when transitioning to full-scale operational ocean models. This work explores the application of convolutional neural networks (CNNs) in data assimilation within the context of the HYbrid Coordinate Ocean Model (HYCOM) in the Gulf of Mexico. The CNNs are trained to correct model errors from a 2-year, high-resolution (1/25°) HYCOM dataset, assimilated using the Tendral Statistical Interpolation System (T-SIS). The CNNs are trained to replicate the increments generated by the T-SIS data assimilation package, aiming to correct model forecasts of sea surface temperature (SST) and sea surface height (SSH). The inputs to the CNNs include real satellite observations of SST from the Group for High Resolution Sea Surface Temperature (GHRSST), along-track altimeter SSH observations (ADT), the model background state (previous forecast), and the innovations (differences between observations and background). We assess the performance of the CNNs across five controlled experiments, designed to provide insights into their application in environments governed by full primitive equations, real observations, and complex topographies. The experiments focus on evaluating (1) the architecture and complexity of the CNNs, (2) the type and quantity of observations, (3) the type and number of assimilated fields, (4) the impact of training window size, and (5) the influence of coastal boundaries. Our findings reveal significant correlations between the chosen training window size – a factor not commonly examined – and the CNNs' ability to assimilate observations effectively. We also establish a clear link between the CNNs' architecture and complexity and their overall performance. This research uses artificial intelligence to enhance ocean forecasting in the Gulf of Mexico. By using convolutional neural networks, the study improves predictions of sea temperatures and heights by integrating real satellite data with existing models. Through five comprehensive experiments, the team found that the amount of training data and the design of the neural networks significantly affect accuracy. These insights pave the way for faster, more reliable ocean models, benefiting environmental monitoring and maritime operations.

This paper provides an overview of the applications of Convolutional Neural Networks (CNN) in the railway maintenance industry. Our research covers specifically the subdomain of railway track maintenance. In this study, we have analyzed the state-of-the-art of CNNs applied to railway track maintenance by conducting an extensive literature review, summarizing different tasks and problems related to the topic and presenting solutions based on CNNs with a special emphasis on the data used to create these models. The results of our research show different applications of CNNs within the scope, including the detection of defects in the surface of railway rails and railway track components, such as fasteners, joints, sleepers, switches and crossings, as well as the recognition of track components, and the continuous monitoring of railway tracks. The architecture of CNNs is fitting to learning spatial hierarchies of features directly from the input data, making them of great use for Computer Vision and other applications related to the topic at hand. The implementation of IoT devices and smart sensors aid the collection of real-time data which can be used to feed powerful CNN models to recognize patterns and identify complex events related to the maintenance of railway tracks. This and more insights are discussed in detail within the contents of this paper.

Early detection and treatment are crucial for recovery from cancer. Due to rapid advancement in medical imaging techniques, today doctors can scan various body parts from different perspectives to diagnose cancer or see the progress during a course of treatment. Such techniques are less painful and require less preparation and recovery. The recent evolution of AI, particularly deep learning algorithms have shown potential in computer-aided diagnosis of cancer. These include analysis of cancer images for detection of tumor region, prognosis, skin tissue inspection, etc. Convolutional Neural Network (CNN) based image processing techniques are showing promising results in the diagnosis of breast cancer, skin cancer, prostate cancer, lung cancer, etc. Literature shows many other CNN-based models such as feed-forward neural network, Deep CNN, Multi-Crop CNN, fully convolutional networks (FCNs), Deep Fully Convolutional Network (DFCNet), Recurrent neural networks, etc. The CNN focus solely on the image-specific features and hence require a lesser number of input parameters. A CNN model reduces the size of the input image vector without losing the features critical for making an accurate prediction. This makes it easier to process and reduces computational complexity. CNNs are also not sensitive to the object positioning in an image since the approximate position of image motifs relative to others is known. CNN deals with the variations in images through augmentation and other pre-processing techniques. This makes them robust to various straining conditions in images. This chapter presents an overview of a generic CNN architecture followed by a discussion on major advances in CNN models and their applications for cancer image analysis.

Convolutional neural networks are a technology that combines artificial neural networks and recent deep learning methods. They have been applied to many image recognition tasks and have attracted the attention of the researchers of many countries in recent years. This paper summarizes the latest development of convolutional neural networks and expounds the relative research of image recognition technology and elaborates on the application of convolutional neural networks in handwritten numeral recognition.

This study aims to address the diagnostic challenges in distinguishing gastric polyps from protrusions, emphasizing the need for accurate and cost-effective diagnosis strategies. It explores the application of Convolutional Neural Networks (CNNs) to improve diagnostic accuracy. This research introduces MultiAttentiveScopeNet, a deep learning model that incorporates multi-layer feature ensemble and attention mechanisms to enhance gastroscopy image analysis accuracy. A weakly supervised labeling strategy was employed to construct a large multi-class gastroscopy image dataset for training and validation. MultiAttentiveScopeNet demonstrates significant improvements in prediction accuracy and interpretability. The integrated attention mechanism effectively identifies critical areas in images to aid clinical decisions. Its multi-layer feature ensemble enables robust analysis of complex gastroscopy images. Comparative testing against human experts shows exceptional diagnostic performance, with accuracy, micro and macro precision, micro and macro recall, and micro and macro AUC reaching 0.9308, 0.9312, 0.9325, 0.9283, 0.9308, 0.9847 and 0.9853 respectively. This highlights its potential as an effective tool for primary healthcare settings. This study provides a comprehensive solution to address diagnostic challenges differentiating gastric polyps and protrusions. MultiAttentiveScopeNet improves accuracy and interpretability, demonstrating the potential of deep learning for gastroscopy image analysis. The constructed dataset facilitates continued model optimization and validation. The model shows promise in enhancing diagnostic outcomes in primary care.

Here we investigate the application of convolutional neural networks (CNNs) to predict the survival time of patients with Glioblastoma Multiforme (GBM) brain tumor. Our dataset consists of T1-weighted high-resolution MRI images of just 68 GBM patients. We compare two analytic methods for predicting survival time. The first consists of training a small convolutional neural network (CNN) and the second uses extracted deep features from a pre-trained CNN. Our method is completely automated, except for tumor region segmentation. In addition, we utilize a snapshot ensemble approach to boost test accuracy when dealing with limited availability of medical images for CNN training purposes. Our approach achieves an accuracy of 72.06% using a trained small network and 66.18% using a pre-trained deep CNN. Our results compare favorably with the accuracy of 54.41% using histogram of oriented gradients (HOG) features and a non-neural network classifier.

Objectives: The paper presents application of convolution neural network and artificial neural network for image classification problem for clothing dataset along with their performance comparison against different optimizers. The major objective of this paper is to perform image classification on fashion-mnist clothing dataset images. Methods: The methods used here are, the traditional ANN and CNN. Here image classification is performed on Fashion-mnist, clothing dataset using CNN and ANN with different optimizers. The performance of the working of ANN and CNN in classifying images from fashion-mnist dataset is compared against different optimizers namely stochastic gradient Descent, Adagrad, RMS prop and Adam optimizer. Findings: The study found that CNN worked better than ANN yielding training accuracy of 95%, 93% and testing accuracy of 91%, 89% when used with Adam and RmsProp respectively. Novelty: The novelty of this work is to present a comparative study of image classification using CNN, ANN using different optimizers, since not many studies or research articles showed the performance comparison of traditional and convolution neural networks in image classification along with different optimizers. Since the real-world scenarios of today require enormous data to be processed, CNN can fit well to diversify applications since they highly reduce the number of parameters to be trained that speeds up the training process. Moreover, to be specific on image classification problems they require the best and most prominent features to be detected and uncovered; this can be achieved using CNN since it has the concept of convolution using filters at its Core. Hence, CNN is highly recommended for such image classification applications than the traditional artificial-neural-networks because of the aforementioned reasons. Keywords: ANN (Artificial NeuralNetworks); CNN (ConvolutionNeural Networks); Optimization; SGD (Stochastic Gradient Descent); RmsProp (Root mean Square propagation); Adam (Adaptive moment estimation); AdaGrad (Adaptive Gradient)

The perceived texture directionality is an important, not fully explored image characteristic. In many applications texture directionality detection is of fundamental importance. Several approaches have been proposed, such as the fast Fourier-based method. We recently proposed a method based on the interpolated grey-level co-occurrence matrix (iGLCM), robust to image blur and noise but slower than the Fourier-based method. Here we test the applicability of convolutional neural networks (CNNs) to texture directionality detection. To obtain the large amount of training data required, we built a training dataset consisting of synthetic textures with known directionality and varying perturbation levels. Subsequently, we defined and tested shallow and deep CNN architectures. We present the test results focusing on the CNN architectures and their robustness with respect to image perturbations. We identify the best performing CNN architecture, and compare it with the iGLCM, the Fourier and the local gradient orientation methods. We find that the accuracy of CNN is lower, yet comparable to the iGLCM, and it outperforms the other two methods. As expected, the CNN method shows the highest computing speed. Finally, we demonstrate the best performing CNN on real-life images. Visual analysis suggests that the learned patterns generalize to real-life image data. Hence, CNNs represent a promising approach for texture directionality detection, warranting further investigation.

With the increased demand for English translators in recent years, more scholars and researchers have begun to focus on college English translation education. The demand for computers to perform automatic natural language translation is at an all-time high, presenting another potential for machine translation (MT) research. Although neural machine translation (NMT) technology has advanced significantly, it still has several flaws. The reply NN-based MT technology is not optimal in the field of long sentence translation, and there are situations such as missing translation and overtranslation. So this paper proposes college English translation based on a convolutional neural network (CNN). This paper proposes that CNNs have stronger feature extraction and information processing capabilities than other deep NNs. It successfully solves the situation of missing translation and overtranslation, lowering the quality of college English translation instruction. The experimental results of this research reveal that when the length of an English sentence grows longer, the translation accuracy based on the reply NN drops to 47.9%, while the translation accuracy based on the CNN is 89.7%. It can be seen that college English translation based on CNN can be integrated into English translation teaching as an innovation of traditional teaching methods, so as to create a learning environment for students and enhance learning effects. MT based on CNN has greatly improved students’ performance, which is conducive to improving the quality of college English translation teaching.