Malware Variant Detection Based on Decomposed Deep Convolutional Network

Predictive analysis on cellular traffic is important for the control and monitoring of wireless networks. Cellular traffic prediction is a challenging problem due to the non- stationarity and dynamic spatial-temporal correlation of the traffic. In this paper, we address the problem of accurate traffic prediction in a base station by proposing a deep neural network called RAConv. Its structure includes residual network, attention mechanism, and deep convolutional network. In the proposed architecture, a deep 3D residual convolutional network (ResConv3D) with three residual blocks are employed to learn the local spatial-temporal features. An attention-aided convolutional long short-term memory network (AConvLSTM) is then used to capture the long-term spatial-temporal dependencies. The use of the attention modules enable the network to focus on the most important spatial-temporal information. We evaluate the performance of the proposed RAConv network using a dataset provided by a Canadian wireless service provider. We consider the traffic prediction on two time scales (i.e., hourly and daily), which exhibit different spatial-temporal dependency patterns. Experimental results show that the proposed RAConv network can achieve accurate prediction under both time scales. Results also show that our proposed network provides a lower root-mean-square error (RMSE) than the conventional ConvLSTM baseline scheme.

Computer-aided diagnosis of ocular disorders like glaucoma can be effectively performed with retinal fundus images. Since the advent of machine learning and later deep learning techniques in medical image analysis, research focusing on automated and early detection of glaucoma has gained prominence. In this paper, we show how a deep Convolutional Neural Network model can be used in visualising the features related to Glaucoma that are consistent with (a subset of) those used by ophthalmologists for diagnosis. However, since the doctors do not necessarily depend entirely on these features but back them up with other important measurements such as visual field testing (HVF), any deep neural network such as CNN may not necessarily be expected to yield 100% accuracy. Further, other important parameters such as Intraocular Pressure (IOP), which may be built into a deep learning model, may not necessarily correlate well with the presence of Glaucoma. In this work, a deep CNN model has been trained and tested on a large number of high-quality fundus images, both normal and glaucomatous. We compare the results with transfer learning models such as VGG16, ResNet50, and MobileNetV2. On an average, we obtained an accuracy of 93.75% in identifying glaucoma, focusing only on the features of the fundus image.

This paper proposes a method of accurately assessing image quality without a reference image by using a deep convolutional neural network. Existing training based methods usually utilize a compact set of linear filters for learning features of images captured by different sensors to assess their quality. These methods may not be able to learn the semantic features that are intimately related with the features used in human subject assessment. Observing this drawback, this work proposes training a deep convolutional neural network (CNN) with labelled images for image quality assessment. The ReLU in the CNN allows non-linear transformations for extracting high-level image features, providing a more reliable assessment of image quality than linear filters. To enable the neural network to take images of any arbitrary size as input, the spatial pyramid pooling (SPP) is introduced connecting the top convolutional layer and the fully-connected layer. In addition, the SPP makes the CNN robust to object deformations to a certain extent. The proposed method taking an image as input carries out an end-to-end learning process, and outputs the quality of the image. It is tested on public datasets. Experimental results show that it outperforms existing methods by a large margin and can accurately assess the image quality on images taken by different sensors of varying sizes.

Cardiovascular MRI image analysis by using the bio inspired (sand piper optimized) fully deep convolutional network (Bio-FDCN) architecture for an automated detection of cardiac disorders

Large convolutional networks have achieved impressive classification performances recently. To achieve better performance, convolutional network tends to develop into deeper. However, the increase of network depth causes the linear growth of computational complexity, but cannot bring equivalent increase to the classification accuracy. To alleviate this inconsistence, we propose a cascading approach to accelerate the classification of very deep convolutional neural network. By exploiting the entropy metric to analyze the statistic differences of basic networks between the correctly and mistakenly classified images, we can assign the easily distinguished images to the shallow networks for reducing the computational complexity, and leave the difficultly classified images to the deep networks for maintaining the overall performance. Besides, the proposed cascaded networks can take advantage of the complementarity between different networks, which may boost the classification accuracy compared to the deepest network. We perform the experiments using residual networks of different depths on cifar100 dataset, on the condition of obtaining the similar accuracy to the deepest network, the results show that our cascaded ResNet32-ResNet110 and cascaded ResNet32-ResNet164 can reduce the computation time by 48.6% and 44.3% compared to ResNet110 and ResNet164, respectively. And the cascaded ResNet32-ResNet110-ResNet164 can reduce the computation time by 85.4% compared to the very deep Resnet1001.

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

COVID-19 virus development has been recognized as a broadly perceived kind of disease. This research work has accomplished several deep convolutional networks (DRN) with pre-training techniques for classifying X-ray chest images into three broad collection, viz., normal or negative, pneumonia positive and COVID-19 positive, based on two other open-source data sets. This fivefold research work is based on data sets consisting of 980 X-ray chest images of contaminated COVID-19 patients, along with experimentation using various deep (Wang et al. in A deep learning algorithm using CT images to screen for COVID-19 virus disease (COVID-19), [10]) learning (Farooq and Hafeez in COVID-ResNet: a deep learning framework for screening of COVID-19 from radiographs, [22]) and neural network methods; organized as primarily it introducing some pre-training techniques that helps the network to learn better, especially in an imbalanced data set, where less cases of COVID-19 are seen along with more cases of other classes. Second, proposing a new and modified convolutional (Wang and Wong in COVID-net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images, [11]) neural network (MCNN) that is convolutional with the simple application of a filter to an input, which results in activation. Third, repeat the application of the same filter to input entails in a map of activation (viz., feature map), indicates the locations/strength of detected feature in the input; which is an image concatenation of the neural networks and VGG Net networks. Fourth, the computational work reveals that the resulted network achieves the best accuracy by utilizing multiple features extracted by two other robust networks. Fifth, our network evaluation (based on 980 images) reveals that the actual accuracy is surely possible and adaptable in real-life circumstances. At the same time, the average accuracy of our proposed network for detecting COVID-19 cases is 95.60%, with overall average accuracy for all other classes at 82.37%. Finally, a comparative study is to be made with others, in terms of its accuracy, time complexity and high performance and is found to reduce computational cost, by working with large amount of training data which is better than the existing system.KeywordsDetectionClassificationNeural networks (NN)CNNMCNNCOVID-19 virusDWICADImage processingDeep learning (DL)Transfer learningDeep learning feature extractionChest X-ray imagesPrediction

Local binary features for texture classification: Taxonomy and experimental study

Purpose: The aim of the article is to develop an algorithm for classifying cracks in the analyzed images using modern methods of deep machine learning and transfer learning based on pretrained convolutional neural network - Inception-ResNet-v2. Design/Methodology/Approach: Transfer learning based on the pretrained convolutional neural network was used to categorize the images. The fully conected layer of the Inception-ResNet-v2 network has been modified. The last layer was trained using a two-class (binary) linear SVM (Support Vector Machine). In total, 20,000 training cases (images) were used to train the fully connected layer within transfer learning process. The research analyzed the possibility of using the deep neural networks for quick and fully automatic identification of cracks / defects on the surface of analyzed parts. Findings: The results indicate that pretrained convolutional neural network using SVM to train a fully connected layer is a very effective solution for visual crack / fault detection. In the analyzed model, a positive classification was obtained at the level of 99.89%. Practical Implications: The model presented in the article can be used in quality control carried out by process monitoring. An effective model for identifying defective parts can be used in both logistics and production processes. Originality/Value: A novelty is the use of a freely available, deep neural network trained to classify 1000 categories of various images for binary categorization of faults (cracks). The algorithm was adjusted by replacing the primary, 1000-output fully connected layer in the Inception-ResNet-v2 network with a binary layer (2 categories). The fully connected layer has been trained using the classification version of the popular SVM learner, but thanks to the combination of this layer with the sophisticated fearure extraction ability of the pre-trained Inception-ResNet-v2 deep network, the resulting predictive model enables the classification of defects with a very high level of accuracy.

This paper models unsupervised learning of an identity-based pattern (or copying) in speech called reduplication from raw continuous data with deep convolutional neural networks. We use the ciwGAN architecture (Beguš, 2021a) in which learning of meaningful representations in speech emerges from a requirement that the CNNs generate informative data. We propose a technique to wug-test CNNs trained on speech and, based on four generative tests, argue that the network learns to represent an identity-based pattern in its latent space. By manipulating only two categorical variables in the latent space, we can actively turn an unreduplicated form into a reduplicated form with no other substantial changes to the output in the majority of cases. We also argue that the network extends the identity-based pattern to unobserved data. Exploration of how meaningful representations of identity-based patterns emerge in CNNs and how the latent space variables outside of the training range correlate with identity-based patterns in the output has general implications for neural network interpretability.

Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks. While the features learned by these high-capacity neural networks are discriminative for categorization, inaccurate localization is still a major source of error for detection. Building upon high-capacity CNN architectures, we address the localization problem by 1) using a search algorithm based on Bayesian optimization that sequentially proposes candidate regions for an object bounding box, and 2) training the CNN with a structured loss that explicitly penalizes the localization inaccuracy. In experiments, we demonstrated that each of the proposed methods improves the detection performance over the baseline method on PASCAL VOC 2007 and 2012 datasets. Furthermore, two methods are complementary and significantly outperform the previous state-of-the-art when combined.

Deep morphological convolutional network for feature learning of vibration signals and its applications to gearbox fault diagnosis

Handwritten digit recognition is the intelligence of computers to recognize digits written by humans. But it becomes one of the most challenging tasks for machines as handwritten digits are not perfect and can be made with many different: flavors, size, thickness. Thus, as a solution to this problem, Handwriting digit recognition model comes into picture. Many machine learning techniques have been employed to solve the handwritten digit recognition problem. This paper focuses on Neural Network (NN) approaches. Among the three famous NN approaches: deep neural network (DNN), deep belief network (DBN) and convolutional neural network (CNN), the specialization of CNN as compared to other NN of being able to detect pattern is what makes it so useful for recognizing handwritten digits. Humans can very easily see, read &amp; write any handwritten digits, when written in proper format. Even if the digits are not written in proper format we can use our logic and predict what digit it could be. But It is a hard task for the machine to recognize handwritten digits as these are not perfect and can be made with many different flavors. Thus handwritten digit recognition is the solution to this problem which uses the image of a digit and recognizes the digit present in the image. The handwritten digit recognition is the ability of computers to recognize human handwritten digits. The goal is to build a model that can efficiently and reliably recognize the digits and output the proper result. Amongst all the other neural networks, working and implementing a model using Convolution Neural Network gives out the most precise results. It is most popularly used for analyzing images as well as for other data analysis or classification problems. CNN has hidden layers called convolutional layers. These layers work the same way as other layers do but here we need to specify the no of filters each layer should have These filters are actually what detects the pattern. Patterns could be edges, corners, circles or any complex other objects like eyes, ears or even deeper full dogs, cats, etc. Thus, the specialization of CNN as compared to other NN of being able to detect patterns is what makes it so useful for recognizing handwritten digits Key Word: Neural Networks; Convolutional Neural Networks (CNN); Image Processing; Optical Character Recognition (OCR); MNIST Dataset

Deep (machine) learning in recent years has significantly increased the predictive modeling strengths for applications in the areas of computer vision, speech recognition, and natural language processing. Being heavily data-driven, conventional general-purpose deep learning methods mainly rely on training data to learn task-specific data representations. In this thesis, we focus on leveraging novel learning algorithms in deep learning to tackle visual recognition challenges at the three key levels -- namely pixel, object, and image levels. Throughout the thesis, we demonstrate how each and every of our proposed methods can address a specific visual recognition problem. Firstly, we apply deep learning to improve the saliency detection performance of deep networks. A coarse saliency map of the input image's salient object is initially predicted by a convolutional-deconvolutional network (or equivalently a fully convolutional network). Following that, the coarse saliency map is sequentially refined by a separate network called Recurrent Attentional Convolutional-Deconvolutional Network (RACDNN) that shares its network weights across its sequential steps. At every sequential (temporal) step, the same RACDNN is applied to a newly selected/attended image sub-region and it performs saliency map refinement for that particular sub-region. Despite having many learnable network parameters and no access to additional training data, RADCNN can be trained well and it outperforms the "single-step" (coarse saliency detection) baseline and state-of-the-art saliency detection methods by large margins, on several saliency detection benchmark datasets. Secondly, we introduce auto-encoder weight transfer networks (AE-WTN+), a deep model for scaling up object detection. Conventional WTN transfers object class knowledge from the classification weights of a pretrained large-scale image classification network, to the classification weights of an object detection network. However, they suffer from under-fitting and are not able to preserve the rich object class information of novel classes. In this work, we propose a new WTN method -- AE-WTN+ which makes use of normalization techniques to ease network optimization. Inputs and intermediate features are normalized to improve trainability for reduced under-fitting. More importantly, a novel auto-encoding (reconstruction) loss is proposed to encourage information preservation of all classes in the weight predictions of AE-WTN+. This enables the predicted weights of AE-WTN+ to maintain the class discriminability and neighbourhood relationships contained in the pre-trained large-scale classification weights. Experiments on large-scale detection datasets validate the effectiveness of AE-WTN+ on both seen and novel classes. Thirdly, we improve the efficiency of cross-layer network connections for very deep convolutional network in the context of image classification. This gives rise to a new convolutional network architecture called Deluge Networks (DelugeNets) which pass the output (feature activations) of any preceding composite network layer as direct inputs to all succeeding composite layers. For the incoming feature activations of preceding composite layers, the succeeding composite layers first apply rather lightweight cross-layer depthwise convolutional weights on them for feature aggregation. The heavy lifting is carried out by the last convolutional weight layer in each preceding composite layer that performs cross-channel convolutions. The proposed technique enables unimpeded information flow among many layers in very deep DelugeNets with great efficiency. Experiments on small-scale and large-scale datasets show that DelugeNets perform competitively to state-of-the-art deep convolutional networks at lower computational and parameter costs. Fourthly, we propose Stochastic Downsampling (SDPoint), a method to overcome a major limitation suffered by conventional deep convolutional networks -- their computational costs during inference are fixed and cannot be changed. SDPoint downsamples the feature maps at a randomly selected point (layer index) in the network hierarchy during training. The many downsampling configurations known as SDPoint instances each entail a unique downsampling point and downsampling ratio. The same convolutional network weights are shared across the different SDPoint instances that downsample feature maps differently. During inference, the computational cost of a network trained with SDPoint can be instantaneously adjusted to fit a given computational budget, by manually selecting an appropriate SDPoint instance. Additionally, sharing weights between different feature map scales provides significant regularization benefits, making the networks trained with SDPoint less sensitive to scales and more competent. In image classification experiments, SDPoint demonstrates significant cost-accuracy and computational advantages, over independently-trained (non-weight-shared) baselines with various inference costs.