A Robust Prototype-Based Network with Interpretable RBF Classifier Foundations

Robust expansion of networks against cascading failures with reinforcement learning

Deep learning has been a successful model which can effectively represent\nseveral features of input space and remarkably improve image recognition\nperformance on the deep architectures. In our research, an adaptive structural\nlearning method of Restricted Boltzmann Machine (Adaptive RBM) and Deep Belief\nNetwork (Adaptive DBN) have been developed as a deep learning model. The models\nhave a self-organize function which can discover an optimal number of hidden\nneurons for given input data in a RBM by neuron generation-annihilation\nalgorithm, and can obtain an appropriate number of RBM as hidden layers in the\ntrained DBN. The proposed method was applied to a concrete image benchmark data\nset SDNET 2018 for crack detection. The dataset contains about 56,000 crack\nimages for three types of concrete structures: bridge decks, walls, and paved\nroads. The fine-tuning method of the Adaptive DBN can show 99.7%, 99.7%, and\n99.4% classification accuracy for test dataset of three types of structures. In\nthis paper, our developed Adaptive DBN was embedded to a tiny PC with GPU for\nreal-time inference on a drone. For fast inference, the fine tuning algorithm\nalso removed some inactivated hidden neurons to make a small model and then the\nmodel was able to improve not only classification accuracy but also inference\nspeed simultaneously. The inference speed and running time of portable battery\ncharger were evaluated on three kinds of Nvidia embedded systems; Jetson Nano,\nAGX Xavier, and Xavier NX.\n

Single channel source separation (SCSS) is ill-posed and thus challenging. In this paper, we apply general stochastic networks (GSNs) – a deep neural network architecture – to SCSS. We extend GSNs to be capable of predicting a time-frequency representation, i.e. softmask by introducing a hybrid generative-discriminative training objective to the network. We evaluate GSNs on data of the 2nd CHiME speech separation challenge. In particular, we provide results for a speaker dependent, a speaker independent, a matched noise condition and an unmatched noise condition task. Empirically, we compare to other deep architectures, namely a deep belief network (DBN) and a multi-layer perceptron (MLP). In general, deep architectures perform well on SCSS tasks.

Radial basis function neural networks (RBFs) are prime candidates for pattern classification and regression and have been used extensively in classical machine learning applications. However, RBFs have not been integrated into contemporary deep learning research and computer vision using conventional convolutional neural networks (CNNs) due to their lack of adaptability with modern architectures. In this paper, we adapt RBF networks as a classifier on top of CNNs by modifying the training process and introducing a new activation function to train modern vision architectures end-to-end for image classification. The specific architecture of RBFs enables the learning of a similarity distance metric to compare and find similar and dissimilar images. Furthermore, we demonstrate that using an RBF classifier on top of any CNN architecture provides new human-interpretable insights about the decision-making process of the models. Finally, we successfully apply RBFs to a range of CNN architectures and evaluate the results on benchmark computer vision datasets.

Combining neuroimaging data with machine learning has an important potential for the classification of Schizophrenia disease. This work presents a short deep-learning history and introduces one novel solution for fMRI image classification. Our deep convolutional Curvelet Transform Network (CTN) is divided into both levels: The feature learning level is to extract the deep features utilizing the deep curvelet transform depend on multiscale approximation MSA. The latter is utilized to determine the hidden inputs layer. Subsequently, those inputs are purified by utilizing the Real Adaboost technique to choose the best corresponding to each fMRI image. The second level involves building an AutoEncoder (AE) utilizing the best choice of the curvelet of all series of fMRI images. An intelligent pooling is applied after a sequence of stacked AEs for each hidden layer to obtain our deep convolutional CTN architecture of the classification level. Our experimental tests use three different datasets: COBRE, WUSTL, and UCLA. These datasets are given classification rates, using our deep architecture, of 99.9%, 99.2%, and 98.6%, respectively. Compared to those cited approaches, our results indicate that our architecture can achieve better performance for all datasets.

Domain-invariant representation learning using an unsupervised domain adversarial adaptation deep neural network

Recent works demonstrated the usefulness of temporal coherence to regularize supervised training or to learn invariant features with deep architectures. In particular, enforcing a smooth output change while presenting temporally-closed frames from video sequences, proved to be an effective strategy. In this paper we prove the efficacy of temporal coherence for semi-supervised incremental tuning. We show that a deep architecture, just mildly trained in a supervised manner, can progressively improve its classification accuracy, if exposed to video sequences of unlabeled data. The extent to which, in some cases, a semi-supervised tuning allows to improve classification accuracy (approaching the supervised one) is somewhat surprising. A number of control experiments pointed out the fundamental role of temporal coherence.

Human preferences play a key role in numerous decision-making processes. The ability to correctly identify likes and dislikes would facilitate novel applications in neuromarketing, affective entertainment, virtual rehabilitation and forensic neuroscience that leverage on sub-conscious human preferences. In this neuroinformatics investigation, we seek to recognize human preferences passively through the use of electroencephalography (EEG) when a subject is presented with some 3D visual stimuli. Our approach employs the use of machine learning in the form of deep neural networks to classify brain signals acquired using a brain-computer interface (BCI). Our previous work has shown that EEG preference classification is possible although accuracy rates remain relatively low at 61%–67% using conventional deep learning neural architectures, where the challenge mainly lies in the accurate classification of unseen data from a cohort-wide sample that introduces inter-subject variability on top of the existing intra-subject variability. Such an approach is significantly more challenging and is known as subject-independent EEG classification as opposed to the more commonly adopted but more time-consuming and less general approach of subject-dependent EEG classification. In this new study, we employ deep networks that allow dropouts to occur in the architecture of the neural network. The results obtained through this simple feature modification achieved a classification accuracy of up to 79%. Therefore, this study has shown that the use of a deep learning classifier was able to achieve an increase in emotion classification accuracy of between 13%–18% through the simple adoption of the use of dropouts compared to a conventional deep learner for EEG preference classification.

Automatic modulation classification receives significant interest in the context of current and future wireless communication systems. Deep learning emerged as a powerful tool for modulation classification, as it allows for joint discriminative features learning and signal classification. However, the optimization of deep neural network architectures for modulation classification is a manual and time-consuming process that requires profound domain knowledge and much effort. Most state-of-the-art solutions focus mainly on classification accuracy, while optimization of network complexity is neglected. This paper presents a novel bi-objective memetic algorithm, BO-NSMA, to search optimal deep neural network architectures for modulation classification to maximize classification accuracy and minimize network complexity. The experiments show that BO-NSMA, with an initial population of six individuals and only ten generations, finds a deep neural network architecture that outperforms all human-crafted architectures. Furthermore, BO-NSMA discovered the first low-complexity Convolutional neural network architecture, which achieves slightly better performance than costly Recurrent neural network architectures, allowing a 2.9-fold reduction in network complexity with 1.43% performance improvement. Compared to counterparts from network architecture search, BO-NSMA finds the best architecture, which achieves up to 18.73% accuracy gain and up to an 82-fold reduction in network complexity. The results are validated using the Wilcoxon signed-rank test.

High accuracy of text classification can be achieved through simultaneous learning of multiple information, such as sequence information and word importance. In this article, a kind of flat neural networks called the broad learning system (BLS) is employed to derive two novel learning methods for text classification, including recurrent BLS (R-BLS) and long short-term memory (LSTM)-like architecture: gated BLS (G-BLS). The proposed two methods possess three advantages: 1) higher accuracy due to the simultaneous learning of multiple information, even compared to deep LSTM that extracts deeper but single information only; 2) significantly faster training time due to the noniterative learning in BLS, compared to LSTM; and 3) easy integration with other discriminant information for further improvement. The proposed methods have been evaluated over 13 real-world datasets from various types of text classification. From the experimental results, the proposed methods achieve higher accuracies than LSTM while taking significantly less training time on most evaluated datasets, especially when the LSTM is in deep architecture. Compared to R-BLS, G-BLS has an extra forget gate to control the flow of information (similar to LSTM) to further improve the accuracy on text classification so that G-BLS is more effective while R-BLS is more efficient.

Well-testing stage analysis is an important way for oilfield operation state decision-making and reservoir management. However, due to the variability and nonlinearity of the downhole data caused by the complex exploration activities and the differences of petroleum type and geological conditions, the classical methods are ineffective in feature extraction, learning network construction, and classifier optimization. In this work, we propose a new well-testing stage classification method based on a deep vector learning model (DVLM). The novelty of this study lies in the combination of multi-feature extraction, deep learning and feature vector mapping. The proposed method can overcome the problems of poor feature representation ability and poor classification model generalization ability in the existing machine learning methods, which mainly caused by the non-optimized training network structure and the unreasonable classifier design,. Firstly, the initial features are obtained by four classical methods. Then a five layers deep belief network embedded with the mutual information coefficient method is implemented for further feature extraction and purification. Finally, the optimized learning vector quantization classifier outputs the predicted tags. For model training and testing 572 field samples total of 4004 data streams are used. By considering the classification errors and accuracy metrics, the neurons number of deep learning network and the classifier are tuned, and an optimal and stable framework is obtained. Comparative experimental results with several classical integration models show that the proposed model achieves the highest classification accuracy of 98.065% as well as the least of features (nine). The results demonstrate that the proposed model has excellent performance in improving the classification accuracy and completing the feature compression. Moreover, the proposed model has very important practical significance for guiding the automatic analysis and processing oil and gas data.

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

The brain tumor is the most serious cancer among people of all ages, and recognition of its grade is a complex task for monitoring health. In addition, the earlier detection and classification of tumors into a particular grade are imperative for diagnosing the tumor effectively. This paper devises a novel method for multigrade tumor classification using deep architecture. First, the pre-processing is performed with the Region of interest (ROI) and Type 2 Fuzzy and Cuckoo Search (T2FCS) filter. After that, segmentation using a pre-processed image is carried out to generate segments, which is performed using a deep fuzzy clustering model. Then, the significant features are mined through segments that involve convolution neural network (CNN) features, Texton features, EMD features, and statistical features such as mean, variance, kurtosis, and entropy. The obtained features are subjected to Deep Residual Network for multigrade tumor classification. The Deep Residual Network training is done with the proposed Harmony search-based Feedback Artificial Tree (HSFAT) algorithm. The proposed HSFAT is devised by combining Harmony search and Feedback Artificial Tree (FAT) algorithm. The proposed HSFAT-based deep residual network provided superior performance with maximum accuracy of 94.33%, maximum sensitivity of 97.27%, and maximum specificity of 92.61%.

Lensless digital inline holographic microscopy (LDIHM) is an emerging quantitative phase imaging modality that uses advanced computational methods for phase retrieval from the interference pattern. The existing end-to-end deep networks require a large training dataset with sufficient diversity to achieve high-fidelity hologram reconstruction. To mitigate this data requirement problem, physics-aware deep networks integrate the physics of holography in the loss function to reconstruct complex objects without needing prior training. However, the data fidelity term measures the data consistency with a single low-resolution hologram without any external regularization, which results in a low performance on complex biological data. We aim to mitigate the challenges with trained and physics-aware untrained deep networks separately and combine the benefits of both methods for high-resolution phase recovery from a single low-resolution hologram in LDIHM. We propose a hybrid deep framework (HDPhysNet) using a plug-and-play method that blends the benefits of trained and untrained deep models for phase recovery in LDIHM. The high-resolution phase is generated by a pre-trained high-definition generative adversarial network (HDGAN) from a single low-resolution hologram. The generated phase is then plugged into the loss function of a physics-aware untrained deep network to regulate the complex object reconstruction process. Simulation results show that the SSIM of the proposed method is increased by 0.07 over the trained and 0.04 over the untrained deep networks. The average phase-SNR is elevated by 8.2dB over trained deep models and 9.8dB over untrained deep networks on the experimental biological cells (cervical cells and red blood cells). We showed improved performance of the HDPhysNet against the unknown perturbation in the imaging parameters such as the propagation distance, the wavelength of the illuminating source, and the imaging sample compared with the trained network (HDGAN). LDIHM, combined with HDPhysNet, is a portable and technology-driven microscopy best suited for point-of-care cytology applications.

Recently, pedestrian detection has become an important problem of interest. Our work primarily depends on robust and fast deep neural network architectures. This paper used very efficient and recent methods for pedestrian detection. Recently, pedestrian detection has become an important problem of interest. This paper suggests robust convolutional neural network models to solve this problem. We primarily evaluate accuracy and speed. Our work primarily depends on robust and fast deep neural network architectures; substantial changes to those models achieve results that are competitive with prior state-of-the-art methods. As a result, we outperformed all the prior state-of-the-art pedestrian detection methods. We also overtook other models that use extra information during testing and training. All experiments used three pedestrian detection challenge benchmarks: Caltech-USA, INRIA, and ETH.