Fully interpretable deep learning model of transcriptional control

Deep neural networks (DNNs) are constructed by considering highly complicated configurations including network structure and several tuning parameters (number of hidden states and learning rate in each layer), which greatly affect the performance of speech processing applications. To reach optimal performance in such systems, deep understanding and expertise in DNNs is necessary, which limits the development of DNN systems to skilled experts. To overcome the problem, this paper proposes an efficient optimization strategy for DNN structure and parameters using evolutionary algorithms. The proposed approach parametrizes the DNN structure by a directed acyclic graph, and the DNN structure is represented by a simple binary vector. Genetic algorithm and covariance matrix adaptation evolution strategy efficiently optimize the performance jointly with respect to the above binary vector and the other tuning parameters. Experiments on phoneme recognition and spoken digit detection tasks show the effectiveness of the proposed approach by discovering the appropriate DNN structure automatically.

Online dynamic stability analysis plays a key role in modern power systems and is becoming even more vital with the penetration of Distributed Energy Resources (DERs). Identification of an accurate lower-order model for the ‘external system” makes it possible to carry out the stability analysis of a study system faster than the analysis for the original detailed system. However, the identified lower-order model becomes invalid when the operating state changes in the power system. ‘Black box’ representation of the ‘external system’ with the application of Deep Neural Networks (DNN) is widely discussed in the literature. In such equivalent models, the DNN response changes based on two aspects i.e. the operating point of the power system and the location/type of the fault. In this paper, a clustering algorithm is proposed to identify the clusters of the ‘external system’ models for different states of the power system, and a DNN is trained for each cluster. The clustering algorithm is based on the pre-fault steady-state power flow from the ‘external system’ to the ‘study system’, which is a reflection of changes in the system state. Mean Absolute Error (MAE) between responses of the actual fault and the predicted fault is used to evaluate the accuracy of the identified system. The proposed technique is easy to implement and requires a lesser number of DNN models compared to identifying an individual DNN model for each operating point to represent the ‘external system’.

Now a days cyber crime growing and has a big effect everywhere globally. ethical hackers are normally involved in identifying flaws and recommending mitigation measures. the cyber safety international, there's a pressing need for the improvement of powerful techniques. Because of the effectiveness of machine learning in cyber security issues, machine learning for cyber security has recently become a hot topic. In cyber security, machine learning approaches have been utilized to handle important concerns such as intrusion detection, malware classification and detection, spam detection, and phishing detection. Although ML cannot fully automate a cyber-security system, it can identify cyber-security threats more efficiently than other software-oriented approaches, relieving security analysts of their burden. As a result, effective adaptive methods, such as machine learning techniques, can yield higher detection rates, lower false alarm rates, and cheaper computing and transmission costs. Our key goal is that the challenge of detecting attacks is fundamentally different from those of these other applications, making it substantially more difficult for the intrusion detection community to apply machine learning effectively. In this study, the CPS is modeled as a network of agents that move in unison with one another, with one agent acting as a leader and commanding the other agents. The proposed strategy in this study is to employ the structure of deep neural networks for the detection phase, which should tell the system of the attack's existence in the early stages of the attack. The use of robust control algorithms in the network to isolate the misbehaving agent in the leader-follower mechanism has been researched. Following the attack detection phase with a deep neural network, the control system uses the reputation algorithm to isolate the misbehaving agent in the presented control method. Experiment results show that deep learning algorithms can detect attacks more effectively than traditional methods, making cyber security simpler, more proactive, and less expensive and more expensive.

Deep neural networks (DNNs) perform excellently in interpreting electrocardiograms (ECGs), both for conventional ECG interpretation and for novel applications such as detection of reduced ejection fraction (EF). Despite these promising developments, implementation is hampered by the lack of trustworthy techniques to explain the algorithms to clinicians. Especially, currently employed heatmap-based methods have shown to be inaccurate. We present a novel pipeline consisting of a variational auto-encoder (VAE) to learn the underlying factors of variation of the median beat ECG morphology (the FactorECG), which are subsequently used in common and interpretable prediction models. As the ECG factors can be made explainable by generating and visualizing ECGs on both the model and individual level, the pipeline provides improved explainability over heatmap-based methods. By training on a database with 1.1 million ECGs, the VAE can compress the ECG into 21 generative ECG factors, most of which are associated with physiologically valid underlying processes. Performance of the explainable pipeline was similar to 'black box' DNNs in conventional ECG interpretation [area under the receiver operating curve (AUROC) 0.94 vs. 0.96], detection of reduced EF (AUROC 0.90 vs. 0.91), and prediction of 1-year mortality (AUROC 0.76 vs. 0.75). Contrary to the 'black box' DNNs, our pipeline provided explainability on which morphological ECG changes were important for prediction. Results were confirmed in a population-based external validation dataset. Future studies on DNNs for ECGs should employ pipelines that are explainable to facilitate clinical implementation by gaining confidence in artificial intelligence and making it possible to identify biased models.

Optimizing power allocation in cellular systems with deep learning enables real-time coordination of inter-cell interference. When channels are time-varying, the deep neural networks (DNNs) need to be re-trained frequently and the training samples need to be re-collected in a timely manner. To achieve higher sum rate with fewer training samples and lower training cost, domain knowledge should be resorted for designing DNNs. In this paper, we propose a DNN structure where the formula of data rate is used to facilitate the learning of power allocation policy, called data-rate based DNN (DRNN). Since such a model-based deep learning method does not exclude the use of prior knowledge for reducing the hypothesis space of a DNN, we further exploit a permutation equivariance prior by introducing parameter sharing into the DNN structure. By integrating the model and prior into DNN, simulations show that either sum rate is improved for given number of training samples or training complexity is reduced to achieve an expected performance.

As a fundamental task in modern data mining, time series classification is powering mission-critical tasks including stock price prediction and network traffic analysis. Due to the non-linear structure of deep neural networks (DNN), deep learning has established as a promising solution to time series classification. However, the excessive learning capacity of DNNs may make them prone to threats of backdoor attacks, where an attacker embeds hidden functionalities (i.e., backdoor) to DNNs and activates the backdoor by specially-designed inputs (i.e., triggers). Despite extensive studies concerning backdoor attacks on image and text domains, there is little known about the vulnerability of DNN based time series classifiers against backdoor attacks. Due to the unique characteristics of time series data, most existing backdoor attack techniques fail to threaten time series classifiers. In this paper, through analyzing the key factors which influence the effectiveness of a backdoor, we systematize a list of practical principles for designing triggers on time series data. In this light, we propose a novel framework called TimeTrojan, which aims to learn to form the trigger pattern through a constrained multi-objective optimization. To solve the hereafter challenging optimization issue, we further design an iterative learning algorithm. Remarkably, the proposed framework is agnostic to a wide range of DNN classifiers. Extensive empirical results on 6 representative DNN classifiers and 6 real-world datasets validate the effectiveness of the proposed attack framework. In most cases, TimeTrojan successfully injects backdoors with 100% attack success rate without affecting the model accuracy on clean samples, which implies the complete control of the behavior of the DNN classifiers by the adversary.

Centralized machine learning algorithms in vehicular networks face privacy and resource constraints.Federated Learning (FL) addresses these by enabling collaborative model training without sharing raw data. To incentivize vehicle participation despite resource constraints, we propose a contract between Roadside Units (RSUs) and vehicles, specifying required computation resources and rewards based on local data quality. The contract is designed to maximize the utility of the RSU, subject to constraints that ensure the participation of the vehicles in the task and truthful reporting of their data quality parameter. However, designing an optimal contract with classical methods is limited to specific utility functions. To extend the design methodology of optimal contracts to a broader range of utility functions, our study proposes a deep learning-based solution. Specifically, we present a novel application of Deep Neural Networks (DNNs), marking the first instance of applying DNNs in contract theory-based incentive mechanisms to achieve near-optimal contracts. We design the DNN's structure to inherently satisfy specific constraints of the optimization problem, enhancing its efficiency and accuracy in determining the optimal contract. We utilize an augmented Lagrangian method to approximate the optimal contract in federated learning tasks. A case study on traffic sign recognition demonstrates the applicability and effectiveness of our method. Comparing our approach with the well-known generative diffusion model shows that in the FL scenarios, the DNN results are closer to the optimal contract compared to those of the generative diffusion model.

Explainable Convolutional Neural Network for Gearbox Fault Diagnosis

With the success of Deep Neural Networks (DNN), many recent works have been focusing on developing hardware accelerator for power and resource-limited system via model compression techniques, such as quantization, pruning, low-rank approximation and etc. However, almost all existing compressed DNNs are fixed after deployment, which lacks run-time adaptive structure to adapt to its dynamic hardware resource allocation, power budget, throughput requirement, as well as dynamic workload. As the countermeasure, to construct a novel run-time dynamic DNN structure, we propose a novel DNN sub-network sampling method via non-uniform channel selection for subnets generation. Thus, user can trade off between power, speed, computing load and accuracy on-the-fly after the deployment, depending on the dynamic requirements or specifications of the given system. We verify the proposed model on both CIFAR-10 and ImageNet dataset using ResNets, which outperforms the same sub-nets trained individually and other related works. It shows that, our method can achieve latency trade-off among 13.4, 24.6, 41.3, 62.1(ms) and 30.5, 38.7, 51, 65.4(ms) for GPU with 128 batch-size and CPU respectively on ImageNet using ResNet18.

The power amplifier (PA) behavior models based on deep neural networks (DNNs) have been widely used. However, it is challenging to balance the network depth, the number of neurons, and the combination of input terms. Accordingly, how to obtain a suitable DNN model becomes a problem. This letter proposes a method to acquire the corresponding optimal DNN structure balancing multiple aspects for a PA by decoupling the relationship of the three factors mentioned above. The algorithm first chooses the optimal input combinations, and then utilizes the inputs to select the DNN structure with excellent capability to fit nonlinear functions and fewer coefficients. Experimental results verify that the DNN model acquired by the algorithm proposed can maintain superior performance with low complexity.

Approximate dynamic programming (ADP) aims to obtain an approximate numerical solution to the discrete-time Hamilton-Jacobi-Bellman (HJB) equation. Heuristic dynamic programming (HDP) is a two-stage iterative scheme of ADP by separating the HJB equation into two equations, one for the value function and another for the policy function, which are referred to as the critic and the actor, respectively. Previous ADP implementations have been limited by the choice of function approximator, which requires significant prior domain knowledge or a large number of parameters to be fitted. However, recent advances in deep learning brought by the computer science community enable the use of deep neural networks (DNN) to approximate high-dimensional nonlinear functions without prior domain knowledge. Motivated by this, we examine the potential of DNNs as function approximators of the critic and the actor. In contrast to the infinite-horizon optimal control problem, the critic and the actor of the finite horizon optimal control (FHOC) problem are time-varying functions and have to satisfy a boundary condition. DNN structure and training algorithm suitable for FHOC are presented. Illustrative examples are provided to demonstrate the validity of the proposed method.

Automated image classification is an essential task of the computer vision field. The tagging of images into a set of predefined groups is referred to as image classification. The implementation of computer vision to automate image classification would be beneficial because manual image evaluation and identification can be time-consuming, particularly when there are many images of different classes. Deep learning approaches are proven to overperform existing machine learning techniques in many fields in recent years, and computer vision is one of the most notable examples. The very deep neural network (VDCNN) is a powerful deep learning model for image classification, and this paper examines it briefly using MNIST handwritten digit dataset. This dataset is used to prove the efficacy of very deep neural networks over other deep learning models. The proposed study aims to comprehend the very deep neural network architecture used to accomplish a handwritten digit recognition task. The feasibility of the proposed model is evaluated using mean accuracy, validation accuracy, and standard deviation. The study results of the very deep neural network model are compared to a convolutional neural network and convolutional neural network with batch normalization. According to the results of the comparison study, very deep neural networks achieve high accuracy of 99.1% for a handwritten dataset. The outcome of the proposed work is used to interpret how well a very deep neural network performs when compared to the other two models of deep neural networks. This proposed architecture may be used to automate the classification of handwritten digits dataset.

With the advancement of machine learning methods, audio sentiment analysis has become an active research area in recent years. For example, business organizations are interested in persuasion tactics from vocal cues and acoustic measures in speech. A typical approach is to find a set of acoustic features from audio data that can indicate or predict a customer's attitude, opinion, or emotion state. For audio signals, acoustic features have been widely used in many machine learning applications, such as music classification, language recognition, emotion recognition, and so on. For emotion recognition, previous work shows that pitch and speech rate features are important features. This thesis work focuses on determining sentiment from call center audio records, each containing a conversation between a sales representative and a customer. The sentiment of an audio record is considered positive if the conversation ended with an appointment being made, and is negative otherwise. In this project, a data processing and machine learning pipeline for this problem has been developed. It consists of three major steps: 1) an audio record is split into segments by speaker turns; 2) acoustic features are extracted from each segment; and 3) classification models are trained on the acoustic features to predict sentiment. Different set of features have been used and different machine learning methods, including classical machine learning algorithms and deep neural networks, have been implemented in the pipeline. In our deep neural network method, the feature vectors of audio segments are stacked in temporal order into a feature matrix, which is fed into deep convolution neural networks as input. Experimental results based on real data shows that acoustic features, such as Mel frequency cepstral coefficients, timbre and Chroma features, are good indicators for sentiment. Temporal information in an audio record can be captured by deep convolutional neural networks for improved prediction accuracy.

Structure estimation of deep neural network for triangulation displacement sensors

Water is a big challenge not only in India but in many countries of the world. Machine learning and forecasting model has been employed towards water demand and ground water level prediction. But in terms of water scarcity, much less work has been carried out by employing machine learning algorithms like 'artificial neural network' (ANN) and 'grey forecasting' model for forecasting water scarcity and none has focused on historical data like water availability, water consumption for a particular area and stress value for predicting water scarcity. So accordingly, we here have developed a water scarcity prediction system based on historical data by employing 'deep neural networks' which is an advanced form of 'artificial neural networks'. We have also compared 'deep neural network' with existing machine learning algorithms such as support vector machine (SVM), logistic regression and Naive Bayes. From the analysis of algorithms based on dataset, deep neural networks have been found as the best prediction model for water scarcity.