Structure discovery of deep neural network based on evolutionary algorithms

Surrogate-assisted evolutionary algorithms provide an effective means for complex and computationally expensive optimization problems. However, due to the scarcity of training samples, the prediction accuracy of frequently-used regression surrogate models can hardly be guaranteed as the difficulty of the problem increases, resulting in performance degradation of the whole algorithm. Since real-world problems rarely exist in isolation, it is expected to alleviate the above issue by properly exploiting the knowledge shared across different problems. In this context, this study proposes a novel evolutionary multitasking optimization algorithm assisted by a classifier rather than a regression model for expensive multitasking problems, where the accuracy of the classifier is boosted by knowledge transfer. To be specific, a support vector classifier (SVC) is first developed and integrated into a classic evolutionary algorithm, i.e., covariance matrix adaptation evolution strategy (CMA-ES). With a low computational cost, it helps CMA-ES to prescreen parent solutions from the current population. Following that, a knowledge transfer strategy is designed to enrich the training samples for each task-oriented classifier by sharing high-quality solutions among different tasks, where a PCA-based subspace alignment technique is employed. Extensive experiments indicate that the SVC-assisted CMA-ES gains significant superiority over general CMA-ES in terms of both robustness and scalability, and the knowledge transfer strategy further helps it earn a competitive edge over some state-of-the-art algorithms on expensive multitasking optimization problems.

Batteries play a vital role in current scenario of energy storage, even though many techniques of energy storage are available, since the time taken to start delivering the stored energy is very less. The battery life time depends upon its charging and discharging characteristics, which are in turn, depend on the internal parameters of battery. These parameters include resistance, capacitance and open circuit voltage. The amount of energy stored in the battery can be calculated by estimating these parameters. In this paper, an optimized model for Lithium ion batteries is presented using evolutionary algorithms to estimate the internal parameters of the battery over different charging and discharging rates. A sample EIG make, 2.5 V, 8 Ahr Lithium ion battery is modeled using two evolutionary algorithms such as genetic algorithm and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for different charging and discharging rates. The results of two algorithms are compared with the catalog values given by the manufacturer in order to identify the appropriate algorithm for battery modeling and validation. This paper concludes that battery characteristics obtained by CMA-ES algorithm match with the measured manufacturer characteristics.

Optimizing power allocation in cellular systems with deep learning enables real-time coordination of inter-cell interference. When channels are time-varying, the deep neural networks (DNNs) need to be re-trained frequently and the training samples need to be re-collected in a timely manner. To achieve higher sum rate with fewer training samples and lower training cost, domain knowledge should be resorted for designing DNNs. In this paper, we propose a DNN structure where the formula of data rate is used to facilitate the learning of power allocation policy, called data-rate based DNN (DRNN). Since such a model-based deep learning method does not exclude the use of prior knowledge for reducing the hypothesis space of a DNN, we further exploit a permutation equivariance prior by introducing parameter sharing into the DNN structure. By integrating the model and prior into DNN, simulations show that either sum rate is improved for given number of training samples or training complexity is reduced to achieve an expected performance.

As a fundamental task in modern data mining, time series classification is powering mission-critical tasks including stock price prediction and network traffic analysis. Due to the non-linear structure of deep neural networks (DNN), deep learning has established as a promising solution to time series classification. However, the excessive learning capacity of DNNs may make them prone to threats of backdoor attacks, where an attacker embeds hidden functionalities (i.e., backdoor) to DNNs and activates the backdoor by specially-designed inputs (i.e., triggers). Despite extensive studies concerning backdoor attacks on image and text domains, there is little known about the vulnerability of DNN based time series classifiers against backdoor attacks. Due to the unique characteristics of time series data, most existing backdoor attack techniques fail to threaten time series classifiers. In this paper, through analyzing the key factors which influence the effectiveness of a backdoor, we systematize a list of practical principles for designing triggers on time series data. In this light, we propose a novel framework called TimeTrojan, which aims to learn to form the trigger pattern through a constrained multi-objective optimization. To solve the hereafter challenging optimization issue, we further design an iterative learning algorithm. Remarkably, the proposed framework is agnostic to a wide range of DNN classifiers. Extensive empirical results on 6 representative DNN classifiers and 6 real-world datasets validate the effectiveness of the proposed attack framework. In most cases, TimeTrojan successfully injects backdoors with 100% attack success rate without affecting the model accuracy on clean samples, which implies the complete control of the behavior of the DNN classifiers by the adversary.

MotivationThe universal expressibility assumption of Deep Neural Networks (DNNs) is the key motivation behind recent worksin the systems biology community to employDNNs to solve important problems in functional genomics and moleculargenetics. Typically, such investigations have taken a ‘black box’ approach in which the internal structure of themodel used is set purely by machine learning considerations with little consideration of representing the internalstructure of the biological system by the mathematical structure of the DNN. DNNs have not yet been applied to thedetailed modeling of transcriptional control in which mRNA production is controlled by the binding of specific transcriptionfactors to DNA, in part because such models are in part formulated in terms of specific chemical equationsthat appear different in form from those used in neural networks.ResultsIn this paper, we give an example of a DNN whichcan model the detailed control of transcription in a precise and predictive manner. Its internal structure is fully interpretableand is faithful to underlying chemistry of transcription factor binding to DNA. We derive our DNN from asystems biology model that was not previously recognized as having a DNN structure. Although we apply our DNNto data from the early embryo of the fruit fly Drosophila, this system serves as a test bed for analysis of much larger datasets obtained by systems biology studies on a genomic scale. .Availability and implementationThe implementation and data for the models used in this paper are in a zip file in the supplementary material.Supplementary informationSupplementary data are available at Bioinformatics online.

With the success of Deep Neural Networks (DNN), many recent works have been focusing on developing hardware accelerator for power and resource-limited system via model compression techniques, such as quantization, pruning, low-rank approximation and etc. However, almost all existing compressed DNNs are fixed after deployment, which lacks run-time adaptive structure to adapt to its dynamic hardware resource allocation, power budget, throughput requirement, as well as dynamic workload. As the countermeasure, to construct a novel run-time dynamic DNN structure, we propose a novel DNN sub-network sampling method via non-uniform channel selection for subnets generation. Thus, user can trade off between power, speed, computing load and accuracy on-the-fly after the deployment, depending on the dynamic requirements or specifications of the given system. We verify the proposed model on both CIFAR-10 and ImageNet dataset using ResNets, which outperforms the same sub-nets trained individually and other related works. It shows that, our method can achieve latency trade-off among 13.4, 24.6, 41.3, 62.1(ms) and 30.5, 38.7, 51, 65.4(ms) for GPU with 128 batch-size and CPU respectively on ImageNet using ResNet18.

The power amplifier (PA) behavior models based on deep neural networks (DNNs) have been widely used. However, it is challenging to balance the network depth, the number of neurons, and the combination of input terms. Accordingly, how to obtain a suitable DNN model becomes a problem. This letter proposes a method to acquire the corresponding optimal DNN structure balancing multiple aspects for a PA by decoupling the relationship of the three factors mentioned above. The algorithm first chooses the optimal input combinations, and then utilizes the inputs to select the DNN structure with excellent capability to fit nonlinear functions and fewer coefficients. Experimental results verify that the DNN model acquired by the algorithm proposed can maintain superior performance with low complexity.

Approximate dynamic programming (ADP) aims to obtain an approximate numerical solution to the discrete-time Hamilton-Jacobi-Bellman (HJB) equation. Heuristic dynamic programming (HDP) is a two-stage iterative scheme of ADP by separating the HJB equation into two equations, one for the value function and another for the policy function, which are referred to as the critic and the actor, respectively. Previous ADP implementations have been limited by the choice of function approximator, which requires significant prior domain knowledge or a large number of parameters to be fitted. However, recent advances in deep learning brought by the computer science community enable the use of deep neural networks (DNN) to approximate high-dimensional nonlinear functions without prior domain knowledge. Motivated by this, we examine the potential of DNNs as function approximators of the critic and the actor. In contrast to the infinite-horizon optimal control problem, the critic and the actor of the finite horizon optimal control (FHOC) problem are time-varying functions and have to satisfy a boundary condition. DNN structure and training algorithm suitable for FHOC are presented. Illustrative examples are provided to demonstrate the validity of the proposed method.

Structure estimation of deep neural network for triangulation displacement sensors

Centralized machine learning algorithms in vehicular networks face privacy and resource constraints.Federated Learning (FL) addresses these by enabling collaborative model training without sharing raw data. To incentivize vehicle participation despite resource constraints, we propose a contract between Roadside Units (RSUs) and vehicles, specifying required computation resources and rewards based on local data quality. The contract is designed to maximize the utility of the RSU, subject to constraints that ensure the participation of the vehicles in the task and truthful reporting of their data quality parameter. However, designing an optimal contract with classical methods is limited to specific utility functions. To extend the design methodology of optimal contracts to a broader range of utility functions, our study proposes a deep learning-based solution. Specifically, we present a novel application of Deep Neural Networks (DNNs), marking the first instance of applying DNNs in contract theory-based incentive mechanisms to achieve near-optimal contracts. We design the DNN's structure to inherently satisfy specific constraints of the optimization problem, enhancing its efficiency and accuracy in determining the optimal contract. We utilize an augmented Lagrangian method to approximate the optimal contract in federated learning tasks. A case study on traffic sign recognition demonstrates the applicability and effectiveness of our method. Comparing our approach with the well-known generative diffusion model shows that in the FL scenarios, the DNN results are closer to the optimal contract compared to those of the generative diffusion model.

With the emergence of deep neural networks, many research fields, such as image classification, object detection, speech recognition, natural language processing, machine translation and automatic driving, have made major breakthroughs in technology and the research achievements have been successfully applied in many real-life applications. Combining evolutionary computation and neural architecture search (NAS) is an important approach to improve the performance of deep neural networks. Usually, the related researchers only focus on precision. Thus, the searched neural architectures always perform poorly in the other indexes such as time cost. In this paper, a multi-objective evolutionary algorithm with a probability stack (MOEA-PS) is proposed for NAS, which considers the two objects of precision and time consumption. MOEA-PS uses an adjacency list to represent the internal structure of deep neural networks. Besides, a unique mechanism is introduced into the multi-objective genetic algorithm to guide the process of crossover and mutation when generating offspring. Furthermore, the structure blocks are stacked using a proxy model to generate deep neural networks. The results of the experiments on Cifar-10 and Cifar-100 demonstrate that the proposed algorithm has a similar error rate compared with the most advanced NAS algorithms, but the time cost is lower. Finally, the network structure searched on Cifar-10 is transferred directly to the ImageNet dataset, which can achieve 73.6% classification accuracy.

Now a days cyber crime growing and has a big effect everywhere globally. ethical hackers are normally involved in identifying flaws and recommending mitigation measures. the cyber safety international, there's a pressing need for the improvement of powerful techniques. Because of the effectiveness of machine learning in cyber security issues, machine learning for cyber security has recently become a hot topic. In cyber security, machine learning approaches have been utilized to handle important concerns such as intrusion detection, malware classification and detection, spam detection, and phishing detection. Although ML cannot fully automate a cyber-security system, it can identify cyber-security threats more efficiently than other software-oriented approaches, relieving security analysts of their burden. As a result, effective adaptive methods, such as machine learning techniques, can yield higher detection rates, lower false alarm rates, and cheaper computing and transmission costs. Our key goal is that the challenge of detecting attacks is fundamentally different from those of these other applications, making it substantially more difficult for the intrusion detection community to apply machine learning effectively. In this study, the CPS is modeled as a network of agents that move in unison with one another, with one agent acting as a leader and commanding the other agents. The proposed strategy in this study is to employ the structure of deep neural networks for the detection phase, which should tell the system of the attack's existence in the early stages of the attack. The use of robust control algorithms in the network to isolate the misbehaving agent in the leader-follower mechanism has been researched. Following the attack detection phase with a deep neural network, the control system uses the reputation algorithm to isolate the misbehaving agent in the presented control method. Experiment results show that deep learning algorithms can detect attacks more effectively than traditional methods, making cyber security simpler, more proactive, and less expensive and more expensive.

Deep learning has become a popularly used tool in large amount of applications. Given its ability to explore the input and output relationship, deep learning can perform well in terms of prediction. However, one important drawback in this framework is that the model cannot be easily trained due to huge search space and large number of parameters. In response to such problem, this paper proposes a novel way to build a deep neural network. Specifically, it tries to consider an unbalanced structure of deep neural network by expanding the number of nodes in the beginning to extract as much information as possible and then shrinking quickly to converge to the final result. The experiment results show that our proposed structure can output equally good output with much faster time compared with traditional methods.

ReaxFF is a computationally efficient force field to simulate complex reactive dynamics in extended molecular models with diverse chemistries, if reliable force-field parameters are available for the chemistry of interest. If not, they must be optimized by minimizing the error ReaxFF makes on a relevant training set. Because this optimization is far from trivial, many methods, in particular genetic algorithms (GAs), have been developed to search for the global optimum in parameter space. Recently, two alternative parameter calibration techniques were proposed, i.e. Monte-Carlo Force Field optimizer (MCFF) and Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). In this work, CMA-ES, MCFF and a GA method (OGOLEM) are systematically compared using three training sets from the literature. GA shows the smallest risk of getting trapped into a local minimum, whereas CMA-ES is capable of reaching the lowest errors for two third of the cases. For each method, we provide reasonable default settings and our analysis offers useful guidelines for their usage in future work. An important side effect impairing the parameter optimization is numerical noise. A detailed analysis reveals that it can be reduced, e.g. by using exclusively unambiguous geometry optimizations in the training set. Even without this noise, many distinct near-optimal parameter vectors can be found, which opens new avenues for improving the training set and detecting overfitting artifacts.

Liver fibrosis in chronic hepatitis B is the pathological repair response of the liver to chronic injury, which is a key step in the development of various chronic liver diseases to cirrhosis and an important link affecting the prognosis of chronic liver diseases. The further development of liver fibrosis in chronic hepatitis B can lead to the disorder of hepatic lobule structure, nodular regeneration of hepatocytes, formation of a pseudolobular structure, namely, cirrhosis, clinical manifestations of liver dysfunction, and portal hypertension. So far, the diagnosis of liver fibrosis in chronic hepatitis B has been made manually by doctors. However, this is very subjective and boring for doctors. Doctors are likely to be interfered with by external factors, such as fatigue and lack of sleep. This paper proposed a 5-layer deep convolution neural network structure for the automatic classification of liver fibrosis in chronic hepatitis B. In the 5-layer deep convolution neural network structure, there were three convolution layers and two fully connected layers, and each convolution layer was connected with a pooling layer. 123 ADC images were collected, and the following results were obtained: the accuracy, sensitivity, specificity, precision, F1, MCC, and FMI were 88.13% ± 1.47%, 81.45% ± 3.69%, 91.12% ± 1.72%, 80.49% ± 2.94%, 80.90% ± 2.39%, 72.36% ± 3.39%, and 80.94% ± 2.37%, respectively.