Reducing Data Bottlenecks in Distributed, Heterogeneous Neural Networks

This paper presents a hybrid algorithm for the creation of heterogeneous single layer neural networks (SLNNs). The proposed self-adaptive heterogeneous hybrid extreme learning machine (SA-He-HyELM) trains a series of SLNNs with different neuron types in the hidden layer utilizing the extreme learning machine (ELM) algorithm. These networks are evolved into heterogeneous networks (networks having different combinations of hidden neurons) with the help of a modified genetic algorithm (GA). The algorithm is able to handle two architecturally different neuron types: traditional low order (linear) units and higher order units with different transfer functions. The GA is fully self-adaptive and uses one novel hybrid crossover operator along with a self-adaptive mutation operator in order to retain ELM’s simplicity and minimize the number of parameters need tuning. The experimental part of the current paper involves testing SA-He-HyELM with traditional ELM and other three ELM-based methods. The experimental part utilized a series of regression and classification experiments on relatively large datasets. In all cases the proposed method managed to get lower MSE or higher classification accuracy when compared to the aforementioned methods.

Logistics distribution vehicle scheduling plays an important role in the supply chain. With the wide application of e-commerce technology and the increasing diversification of urban industrial and commercial development mode, the optimal scheduling of logistics distribution vehicles can improve the economic benefits of logistics and realize the scientization of logistics. Aiming at the problems existing in the logistics allocation system, this paper proposes a logistics allocation system model based on a heterogeneous neural network, uses the heterogeneous neural network to optimize vehicle scheduling, and gives the specific steps to solve the problem of optimal scheduling of distribution vehicles. The simulation consequences exhibit that the proposed technique cannot solely efficaciously clear up the automobile scheduling optimization model and however additionally has low pc complexity, excessive computational effectivity, and speedy convergence speed. The practicability and effectiveness of the improved algorithm are verified. When the number of distribution customers and distribution cycle is the same, the proposed algorithm effectively reduces the total distribution mileage, reduces the number of vehicles, and improves the efficiency of logistics distribution.

By the development of social media, sentiment analysis has changed to one of the most remarkable research topics in the field of natural language processing which tries to dig information from textual data containing users’ opinions or attitudes toward a particular topic. In this regard, deep neural networks have emerged as promising techniques that have been extensively used for this aim in recent years and obtained significant results. Considering the fact that deep neural networks can automatically extract features from data, it can be claimed that intermediate representations extracted from these networks can be also used as appropriate features. While different deep neural networks are able to extract various types of features due to their distinct structures, we decided to combine features extracted from heterogeneous neural networks using multi-view classifiers to enhance the overall performance of document-level sentiment analysis by considering the correlation between them. The proposed multi-view deep network makes use of intermediate features extracted from convolutional and recursive neural networks to perform classification. Based on the results of the experiments, the proposed multi-view deep network not only outperforms single-view deep neural networks but also has superior efficiency and generalization performance.

This paper proposes heterogeneous modular deep neural network (DNN) to address a complex problem of detection of diabetic retinopathy and simultaneously the five types of abnormalities. The modular approach gives the advantage to extract class specific features for the classifier, which helps to outperform the classical convolutional neural networks. Moreover, the heterogeneous nature of modular DNN provides the economy in the overall architecture and also enables to extract region specific features which further contribute to higher accuracy of detection. Extensive simulation studies were performed using benchmark dataset DIARETDB0 and results show that the proposed approach performs better or equivalently good than the other standard approaches.

This paper describes a hierarchical evolutionary technique developed to design and train feedforward neural networks with different activation functions on their hidden-layer neurons (heterogeneous neural networks). At the upper level, a genetic algorithm is used to determine the number of neurons in the hidden layer and the type of the activation function of those neurons. At the second level, neural nets compete against each other across generations so that the nets with the lowest test errors survive. Finally, on the third level, a co-evolutionary approach is used to train each of the created networks by adjusting both the weights of the hidden-layer neurons and the parameters for their activation functions.

Train Operators can improve railway passengers’ service quality and traffic management by accurately predicting travel arrangements and delays. Precise prediction of train delays is vital for creating feasible scheduled timetables. The import of pruning stacked ensemble deep neural networks into train delay prediction helps improve model prediction accuracy and computational time. In this study, we propose a novel pruning stacked ensemble learning model that uses pruned multilayer perceptron (MLP) neural networks as a meta-learner and heterogeneous neural networks as sub-models to improve the prediction accuracy for passenger train arrival delays and evaluate the model performance using Amtrak, the most extensive US passenger railroad service data, to determine the enhanced model accuracy. We used stacked generalization regression neural networks for artificial neural networks and deep learning neural networks to create the stacked ensemble models. We optimized the models using OPTUNA and produced a pruning stacked ensemble model using the best neural network sub-models and an MLP meta-learner. Our methodology comprises data preprocessing, feature engineering, modeling, case studies, and evaluation phases. Our experiments demonstrate that our proposed pruning stacked ensemble multilayer perceptron Neural Network (PST-NN) model has various degrees of improvement in model evaluation indicators compared to the existing model by 35.20%, 53.40% in terms of accuracy and prediction error and outperforms the benchmark models by 85.22% in terms prediction error in Passenger train delay prediction respectively, which provides a new approach to building models efficiently in Passenger train arrival delay prediction.

Many potential cancer therapies fail during clinical trials, in part due to difficulty in predicting how a particular population of cancer cells will respond to a given drug. Here we develop DrugCell, a “visible” neural network that predicts anti-cancer drug responses by modeling the hierarchical organization of a human cancer cell. We devised a branched visible neural network combining two heterogeneous neural networks. The structure of the first neural network mirrors the hierarchical organization of cellular subsystems (e.g., complexes, pathways and organelles). During training, this neural network learns which cellular subsystems contribute to therapeutic responses based on the mutational profile of the genome. The second neural network is fully connected and models the chemical structure of each drug. These two neural networks connect to a series of small, fully connected layers that simulate a cellular drug treatment conditioned on a genetic mutation landscape. By this design, DrugCell can be used to predict how any cancer cell would respond to any small molecule inhibitor. DrugCell outperformed Elastic Net (rho = 0.80 versus 0.74) while maintaining performance similar to that of a fully connected (black box) neural network (rho = 0.82). When making predictions using DrugCell, genotypes and chemical compound structures induce differential patterns of subsystem activity, enabling in silico investigations of the molecular mechanisms underlying cancer drug response. We therefore applied DrugCell to highlight pathways mediating sensitivity to multiple FDA-approved therapies. Top pathways regulating paclitaxel sensitivity included G2 DNA damage checkpoint, consistent with previous studies showing BRCA1 mediates G2/M arrest in response to paclitaxel, and regulation of TOR signaling, which has previously been described to mediate paclitaxel resistance in multiple cancers. Accordingly, combination of paclitaxel with the mTOR inhibitor, MK8669, was highly synergistic across multiple cell lines. Similarly, pathways mediating sensitivity to topoisomerase II inhibitors included cholinergic receptor signaling and regulation of histone deacetylation. In support of these mechanisms, combination of etoposide, a topoisomerase II inhibitor, with AKT or MEK inhibitors (downstream of the cholinergic receptor) or the HDAC inhibitor vorinostat were highly synergistic across multiple cell lines. Similar results were observed upon CRISPR/Cas9 mediated knockdown of key genes in these pathways. Finally, we used DrugCell to predict effective therapeutic combinations for acute myeloid leukemia patients, leading to identification of drug combinations currently in clinical trials including lenalidomide + DNMT inhibitors and tandutinib + PI3K/MTOR inhibitors. Armed with interpretability and generalizability, DrugCell serves as an important step towards a next generation of intelligent systems in drug discovery and precision medicine. Citation Format: Brent M Kuenzi, Jisoo Park, Samson Fong, Jason Kreisberg, Trey Ideker. DrugCell: A visible neural network to guide precision medicine [abstract]. In: Proceedings of the AACR-NCI-EORTC International Conference on Molecular Targets and Cancer Therapeutics; 2019 Oct 26-30; Boston, MA. Philadelphia (PA): AACR; Mol Cancer Ther 2019;18(12 Suppl):Abstract nr C006. doi:10.1158/1535-7163.TARG-19-C006

ABSTRACTIn recent years, object-based segmentation methods and shallow-model classification algorithms have been widely integrated for remote sensing image supervised classification. However, as the image resolution increases, remote sensing images contain increasingly complex characteristics, leading to higher intraclass heterogeneity and interclass homogeneity and thus posing substantial challenges for the application of segmentation methods and shallow-model classification algorithms. As important methods of deep learning technology, convolutional neural networks (CNNs) can hierarchically extract higher-level spatial features from images, providing CNNs with a more powerful recognition ability for target detection and scene classification in high-resolution remote sensing images. However, the input of the traditional CNN is an image patch, the shape of which is scarcely consistent with a given segment. This inconsistency may lead to errors when directly using CNNs in object-based remote sensing classification: jagged errors may appear along the land cover boundaries, and some land cover areas may overexpand or shrink, leading to many obvious classification errors in the resulting image. To address the above problem, this paper proposes an object-based and heterogeneous segment filter convolutional neural network (OHSF-CNN) for high-resolution remote sensing image classification. Before the CNN processes an image patch, the OHSF-CNN includes a heterogeneous segment filter (HSF) to process the input image. For the segments in the image patch that are obviously different from the segment to be classified, the HSF can differentiate them and reduce their negative influence on the CNN training and decision-making processes. Experimental results show that the OHSF-CNN not only can take full advantage of the recognition capabilities of deep learning methods but also can effectively avoid the jagged errors along land cover boundaries and the expansion/shrinkage of land cover areas originating from traditional CNN structures. Moreover, compared with the traditional methods, the proposed OHSF-CNN can achieve higher classification accuracy. Furthermore, the OHSF-CNN algorithm can serve as a bridge between deep learning technology and object-based segmentation algorithms thereby enabling the application of object-based segmentation methods to more complex high-resolution remote sensing images.

We consider different parallel architectures and methods for self-organization and minimization of complexity for heterogeneous polynomial neural networks (PNN) in problems of pattern recognition and in diagnostics of states. Constructive estimates for the heterogeneity index and parallelism in the process of autonomous classifying decision making with the use of PNNs of different kinds are obtained. It is shown that the parallelism, self-organization, and robustness of heterogeneous PNNs can significantly increase in group (multiagent) solutions of difficult problems in pattern recognition, image analysis, large-scale (vector) diagnostics of states, and adaptive routing of data flows.

We consider different mathematical models, architectures and methods for learning, selforganization and minimization of complexity for heterogeneous polynomial neural networks (PNN) in problems of vector (widened) pattern recognition, data classification and diagnostics of states. Constructive estimates for the heterogeneity index and parallelism in the process of autonomous classifying decision making with the use of PNNs of different kinds are obtained. It is shown that the parallelism, self-organization, and robustness of heterogeneous PNNs can significantly increase in group (multiagent) solutions of difficult problems in pattern recognition, image analysis, large-scale (vector) diagnostics of states, and adaptive routing of data flows.

A cascade and heterogeneous neural network for CT pulmonary nodule detection and its evaluation on both phantom and patient data.

As the number and diversity of distributed information sources on the Internet exponentially increase, various search services are developed to help the users to locate relevant information. But they still exist some drawbacks such as the difficulty of mathematically modeling retrieval process, the lack of adaptivity and the indiscrimination of search. This paper shows how heterogeneous neural networks can be used in the design of an intelligent distributed information retrieval (DIR) system. In particular, three typical neural network models - Kohoren's SOFM Network, Hopfield Network, and Feed Forward Network with Back Propagation algorithm are introduced to overcome the above drawbacks in current research of DIR by using their unique properties. This preliminary investigation suggests that Neural Networks are useful tools for intelligent search for distributed information sources.

This paper presents HEDNet, an emotionally supportive dialogue system based on a heterogeneous neural network. HEDNet integrates CNN, Bi-LSTM, and an attention mechanism to enhance emotion recognition, intent recognition, and response generation, improving emotional and contextual understanding of user input. Evaluated on multiple sentiment dialogue datasets, HEDNet outperformed baseline models in emotion classification, response quality, and user satisfaction. Ablation studies verified the contribution of each module, emphasizing the importance of CNN, Bi-LSTM, and attention mechanisms. This work aims to enhance user experience through personalized, empathetic interactions.

Heterogeneous graph knowledge enhanced stock market prediction

This article is devoted to the process of modeling dynamic systems of city intelligent transport systems (CITS) with the help of information technologies. The proposed means of fundamentally new integrated subsystems CITS, simultaneously covers arrays of heterogeneous data and use modern neural network technologies to provide information support for the management of transport systems. The safety of the ecological state as a result of the influence of urban traffic in the conditions of industrial centers was formalized for information technologies. Descriptions are executed as mathematical models of complex particles of objects of the city intellectual transport network with the use of heterogeneous fragments. Strategies and methods of systematic management are analyzed for solving complex problems of reducing the ecological load of the city. Ontological descriptions in basic model forms that are focused on ensuring the safety and ecology of urban applications to overcome contamination, risks and threats. This direction involves the creation of systems for automated management of transport infrastructure, which at the moment requires the solution of a range of scientific and technical tasks.The built-in neural network models of dependence will effectively solve planning problems with controlling influences on the infrastructure of urban intelligent transport systems, which will improve the characteristics of the traffic flow and reduce the environmental burden on the environment. The analysis of transport infrastructure and the activities of organizations that have a direct influence on it, has allowed us to propose a scheme for the use of heterogeneous information in the information support of the management of environmental safety and the throughput of the street-road network of urban intelligent transport systems. Keywords: information technology, transport system, situational management, risks, models of the situation.