Artificial intelligence: Deep neural reasoning.

Differentiable neural computer (DNC) has demonstrated remarkable capabilities in solving complex problems. In this paper, we propose to stack an enhanced version of differentiable neural computer together to extend its learning capabilities. Firstly, we give an intuitive interpretation of DNC to explain the architectural essence and demonstrate the stacking feasibility by contrasting it with the conventional recurrent neural network. Secondly, the architecture of stacked DNCs is proposed and modified for electroencephalogram (EEG) data analysis. We substitute the original Long Short-Term Memory network controller by a recurrent convolutional network controller and adjust the memory accessing structures for processing EEG topographic data. Thirdly, the practicability of our proposed model is verified by an open-sourced EEG dataset with the highest average accuracy achieved; then after fine-tuning the parameters, we show the minimal mean error obtained on a proprietary EEG dataset. Finally, by analyzing the behavioral characteristics of the trained stacked DNCs model, we highlight the suitableness and potential of utilizing stacked DNCs in EEG signal processing.

Employing Differentiable Neural Computers for Image Captioning and Neural Machine Translation

In recent years, deep learning for de novo molecular generation has become a rapidly growing research area. Recurrent neural networks (RNN) using the SMILES molecular representation is one of the most common approaches used. Recent study shows that the differentiable neural computer (DNC) can make considerable improvement over the RNN for modeling of sequential data. In the current study, DNC has been implemented as an extension to REINVENT, an RNN-based model that has already been used successfully to make de novo molecular design. The model was benchmarked on its capacity to learn the SMILES language on the GDB-13 and MOSES datasets. The DNC shows improvement on all test cases conducted at the cost of significantly increased computational time and memory consumption.

Neural network, a nonalgorithmic, nondigital, intensely parallel and distributive information processing system, is being used more and more everyday. The main interest in neural networks is rooted in the recognition that the human brain processes information in a different manner than conventional digital computers. Computers are extremely fast and precise at executing sequences of instructions that have been formulated for them. A human information processing system is composed of neurons switching at speeds about a million times slower than computer gates. Yet, humans are more efficient than computers at such computationally complex tasks as speech and other pattern-recognition problems. Artificial neural systems, or neural networks, are physical cellular systems that can acquire, store, and use experiential knowledge. The knowledge is in the form of stable states or mapping embedded in networks that can be recalled in response to the presentation of cues. In a typical neural data processing procedure, the database is divided into two separate portions called training and test sets. The training set is used to develop the desired network. In this process (depending on the paradigm that is being used), the desired output in the training set is used to help the network learn by adjusting the weights between its neurons or processing elements. Once the network has learned the information in the training set and has"converged," the test set is applied to the network for verification. It is important to note that although the user has the desired output of the test set, it has not been seen by the network. This ensures the integrity and robustness of the trained network. A handful of articles on the use of neural networks in the petroleum industry has appeared in SPE conferences, proceedings, and publications in the past 2 years. These articles can be divided into two categories: those that use neural networks to analyze formation lithology from well logs and those that use neural networks to pick a reservoir model to be used in conventional welltest interpretation studies. These tasks are usually done by log analysts and reservoir engineers, and their automation using a fault-tolerant process may prove valuable. Neural networks can help engineers and researchers by addressing some fundamental petroleum engineering problems as well as specific ones that conventional computing has been unable to solve. Petroleum engineers may benefit from neural networks on occasions when engineering data for design and interpretations are less than adequate. This is an especially common occurrence in the Appalachian basin, where some fields are quite old. Lack of adequate engineering data may also be encountered because of the high cost of coring, well testing, and so on. Neural networks have shown great potential for generating accurate analysis and results from large amounts of historical data that otherwise would seem not to be useful or relevant in the analysis. An example of such a problem was encountered by a gas company for a gas storage field in Ohio. In the absence of appropriate data, which normally would make engineering design and evaluation of the fracturing jobs virtually impossible, a carefully designed neural network was able to predict the performance of fracturing jobs with great accuracy. A linear plot of the actual fracturing job results (data never seen by the network during training) and network predictions resulted in a correlation coefficient of 0.98, where 1.00 is a perfect match. Neural networks have proved to be valuable pattern-recognition tools. They are capable of finding highly complex patterns within large amounts of data. A relevant example is well log interpretation. It is generally accepted that there is more information embedded in well logs than meets the eye. Determination, prediction, or estimation of formation permeability without actual laboratory measurement of the cores or interruption in production for well test data collection has been a fundamental problem for petroleum engineers. From geophysical well log data, it was possible to predict and/or estimate permeability of a highly heterogeneous formation in West Virginia, as shown in Fig 1. Reference Mohaghegh, S., Arefi, R., and Ameri, S.: "Design and Development of an Artificial Neural Network for Prediction of Formation Permeability,"paper SPE 28237 presented at the 1994 SPE Petroleum Computer Conference, July31-Aug. 3, Dallas.

Memory-augmented neural networks (MANNs) provide better inference performance in many tasks with the help of an external memory. The recently developed differentiable neural computer (DNC) is a MANN that has been shown to outperform in representing complicated data structures and learning long-term dependencies. DNC's higher performance is derived from new history-based attention mechanisms in addition to the previously used content-based attention mechanisms. History-based mechanisms require a variety of new compute primitives and state memories, which are not supported by existing neural network (NN) or MANN accelerators. We present HiMA, a tiled, history-based memory access engine with distributed memories in tiles. HiMA incorporates a multi-mode network-on-chip (NoC) to reduce the communication latency and improve scalability. An optimal submatrix-wise memory partition strategy is applied to reduce the amount of NoC traffic; and a two-stage usage sort method leverages distributed tiles to improve computation speed. To make HiMA fundamentally scalable, we create a distributed version of DNC called DNC-D to allow almost all memory operations to be applied to local memories with trainable weighted summation to produce the global memory output. Two approximation techniques, usage skimming and softmax approximation, are proposed to further enhance hardware efficiency. HiMA prototypes are created in RTL and synthesized in a 40nm technology. By simulations, HiMA running DNC and DNC-D demonstrates 6.47 × and 39.1 × higher speed, 22.8 × and 164.3 × better area efficiency, and 6.1 × and 61.2 × better energy efficiency over the state-of-the-art MANN accelerator. Compared to an Nvidia 3080Ti GPU, HiMA demonstrates speedup by up to 437 × and 2,646 × when running DNC and DNC-D, respectively.

In past ten years, modern societies developed enormous communication and\nsocial networks. Their classification and information retrieval processing\nbecome a formidable task for the society. Due to the rapid growth of World Wide\nWeb, social and communication networks, new mathematical methods have been\ninvented to characterize the properties of these networks on a more detailed\nand precise level. Various search engines are essentially using such methods.\nIt is highly important to develop new tools to classify and rank enormous\namount of network information in a way adapted to internal network structures\nand characteristics. This review describes the Google matrix analysis of\ndirected complex networks demonstrating its efficiency on various examples\nincluding World Wide Web, Wikipedia, software architecture, world trade, social\nand citation networks, brain neural networks, DNA sequences and Ulam networks.\nThe analytical and numerical matrix methods used in this analysis originate\nfrom the fields of Markov chains, quantum chaos and Random Matrix theory.\n

Throughout history, the development of artificial intelligence, especially artificial neural networks, has been continuously influenced by a deeper understanding of the brain. This influence includes the development of the neocognitron, considered a precursor to convolutional neural networks. The emerging field of NeuroAI posits that leveraging neuroscience knowledge could significantly advance AI by imbuing networks with enhanced capabilities. Unlike the human brain, which features a variety of morphologically and functionally distinct neurons, artificial neural networks typically rely on a homogeneous neuron model. In the human brain, the diversity of neurons facilitates a wide range of intelligent behaviors. Given that artificial networks aim to mimic the human brain, incorporating a diversity of neuron models could address key challenges in artificial intelligence, such as efficiency, interpretability, and memory capacity. This perspective begins by examining the basics of biological neuronal diversity and how biological neurons transmit and process information. We then explore research efforts to design novel neuron models for artificial networks and discuss the potential benefits of neuronal diversity, including applications in several critical areas. Finally, we address the challenges and future directions for integrating neuronal diversity into artificial networks, highlighting its potential to enrich NeuroAI.

Analyzing factors of affecting Web Service invocation performance is a hot topic. Among the factors, service interface complexity is a key one investigated by much research work. However, these researches mainly analyze the impact on the performance of primitives, some simple data structures like mesh interface object, or array of them. For the complex data structures, these works lack of a systematic approach to characterize the impact. This paper firstly makes a detailed statistics of service interface complexity based on a large sample space (10K+ WSDL files) on the World Wide Web, and we find about 41.6% services contain complex data structures. The statistic results guide us to conduct many experiments for finding out the correlation of service interface complexity to invocation performance. We discover an interesting feature on commonly used Web Service platforms (Axis/gSOAP/.Net), called Data Structural Form Unaware. As each parameter or return value can be represented as a tree with two kinds of nodes, structure type node and primitive type node, then the feature means that the overhead caused by parameters or return values is independent of the organization of nodes in the trees, but it is only related with the number and the type of nodes. By this feature, we present a simple model to quantify the impact of service interface complexity. Using our model, a Service Interface Performance Vector can be calculated by parsing a WSDL file to estimate the overhead of service interface design. This vector, together with the invocation probability of each operation, generates the Service Interface Performance Score, a comprehensive complexity indicator which can be used to evaluate and rank the service interfaces. At last, a rank report of services on the WWW is shown. Our work can play a guiding role in service interface designing, service interface performance predicting, and ranking.

The overwhelming need to improve the quality of complex data structures in healthcare is more important than ever. Although data quality has been the point of interest in many studies, none of them has focused on the development of quantitative and explainable methods for data imputation. In this work, we propose a "smart" imputation workflow to address missing data across complex data structures in the context of in silico clinical trials. AI algorithms were utilized to produce high-quality virtual patient profiles. A search algorithm was then developed to extract the best virtual patient profiles through the definition of a profile matching score (PMS). A case study was conducted, where the real dataset was randomly contaminated with multiple missing values (e.g., 10 to 50%). In total, 10000 virtual patient profiles with less than 0.02 Kullback-Leibler (KL) divergence were produced to estimate the PMS distribution. The best generator achieved the lowest average squared absolute difference (0.4) and average correlation difference (0.02) with the real dataset highlighting its increased effectiveness for data imputation across complex clinical data structures.

The research paper is an introduction to the artificial neural networks and wireless network by using sensor technology. There are myriads types of neural networks which are used in computer network and wireless network technology. Neural networks explain and give the proper demonstration for the applications of neural networks and wireless network. The artificial network and wireless network have strong connection and can be investigated and explained properly. There are also some models which are called mathematical models which can explain and demonstrate the artificial and wireless networks in detail. There are many organizations which are using artificial neural networks by using sensor technology for wide range of purpose. A new technology ZigBee introduced in artificial networks and wireless networks by evolving the new protocol system based on meter reading. Artificial neural networks and wireless network along database management system have great importance in revolutionizing the world in artificial and wireless technology by using different wireless sensors, RF chip, and fusion of RFID system along with nodes of ZigBee technology.

BACKGROUNDTraditional methods of developing predictive models in inflammatory bowel diseases (IBD) rely on using statistical regression approaches to deriving clinical scores such as the Crohn's disease (CD) activity index. However, traditional approaches are unable to take advantage of more complex data structures such as repeated measurements. Deep learning methods have the potential ability to automatically find and learn complex, hidden relationships between predictive markers and outcomes, but their application to clinical prediction in CD and IBD has not been explored previously.AIMTo determine and compare the utility of deep learning with conventional algorithms in predicting response to anti-tumor necrosis factor (anti-TNF) therapy in CD.METHODSThis was a retrospective single-center cohort study of all CD patients who commenced anti-TNF therapy (either adalimumab or infliximab) from January 1, 2010 to December 31, 2015. Remission was defined as a C-reactive protein (CRP) < 5 mg/L at 12 mo after anti-TNF commencement. Three supervised learning algorithms were compared: (1) A conventional statistical learning algorithm using multivariable logistic regression on baseline data only; (2) A deep learning algorithm using a feed-forward artificial neural network on baseline data only; and (3) A deep learning algorithm using a recurrent neural network on repeated data. Predictive performance was assessed using area under the receiver operator characteristic curve (AUC) after 10× repeated 5-fold cross-validation.RESULTSA total of 146 patients were included (median age 36 years, 48% male). Concomitant therapy at anti-TNF commencement included thiopurines (68%), methotrexate (18%), corticosteroids (44%) and aminosalicylates (33%). After 12 mo, 64% had CRP < 5 mg/L. The conventional learning algorithm selected the following baseline variables for the predictive model: Complex disease behavior, albumin, monocytes, lymphocytes, mean corpuscular hemoglobin concentration and gamma-glutamyl transferase, and had a cross-validated AUC of 0.659, 95% confidence interval (CI): 0.562-0.756. A feed-forward artificial neural network using only baseline data demonstrated an AUC of 0.710 (95%CI: 0.622-0.799; P = 0.25 vs conventional). A recurrent neural network using repeated biomarker measurements demonstrated significantly higher AUC compared to the conventional algorithm (0.754, 95%CI: 0.674-0.834; P = 0.036).CONCLUSIONDeep learning methods are feasible and have the potential for stronger predictive performance compared to conventional model building methods when applied to predicting remission after anti-TNF therapy in CD.

Product Data Management (PDM) system as primary component of the PLM concept has the goal of supporting product development by appropriate methods. Interactive visualization methods support various user groups (e.g. designers, project leaders, managers) in the interpretation and analysis of large and complex data structures in PDM systems which lead to improved decision making. This paper presents a novel approach for the application of visualization methods in PDM systems and a generic architecture for efficient data interpretation.

We introduce a novel statistical modeling technique to cluster analysis and apply it to financial data. Our main goals are to handle missing data and to find homogeneous groups within the data. Our approach is flexible and handles large and complex data structures with missing observations and with quantitative and qualitative measurements. We achieve this by mapping the data to a new structure that is free of distributional assumptions in choosing homogeneous groups of observations. Our new method also provides insight into the number of different categories needed for classifying the data. We use this approach to partition a matched sample of stocks. One group offers dividend reinvestment plans, and the other does not. Our approach partitions this sample with almost 97 percent accuracy even when using only easily available financial variables. One interpretation of this result is that the misclassified companies are the best candidates either to adopt a dividend reinvestment plan (if they have none) or to abandon one (if they currently offer one). We offer other suggestions for applications in the field of finance.

Discovering clusters remains challenging when dealing with complex data structures, including those with varying densities, arbitrary shapes, weak separability, or the presence of noise. In this article, we propose a novel clustering algorithm called representative point-based clustering with neighborhood information (RPC-NI), which highlights the significance of neighborhood information often overlooked by existing clustering methods. The proposed algorithm first introduces a new local centrality metric that integrates both neighborhood density and topological convergence to identify core representative points, effectively capturing the structural characteristics of the data. Subsequently, a density-adaptive distance is defined to evaluate dissimilarities between these core representative points, and such distance is used to construct a minimum spanning tree (MST) over these points. Finally, an MST-based clustering algorithm is employed to yield the desired clusters. Incorporating neighborhood information enables RPC-NI to comprehensively determine representative points, and having multiple representative points per cluster allows RPC-NI to adapt to clusters of arbitrary shapes, varying densities, and different sizes. Extensive experiments on widely used datasets demonstrate that RPC-NI outperforms baseline algorithms in terms of clustering accuracy and robustness. These results provide further evidence for the importance of incorporating neighborhood information discovering clusters with complex structures.

An object-oriented approach to modeling financial accounting systems