AI for Condensed Matter Physics

The Sixth International Symposium on Neural Networks (ISNN 2009)

Transformability, generalizability, but limited diffusibility: Comparing global vs. task-specific language representations in deep neural networks

The brain can be viewed as a complex modular structure with features of information processing through knowledge storage and retrieval. Modularity ensures that the knowledge is stored in a manner where any complications in certain modules do not affect the overall functionality of the brain. Although artificial neural networks have been very promising in prediction and recognition tasks, they are limited in terms of learning algorithms that can provide modularity in knowledge representation that could be helpful in using knowledge modules when needed. Multi-task learning enables learning algorithms to feature knowledge in general representation from several related tasks. There has not been much work done that incorporates multi-task learning for modular knowledge representation in neural networks. In this paper, we present multi-task learning for modular knowledge representation in neural networks via modular network topologies. In the proposed method, each task is defined by the selected regions in a network topology (module). Modular knowledge representation would be effective even if some of the neurons and connections are disrupted or removed from selected modules in the network. We demonstrate the effectiveness of the method using single hidden layer feedforward networks to learn selected n-bit parity problems of varying levels of difficulty. Furthermore, we apply the method to benchmark pattern classification problems. The simulation and experimental results, in general, show that the proposed method retains performance quality although the knowledge is represented as modules.

Dispatching rules are frequently used for real-time, online scheduling in complex manufacturing systems. Design of such rules is usually done by experts in a time consuming trial-and-error process. Recently, evolutionary algorithms have been proposed to automate the design process. There are several possibilities to represent rules for this hyper-heuristic search. Because the representation determines the search neighborhood and the complexity of the rules that can be evolved, a suitable choice of representation is key for a successful evolutionary algorithm. In this paper we empirically compare three different representations, both numeric and symbolic, for automated rule design: A linear combination of attributes, a representation based on artificial neural networks, and a tree representation. Using appropriate evolutionary algorithms (CMA-ES for the neural network and linear representations, genetic programming for the tree representation), we empirically investigate the suitability of each representation in a dynamic stochastic job shop scenario. We also examine the robustness of the evolved dispatching rules against variations in the underlying job shop scenario, and visualize what the rules do, in order to get an intuitive understanding of their inner workings. Results indicate that the tree representation using an improved version of genetic programming gives the best results if many candidate rules can be evaluated, closely followed by the neural network representation that already leads to good results for small to moderate computational budgets. The linear representation is found to be competitive only for extremely small computational budgets.

In this paper, we present a novel approach for driving style classification based on time series data. Instead of automatically learning the embedding vector for temporal representation of the input data with Recurrent Neural Networks, we propose a combination of Hyperdimensional Computing (HDC) for data representation in high-dimensional vectors and much simpler feed-forward neural networks. This approach provides three key advantages: first, instead of having a “black box” of Recurrent Neural Networks learning the temporal representation of the data, our approach allows to encode this temporal structure in high-dimensional vectors in a human-comprehensible way using the algebraic operations of HDC while only relying on feed-forward neural networks for the classification task. Second, we show that this combination is able to achieve at least similar and even slightly superior classification accuracy compared to state-of-the-art Long Short-Term Memory (LSTM)-based networks while significantly reducing training time and the necessary amount of data for successful learning. Third, our HDC-based data representation as well as the feed-forward neural network, allow implementation in the substrate of Spiking Neural Networks (SNNs). SNNs show promise to be orders of magnitude more energy-efficient than their rate-based counterparts while maintaining comparable prediction accuracy when being deployed on dedicated neuromorphic computing hardware, which could be an energy-efficient addition in future intelligent vehicles with tight restrictions regarding on-board computing and energy resources. We present a thorough analysis of our approach on a publicly available data set including a comparison with state-of-the-art reference models.

This paper explores the use and applications of neural networks in the construction of a text markup program. This research paper describes various tasks that require textual data analysis and discusses the problems, issues, and solutions that accompany them. Interest in neural networks has increased in recent years, and they are find­ing applications in a wide variety of fields, such as business, medicine, engineering, geology, and physics. Neural networks have made great strides in forecasting, planning, and management. There are several reasons for this situation. Neural networks are very powerful modeling systems capable of creating complex dependencies. Neural networks can be widely used in areas such as text/speech recognition, semantic search, decision support/expert systems, inventory forecasting, data storage systems and content analysis. The object of the work is the process of functioning of neural networks and an algorithm for text markup recognition. The purpose of the scientific paper is the application of neural networks in the construction of the program for the markup of the text. In order to achieve the goal, the following tasks were put forward: a) study existing neural networks, b) choose a neural network to cre­ate a model and study its structure, c) convert input data to feed it into a neural network model. Representation and analysis of symbolic structures in neural networks seems to be an interesting and useful direction in neural network theory. In this paper, we have reviewed some neural network architectures that may merit further consideration in these circumstances. In what follows, we will focus on specific experiments in this area. Thus, this paper outlines a problem area for further research and testing.

Combining dual-energy computed tomography (DECT) with positron emission tomography (PET) offers many potential clinical applications but typically requires expensive hardware upgrades or increases radiation doses on PET/CT scanners due to an extra X-ray CT scan. The recent PET-enabled DECT method allows DECT imaging on PET/CT without requiring a second X-ray CT scan. It combines the already existing X-ray CT image with a 511 keV γ -ray CT (gCT) image reconstructed from time-of-flight PET emission data. A kernelized framework has been developed for reconstructing gCT image but this method has not fully exploited the potential of prior knowledge. Use of deep neural networks may explore the power of deep learning in this application. However, common approaches require a large database for training, which is impractical for a new imaging method like PET-enabled DECT. Here, we propose a single-subject method by using neural-network representation as a deep coefficient prior to improving gCT image reconstruction without population-based pre-training. The resulting optimization problem becomes the tomographic estimation of nonlinear neural-network parameters from gCT projection data. This complicated problem can be efficiently solved by utilizing the optimization transfer strategy with quadratic surrogates. Each iteration of the proposed neural optimization transfer algorithm includes: PET activity image update; gCT image update; and least-square neural-network learning in the gCT image domain. This algorithm is guaranteed to monotonically increase the data likelihood. Results from computer simulation, real phantom data and real patient data have demonstrated that the proposed method can significantly improve gCT image quality and consequent multi-material decomposition as compared to other methods.

Using the trajectory conception of state, we give a simple demonstration that the quantum state of a many-body system may be expressed as a set of states in three-dimensional space, each associated with a different particle. It follows that the many-body wavefunction may be derived from a set of waves in 3-space. Entanglement is represented in the trajectory picture by the mutual dependence of the 3-states on the trajectory labels.

A fundamental issue in the implementation of symbol-manipulation with neural networks is the unlimited productivity of symbol-production systems. Because each realistic neural network is a finite-state system [1], unlimited productivity cannot be implemented with neural networks. Therefore, the only symbol-production which seems possible with neural networks is the production of a regular (finite-state) language. This has serious consequences for the generation of cognitive behaviour with neural networks, in particular natural language processing, which requires the productivity of non-regular production systems [2]. If it is not possible to implement such production systems in neural networks, neural network theory will remain on the level of behaviourism.

Neurons in the brain form thousands of connections, or synapses, with one another, allowing signals to pass from one cell to the next. To activate a neuron, a high enough activating signal or ‘action potential’ must be reached. However, the accepted view of signal transmission assumes that the great majority of synapses are too weak to activate neurons. This means that often simultaneous inputs from many neurons are required to trigger a single neuron’s activation. However, such coordination is likely unreliable as neurons can react differently to the same stimulus depending on the circumstances. An alternative way of transmitting signals has been reported in turtle brains, where impulses from a single neuron can trigger activity across a network of connections. Furthermore, these responses are reliably repeatable, activating the same neurons in the same order. Riquelme et al. set out to understand the mechanism that underlies this type of neuron activation using a mathematical model based on data from the turtle brain. These data showed that the neural network in the turtle’s brain had many weak synapses but also a few, rare, strong synapses. Simulating this neural network showed that those rare, strong synapses promote the signal’s reliability by providing a consistent route for the signal to travel through the network. The numerous weak synapses, on the other hand, have a regulatory role in providing flexibility to how the activation spreads. This combination of strong and weak connections produces a system that can reliably promote or stop the signal flow depending on the context. Riquelme et al.’s work describes a potential mechanism for how signals might travel reliably through neural networks in the brain, based on data from turtles. Experimental work will need to address whether strong connections play a similar role in other animal species, including humans. In the future, these results may be used as the basis to design new systems for artificial intelligence, building on the success of neural networks.

The goal of this paper is to give an overview of neural networks from the viewpoint of measurements. It shows that neural networks can play an important role in measurement systems; for example they can be used in complex sensor's systems, and in utilization of the observation data, to build system's models. From the point of view of measurement systems the key property of neural networks is their universal approximation capability. The paper formulates questions about neural networks - which are important for their applications in measurement systems - and gives an overview of the possible answers obtained from the theory of neural networks. The paper contrasts the theoretical results and the engineering questions. It points out that the theoretical results achieved in the last few years are of primary importance; however there are strong limits of the application of the theoretical results.

The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of O(N), where N is the system size. Beyond the threshold, they are completely lost. Since the introduction of the Hopfield model, the theory of neural networks has been further developed toward realistic neural networks using analog neurons, spiking neurons, etc. Nevertheless, those advances are based on fully connected networks, which are inconsistent with recent experimental discovery that the number of connections of each neuron seems to be heterogeneous, following a heavy-tailed distribution. Motivated by this observation, we consider the Hopfield model on scale-free networks and obtain a different pattern of associative memory retrieval from that obtained on the fully connected network: the storage capacity becomes tremendously enhanced but with some error in the memory retrieval, which appears as the heterogeneity of the connections is increased. Moreover, the error rates are also obtained on several real neural networks and are indeed similar to that on scale-free model networks.

In this paper an classification method based on polarimetric decomposition technique and neural network theory, is proposed for polarimetric SAR data sets. The main advantage of this polarimetric decomposition technique is to provide dominant polarimetric scattering properties identification information where the most important kinds of scattering medium can be discriminated. Feature vector extracted from full POLSAR data sets by polarimetric decomposition is used as input data of the feed-forward neural network (FNN). Neural networks have the advantage to be independent to the input signal statistics and the ability to combine many parameters in their inputs. To speed convergence and improve stability of the FNN Kalman filter plus scaled conjugate gradient algorithm is used in the training stage. The NASA/JPL AIRSAR c-band data of San Francisco is used to illustrate the effectiveness of the proposed approach to classification. Quantitative results of performance are provided, as compared to the Wishart classifier.

AI and ML have emerged as transformative tools in various scientific domains, including hydrogel design. This work explores the integration of AI and ML techniques in the realm of hydrogel development, highlighting their significance in enhancing the design, characterisation, and optimisation of hydrogels for diverse applications. We introduced the concept of AI train hydrogel design, underscoring its potential to decode intricate relationships between hydrogel compositions, structures, and properties from complex data sets. In this work, we outlined classical physical and chemical techniques in hydrogel design, setting the stage for AI/ML advancements. These methods provide a foundational understanding for the subsequent AI-driven innovations. Numerical and analytical methods empowered by AI/ML were also included. These computational tools enable predictive simulations of hydrogel behaviour under varying conditions, aiding in property customisation. We also emphasised AI's impact, elucidating its role in rapid material discovery, precise property predictions, and optimal design. ML techniques like neural networks and support vector machines that expedite pattern recognition and predictive modelling using vast datasets, advancing hydrogel formulation discovery are also presented. AI and ML's have a transformative influence on hydrogel design. AI and ML have revolutionised hydrogel design by expediting material discovery, optimising properties, reducing costs, and enabling precise customisation. These technologies have the potential to address pressing healthcare and biomedical challenges, offering innovative solutions for drug delivery, tissue engineering, wound healing, and more. By harmonising computational insights with classical techniques, researchers can unlock unprecedented hydrogel potentials, tailoring solutions for diverse applications.

An emission predictive system for CO and NOx from gas turbine based on ensemble machine learning approach