A simple linear algebra identity to optimize large-scale neural network quantum states

We utilize neural network quantum states (NQS) to investigate the ground state properties of the Heisenberg model on a Shastry-Sutherland lattice using the variational Monte Carlo method. We show that already relatively simple NQSs can be used to approximate the ground state of this model in its different phases and regimes. We first compare several types of NQSs with each other on small lattices and benchmark their variational energies against the exact diagonalization results. We argue that when precision, generality, and computational costs are taken into account, a good choice for addressing larger systems is a shallow restricted Boltzmann machine NQS. We then show that such NQS can describe the main phases of the model in zero magnetic field. Moreover, NQS based on a restricted Boltzmann machine correctly describes the intriguing plateaus forming in magnetization of the model as a function of increasing magnetic field.

Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansätze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in ab initio quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet’s time complexity outperforms that of the transformer. Though this efficiency comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model—namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS.

To distill the essential elements of nuclear binding, we seek the simplest Hamiltonian capable of modeling atomic nuclei with percent-level accuracy. A critical aspect of this endeavor consists of accurately solving the quantum many-body problem without incurring an exponential computing cost with the number of nucleons. We address this challenge by leveraging a variational MonteCarlo method based on a highly expressive neural-network quantum state ansatz. In addition to computing binding energies and charge radii of nuclei with up to A=20 nucleons, by evaluating their magnetic moments, we demonstrate that neural-network quantum states are able to correctly capture the self-emerging nuclear shell structure. To this end, we introduce a novel computational protocol based on adding an external magnetic field to the nuclear Hamiltonian, which allows the neural network to learn the preferred polarization of the nucleus within the given magnetic field.

Motivated by the Carleo’s work (Science, 2017, 355: 602), we focus on finding the neural network quantum statesapproximation of the unknown ground state of a given Hamiltonian H in terms of the best relative error and explore the influences of sum, tensor product, local unitary of Hamiltonians on the best relative error. Besides, we illustrate our method with some examples.

Solving quantum many-body problems is one of the fundamental challenges in quantum chemistry. While neural network quantum states (NQS) have emerged as a promising computational tool, its training process incurs exponentially growing computational demands, becoming prohibitively expensive for large-scale molecular systems and creating fundamental scalability barriers for real-world applications. To address the above challenges, we present QChem-Trainer, a highperformance NQS training framework for ab initio electronic structure calculations. First, we propose a scalable sampling parallelism strategy with multi-layer workload division and hybrid sampling scheme, which breaks the scalability barriers in large-scale NQS training. Second, we introduce multi-level energy calculation parallelism, enabling more efficient local energy computation in NQS training. Last, we employ cachecentric optimization for transformer-based ansatz and incorporate it with sampling parallelism strategy, which further accelerates the NQS training and achieves stable memory footprints at scale. Experiments demonstrate that QChem-Trainer provides up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$8.41 \times$</tex> speedup in NQS training and attains a parallel efficiency of up to 95.8% when scaling to 1,536 nodes.

Neural network quantum states such as ansatz wave functions have shown a great deal of promise for finding the ground state of spin models. Recently, work has focused on extending this idea to mixed states for simulating the dynamics of open systems. Most approaches so far have used a purification ansatz where a copy of the system Hilbert space is added, which when traced out gives the correct density matrix. Here we instead present an extension of the restricted Boltzmann machine which directly represents the density matrix in Liouville space. This allows the compact representation of states which appear in mean-field theory. We benchmark our approach on two different versions of the dissipative transverse-field Ising model, which show our ansatz is able to compete with other state-of-the-art approaches. Published by the American Physical Society 2024

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Introduction. The article considers the solution of the problem of detecting fires at technological facilities in automatic mode. To solve this problem, an approach was chosen to create a convolutional neural network capable of operating on a real-time video stream. Aims and Purposes. The aim of this work is to create a neural network capable of detecting flames and smoke in the image from CCTV cameras. The purposes of the work include: selection of the optimal architecture of the neural network in accordance with the latest research in this field; speeding up the chosen architecture using quantization and filter thinning techniques. Methods. Different architectures of convolutional neural networks performing the task of detecting objects in an image are considered. Their performance and quality of work are compared. The YOLOv5 architecture, its target function, training methods and ways of speeding up work are considered. Results and discu ssion. The paper shows the results of training a convolutional neural network of the YOLOv5 architecture for the task of flame and smoke detection, as well as how the results change when applying neural network acceleration methods. It was determined that the use of such acceleration methods such as quantization and filter cleaning can significantly increase up the speed of the neural network, while almost no loss in accuracy of operation. Conclusions. As a result of the conducted work, the architecture of the neural network was determined for performing the task of detecting a fire source. Based on the chosen architecture, a neural network was trained to detect flames and smoke in the image. The speed of its work allows to process video stream in real-time without using graphic accelerator.

The design of neural network models involves numerous complexities, including the determination of input vectors, choosing the number of hidden layers and their computational units, and specifying activation functions for the latter. The combinatoric possibilities are daunting, yet experience has yielded informal guidelines that can be useful. Alternatively, current research on genetic algorithms (GA) suggests that they might be of practical use as a formal method of determining ‘good’ architectures for neural networks. In this paper, we use a genetic algorithm to find effective architectures for backpropagation neural networks (BP). We compare the performance of heuristically designed BP networks with that of GA-designed BP networks. Our test domains are sets of problems having compensatory, conjunctive, and mixed-decision structures. The results of our experiment suggest that heuristic methods produce architectures that are simpler and yet perform comparatively well. © 1998 John Wiley & Sons, Ltd.

Determining the best model for the neural network architecture and how to optimize the architecture with the metaheuristic Protis Approach is a subject of the study. A comprehensive investigation and utilization of metaheuristic methods are necessary. These methods aim to solve problems and adapt from the lifestyle of the amoeba protis. In this study, the proposed method modifies the life cycle of the amoeba, which consists of four phases: prophase, metaphase, anaphase, and telophase. These four phases are modified in the neural network architecture to optimize the appropriate number of hidden layers and produce an efficient architecture model. The results show that the protis approach optimized the neural network architecture, especially in generating hidden layers to improve the neural network model. Distinctive features of the results obtained are that the average range of degenerate neurons in the hidden layer is 0 to 35 neurons in each layer. The standard number of neurons makes it possible to solve the problem of determining the best model on the neural network architecture. The protis algorithm embedded in the protis recurrent neural network for categorical data measurements produces an average RMSE value, representing the difference between actual measurements and predictions, equal to 0.066. Consequently, the developed model surpasses the current classical neural network model in terms of performance. Regarding accuracy, the protis algorithm embedded in the neural network for categorical and time series data achieves an average precision of 0.952 and a recall of 0.950. The protis convolutional neural network achieves an accuracy of 95.9 %. Therefore, from the three tested datasets, the protis convolutional neural network exhibits the highest accuracy value

Deep neural networks (DNNs) are advantageous for representing complex nonlinear relationships. This paper applies DNNs to source localization in a shallow water environment. Two methods are proposed to estimate the range and depth of a broadband source through different neural network architectures. The first adopts the classical two-stage scheme, in which feature extraction and DNN analysis are independent steps. The eigenvectors associated with the modal signal space are extracted as the input feature. Then, the time delay neural network is exploited to model the long term feature representation and constructs the regression model. The second concerns a convolutional neural network-feed-forward neural network (CNN-FNN) architecture, which trains the network directly by taking the raw multi-channel waveforms as input. The CNNs are expected to perform spatial filtering for multi-channel signals, in an operation analogous to time domain filters. The outputs of CNNs are summed as the input to FNN. Several experiments are conducted on the simulated and experimental data to evaluate the performance of the proposed methods. The results demonstrate that DNNs are effective for source localization in complex and varied water environments, especially when there is little precise environmental information.

Multilevel image segmentation with adaptive image context based thresholding

Exploiting photoplethysmography signals (PPG) for non-invasive blood pressure (BP) measurement is interesting for various reasons. First, PPG can easily be measured using fingerclip sensors. Second, camera based approaches allow to derive remote PPG (rPPG) signals similar to PPG and therefore provide the opportunity for non-invasive measurements of BP. Various methods relying on machine learning techniques have recently been published. Performances are often reported as the mean average error (MAE) on the data which is problematic. This work aims to analyze the PPG- and rPPG based BP prediction error with respect to the underlying data distribution. First, we train established neural network (NN) architectures and derive an appropriate parameterization of input segments drawn from continuous PPG signals. Second, we use this parameterization to train NNs with a larger PPG dataset and carry out a systematic evaluation of the predicted blood pressure. The analysis revealed a strong systematic increase of the prediction error towards less frequent BP values across NN architectures. Moreover, we tested different train/test set split configurations which underpin the importance of a careful subject-aware dataset assignment to prevent overly optimistic results. Third, we use transfer learning to train the NNs for rPPG based BP prediction. The resulting performances are similar to the PPG-only case. Finally, we apply different personalization techniques and retrain our NNs with subject-specific data for both the PPG-only and rPPG case. Whilst the particular technique is less important, personalization reduces the prediction errors significantly.

A Quantum-Inspired Self-Supervised Network model for automatic segmentation of brain MR images

Due to the advent of modern embedded systems and mobile devices with constrained resources, there is a great demand for incredibly efficient deep neural networks for machine learning purposes. There is also a growing concern of privacy and confidentiality of user data within the general public when their data is processed and stored in an external server which has further fueled the need for developing such efficient neural networks for real-time inference on local embedded systems. The scope of our work presented in this paper is limited to image classification using a convolutional neural network. A Convolutional Neural Network (CNN) is a class of Deep Neural Network (DNN) widely used in the analysis of visual images captured by an image sensor, designed to extract information and convert it into meaningful representations for real-time inference of the input data. In this paper, we propose a neoteric variant of deep convolutional neural network architecture to ameliorate the performance of existing CNN architectures for real-time inference on embedded systems. We show that this architecture, dubbed CondenseNeXt, is remarkably efficient in comparison to the baseline neural network architecture, CondenseNet, by reducing trainable parameters and FLOPs required to train the network whilst maintaining a balance between the trained model size of less than 3.0 MB and accuracy trade-off resulting in an unprecedented computational efficiency.