Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers

Over the last 20 years, our learning environment for linear algebra has changed dramatically mathematical tools take an important role in our classes. Sage is popular mathematical software which was released in 2005. This software has efficient features to adapt the internet environment and it can cover most of mathematical problems, for example, algebra, combinatorics, numerical mathematics, calculus and linear algebra. Nowadays there are more mobile/smartphones than the number of personal computers in the world. Furthermore, the most sophisticated smartphones have almost the same processing power as personal computer and it can be connected to the internet. For example, we can connect from mobile phone to any Sage server through the internet. We have developed over the years on Mobile mathematics with Smartphone for teaching linear algebra (Ko et al. 2009; Lee and Kim 2009; Lee et al. 2001). In this article, we introduce Sage and how we can use it in our linear algebra classes. We aim to show the mobile infrastructure of the Sage and the mobile-learning environment. We shall also introduce mobile contents for linear algebra using Sage. In fact, almost all the concepts of linear algebra can be easily covered.

Graph neural networks (GNNs) have been widely used to learn node representations from graph data in an unsupervised way for downstream tasks. However, when applied to detect anomalies (e.g., outliers, unexpected density), they deliver unsatisfactory performance as existing loss functions fail. For example, any loss based on random walk (RW) algorithms would no longer work because the assumption that anomalous nodes were close with each other could not hold. Moreover, the nature of class imbalance in anomaly detection tasks brings great challenges to reduce the prediction error. In this work, we propose a novel loss function to train GNNs for anomaly-detectable node representations. It evaluates node similarity using global grouping patterns discovered from graph mining algorithms. It can automatically adjust margins for minority classes based on data distribution. Theoretically, we prove that the prediction error is bounded given the proposed loss function. We empirically investigate the GNN effectiveness of different loss variants based on different algorithms. Experiments on two real-world datasets show that they perform significantly better than RW-based loss for graph anomaly detection.

Graph drawing techniques have been developed in the last few years with the purpose of producing esthetically pleasing node-link layouts. Recently, the employment of differentiable loss functions has paved the road to the massive usage of gradient descent and related optimization algorithms. In this article, we propose a novel framework for the development of Graph Neural Drawers (GNDs), machines that rely on neural computation for constructing efficient and complex maps. GND is Graph Neural Networks (GNNs) whose learning process can be driven by any provided loss function, such as the ones commonly employed in Graph Drawing. Moreover, we prove that this mechanism can be guided by loss functions computed by means of feedforward neural networks, on the basis of supervision hints that express beauty properties, like the minimization of crossing edges. In this context, we show that GNNs can nicely be enriched by positional features to deal also with unlabeled vertexes. We provide a proof-of-concept by constructing a loss function for the edge crossing and provide quantitative and qualitative comparisons among different GNN models working under the proposed framework.

We study the excess mean square error (EMSE) above the minimum mean square error (MMSE) in large linear systems where the posterior mean estimator (PME) is evaluated with a postulated prior that differs from the true prior of the input signal. We focus on large linear systems where the measurements are acquired via an independent and identically distributed random matrix, and are corrupted by additive white Gaussian noise (AWGN). The relationship between the EMSE in large linear systems and EMSE in scalar channels is derived, and closed form approximations are provided. Our analysis is based on the decoupling principle, which links scalar channels to large linear system analyses. Numerical examples demonstrate that our closed form approximations are accurate.

Video on learning of mathematics is one of the audio-visual media-based learning which is currently not widely used, because in addition need quite a long time, and also need requires special mastery in the field of computers and learning software. The background of this research focused on the importance of strengthening the ability of students to the course material which is conducted independently after the learning process in the classroom implemented. This study aims to assess and analyze the results of learning mathematics in linear algebra course in STKIP Sebelas April Sumedang. This research method using pre-experimental design with type One-Shot Case Study of the entire fourth semester students of the majoring in mathematics education in STKIP Sebelas April Sumedang that consisting of two groups taken purposively. The data was analyzed by two different test average (t test) of the two groups that received different treatment. These results indicate that there are significant differences in learning outcomes between the groups were given a course with additional self learning video by using Camstudio Software and Pen Tablet on Linear Algebra Class and the group given regular course

During the recent decades, multigrid methods have been extensively used in order to solve large scale linear systems derived from the discretization of partial differential equations using the finite difference method. The purpose of this work is the derivation of hybrid heterogeneous schemes based on time implicit approximating schemes, namely backward differences and Crank–Nicolson method, with the multigrid method in conjunction with explicit approximate inverses for the efficient iterative solution of unsymmetric linear systems. Thus, a new class of smoothers, based on approximate inverses, has been derived. Moreover, a dynamic relaxation scheme is proposed based on the dynamic over - under relaxation (DOUR) algorithm. In addition, a new class of Krylov subspace iterative methods in conjunction with multigrid preconditioning based on explicit approximate inverses is used for solving initial value problems. Finally, the applicability of the new proposed hybrid heterogeneous implicit time - explicit approximate inverses combined with multigrid methods for parabolic PDEs is discussed by solving characteristic time-dependent problems in two dimensions and the numerical results are given.

Solution of a large sparse or dense linear systems often constitutes the most time consuming step when solving large multidimensional industrial problems on modern high performance supercomputers. In most practical cases these problems produce linear systems with ill-conditioned and off-diagonal dominant coefficient matrices. As the problem size grows, especially in the 3-dimensional case, direct methods for solving these systems (such as triangular decomposition) begin to lose their effectiveness due to the large arithmetic costs and memory requirements. These difficulties motivate resorting to iterative solution methods which are, at least theoretically, more efficient with respect to the arithmetic cost and memory requirements. On March 12, 1993 Qasim Sheikh from Cray Research presented numerical experiments with iterative solvers that are capable of solving large linear systems which arise in industrial applications. He then compared the effectiveness and some performance aspects of these iterative methods with other state of the art, highly optimized, iterative as well as direct solvers. The main results of his talk are based on a recent paper [1] written jointly with Kharchenko, Nikishin, Yeremin and Heroux.KeywordsIterative SolverPreconditioned MatrixLarge Linear SystemNonsymmetric Linear SystemTurbine PumpThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Solving a linear system of equations is one of the easiest computational tasks imaginable, but when the matrix is very large, the algorithms we learned in linear algebra class take too long, and we need to use a different type of method. In this homework assignment, we investigate these iterative methods.

E. ARTIN (1957), R. BAER (1952) and J.DIEUDONNE (1951) emphasized many times the structural identity between classical projective geometry and linear algebra over a division ring. Then Baer pointed out a possible extension of this structural identity to the case of a ring, generating intense research activity in the area of geometric algebra over rings. In this direction the most significant results appear: Ojanguren and Sridharan’s article on the fundamental theorem of projective geometry over a commutative ring; some theorems due to Klingenberg, Bass and Suslin on the structure of the general linear group over appropriate rings and some valuable notes of O’Meara’s on the automorphisms of linear groups over an integral domain.

Graph neural networks (GNNs) have known an increasing success recently, with many GNN variants achieving state-of-the-art results on node and graph classification tasks. The proposed GNNs, however, often implement complex node and graph embedding schemes, which makes it challenging to explain their performance. In this paper, we investigate the link between a GNN’s expressiveness, that is, its ability to map different graphs to different representations, and its generalization performance in a graph classification setting. In particular, we propose a principled experimental procedure where we (i) define a practical measure for expressiveness, (ii) introduce an expressiveness-based loss function that we use to train a simple yet practical GNN that is permutation-invariant, (iii) illustrate our procedure on benchmark graph classification problems and on an original real-world application. Our results reveal that expressiveness alone does not guarantee a better performance, and that a powerful GNN should be able to produce graph representations that are well separated with respect to the class of the corresponding graphs.

At present, the urban traffic system is faced with the problems of congestion and low efficiency, and the traditional methods have certain limitations when dealing with the complex urban road network. This paper aims to explore a new method of capacity Optimization of Road Networks in a Smart City environment and proposes a method based on a Graph Neural network, named “Smart City Road optimization Graph Neural Network” (SCRO-GNN). SCRO-GNN first collects and preprocesses multi-source data, including road network data, traffic flow, accident records, environmental factors, etc. The key node and edge features, including the number of lanes at the intersection, the traffic flow of the section, and the adjacency matrix of the road network are defined to characterize the structure of the road network. Then, the graph neural network model is constructed and trained to predict the traffic flow of different sections and evaluate the road capacity by using the graph structure of the road network. This paper tests the performance of SCRO-GNN on real road networks in multiple cities. The results show that, compared with the traditional traffic flow prediction model, SCRO-GNN can significantly improve the prediction accuracy, especially when dealing with the highly complex urban road network structure. Based on these predictions, the proposed optimization strategy performed well in reducing traffic congestion and improving the efficiency of road use. The research in this paper not only demonstrates the potential of graph neural networks in smart city road network optimization, but also provides a new direction for future traffic system research. The successful implementation of the SCRO-GNN approach is expected to provide more efficient and intelligent solutions for urban traffic management and planning.

Graph learning on dynamical systems has recently surfaced as an emerging research domain.By leveraging a novel electronic Dynamical System (DS), various graph learning challenges have been effectively tackled through a rapid, spontaneous natural annealing process.This method has attracted increasing attention due to its orders-of-magnitude improvements in speed and energy efficiency compared to traditional Graph Neural Network (GNN) approaches for inference tasks.However, (1) the current DS hardware only supports inference, missing its native solution for training; while relying on conventional hardware is likely more expensive than GNNs.(2) The current DS architecture only allows linear interactions among its nodes, limiting training accuracy.In this work, we present a Dynamical-System Training-Processing Unit (DS-TPU) developed through algorithm-architecture co-design to tackle the two major challenges: (1) An on-device lifelong learning mechanism that leverages feedback electric current as the loss function in response to the observed training data, allowing electron-speed refinement on the present model parameters.(2) A nonlinear DS node interaction mechanism constructed from Chebyshev polynomials to significantly improve the compatibility between the DS hardware and the embedded relation of graph data.Extensive evaluations using six real-world graph learning applications demonstrate that for accuracy, DS-TPU achieves 10.8% MAE reduction over the best results of five widely used GNNs.In terms of training performance, the 5-Watt DS-TPU architecture achieves onaverage 810 speedup over the offline training for DS on an Nvidia A100 GPU, and 640 over GNN training on the same GPU.

Simulating the mechanical response of advanced materials can be done more accurately using concurrent multiscale models than with single-scale simulations. However, the computational costs stand in the way of the practical application of this approach. The costs originate from microscale Finite Element (FE) models that must be solved at every macroscopic integration point. A plethora of surrogate modeling strategies attempt to alleviate this cost by learning to predict macroscopic stresses from macroscopic strains, completely replacing the microscale models. In this work, we introduce an alternative surrogate modeling strategy that allows for keeping the multiscale nature of the problem, allowing it to be used interchangeably with an FE solver for any time step. Our surrogate provides all microscopic quantities, which are then homogenized to obtain macroscopic quantities of interest. We achieve this for an elasto-plastic material by predicting full-field microscopic strains using a graph neural network (GNN) while retaining the microscopic constitutive material model to obtain the stresses. This hybrid data-physics graph-based approach avoids the high dimensionality originating from predicting full-field responses while allowing non-locality to arise. In addition, this approach introduces beneficial inductive bias to the model by encoding microscopic geometrical features. By training the GNN on a variety of meshes, it learns to generalize to unseen meshes, allowing a single model to be used for a range of microstructures. The embedded microscopic constitutive model in the GNN implicitly tracks history-dependent variables and leads to improved accuracy. While the microscopic stresses are fully dependent on the microscopic strains, we found it crucial to include both microscopic strains and stresses in the loss function. We demonstrate for several challenging scenarios that the surrogate can predict complex macroscopic stress–strain paths. As the computation time of our method scales favorably with the number of elements in the microstructure compared to the FE method, our method can significantly accelerate FE2 simulations.

Structure-aware deep clustering network based on contrastive learning

The execution of iteration methods, including iteration methods for time-dependent differential equations, is one of the most time-consuming tasks in scientific computing, especially when large systems have to be solved, large time scales have to be computed or solutions of high quality are required. To decrease the computation time, parallel machines can be used but a detailed analysis of the potential parallelism is needed for an efficient implementation as data dependencies can strongly limit the performance. This article describes and compares the potential parallelism of various iterative solution methods for linear and non-linear equation systems and ordinary differential equations. Methods that have been especially designed for parallel execution as well as parallel implementations of modern efficient sequential solution methods are considered.