Deep learning for inverse problems with unknown operator

The paper considers the problem of performing a task defined on a model parameter that is only observed indirectly through noisy data in an ill-posed inverse problem. A key aspect is to formalize the steps of reconstruction and task as appropriate estimators (non-randomized decision rules) in statistical estimation problems. The implementation makes use of (deep) neural networks to provide a differentiable parametrization of the family of estimators for both steps. These networks are combined and jointly trained against suitable supervised training data in order to minimize a joint differentiable loss function, resulting in an end-to-end task adapted reconstruction method. The suggested framework is generic, yet adaptable, with a plug-and-play structure for adjusting both the inverse problem and the task at hand. More precisely, the data model (forward operator and statistical model of the noise) associated with the inverse problem is exchangeable, e.g., by using neural network architecture given by a learned iterative method. Furthermore, any task that is encodable as a trainable neural network can be used. The approach is demonstrated on joint tomographic image reconstruction, classification and joint tomographic image reconstruction segmentation.

Indonesian Sign Language or BISINDO is an alternative language used by people who suffer from disabilities, especially those who have hearing impairments. This language grew and developed from the deaf community, so its use is based on the visual aspect. This research aims to apply Residual Networks to detect objects in the context of Bisindo Letter Sign Language, with the hope of increasing accuracy and efficiency in letter recognition. Object detection goes through 2 stages, namely feature extraction and model training. ResNet is a type of Convolutional Neural Network (CNN) architecture that utilizes models that have been previously trained, so it can save the time required in the model development process. In this research, Residual Network (ResNet) was used for feature extraction to recognize important aspects in the Bisindo letter sign image, such as hand position, finger shape characteristics, and direction of movement. The research results show that the new dataset used as training data and test data has a fairly good ability to detect with a division of 70% train set, 20% valid set and 10% test set with size 640x640 with 300 epochs for the training model.

When we use a discretization by finite differences, to solve differential equations we find problems at the border of the domain of the solution. If the solution is also immersed in a ill-posed inverse problem; we can find very bad solutions. In this paper we apply a discretization of two - sided difference quotients method to solve Ill-posed inverse self-adjoint elliptic problem [Kirsch(2011)]. Some numerical examples showing the effectiveness of this method and we will use mollification techniques to smooth the solutions.

Distribution of electrical potentials over the surface of the heart, which is called the epicardial potential distribution, is a valuable tool to understand whether there is a defect in the heart. Direct measurement of these potentials requires highly invasive procedures. An alternative is to reconstruct these epicardial potentials non-invasively from the body surface potentials, which constitutes one form of the ill-posed inverse problem of electrocardiography (ECG). The goal of this study is to solve the inverse problem of ECG using several regularization methods and compare their performances. We employed Tikhonov Regularization, Truncated Singular Value Decomposition (TSVD), Least Squares QR (LSQR) methods in this study. We compared the effectiveness of these regularization methods to solve the ill-posed inverse ECG problem. Some of the regularization methods require a regularization parameter to solve the inverse problem. We used the well-known L-Curve method to obtain the regularization parameter. The performance of the regularization methods for solving the inverse ECG problem was also evaluated based on a realistic heart-torso model simulation protocol. In this paper, we also investigated the usage of genetic algorithm (GA) for regularizing the ill-posed inverse ECG problem. The results showed that GA can be applied to regularize the ill-posed problem when combined with the results of conventional regularization methods or additional information about solutions.

We consider a model problem of recovering a function $f(x_1,x_2)$ from noisy Radon data. The function $f$ to be recovered is assumed smooth apart from a discontinuity along a $C^2$ curve, that is, an edge. We use the continuum white-noise model, with noise level $\\varepsilon$.\n¶ Traditional linear methods for solving such inverse problems behave poorly in the presence of edges. Qualitatively, the reconstructions are blurred near the edges; quantitatively, they give in our model mean squared errors (MSEs) that tend to zero with noise level $\\varepsilon$ only as $O(\\varepsilon^{1/2})$ as $\\varepsilon\\to 0$. A recent innovation--nonlinear shrinkage in the wavelet domain--visually improves edge sharpness and improves MSE convergence to $O(\\varepsilon^{2/3})$. However, as we show here, this rate is not optimal.\n¶ In fact, essentially optimal performance is obtained by deploying the recently-introduced tight frames of curvelets in this setting. Curvelets are smooth, highly anisotropic elements ideally suited for detecting and synthesizing curved edges. To deploy them in the Radon setting, we construct a curvelet-based biorthogonal decomposition of the Radon operator and build "curvelet shrinkage" estimators based on thresholding of the noisy curvelet coefficients. In effect, the estimator detects edges at certain locations and orientations in the Radon domain and automatically synthesizes edges at corresponding locations and directions in the original domain.\n¶ We prove that the curvelet shrinkage can be tuned so that the estimator will attain, within logarithmic factors, the MSE $O(\\varepsilon^{4/5})$ as noise level $\\varepsilon\\to 0$. This rate of convergence holds uniformly over a class of functions which are $C^2$ except for discontinuities along $C^2$ curves, and (except for log terms) is the minimax rate for that class. Our approach is an instance of a general strategy which should apply in other inverse problems; we sketch a deconvolution example.

In this paper we study the relation between convergence rates of spectral regularization methods under Hölder-type source conditions resulting from the theory of ill-posed inverse problems, when the noise level $\delta$ goes to 0, and convergence rates resulting from statistical kernel learning, when the number of samples n goes to infinity. Toward this aim, we introduce a family of hybrid estimators in the statistical learning context whose convergence rates have the following properties: first, they are equal to those of spectral methods, and second, they are connected to the rates of spectral regularization in ill-posed inverse problems, provided that a suitable inverse proportionality relation between n and $\delta$ holds true. This family of estimators allows us to convert upper rates depending on $n$ to upper rates depending on $\delta$ and to convert lower rates vice versa, quantifying their deviation. The analysis is carried out under general source conditions in the case the rank of the forward operator is both finite and infinite, and, in the latter case, both by not making any assumptions on the eigenvalues and by assuming a polynomial eigenvalue decay.

In recent years, the rapid development of computer software has led to numerous security problems, particularly software vulnerabilities. These flaws can cause significant harm to users’ privacy and property. Current security defect detection technology relies on manual or professional reasoning, leading to missed detection and high false detection rates. Artificial intelligence technology has led to the development of neural network models based on machine learning or deep learning to intelligently mine holes, reducing missed alarms and false alarms. So, this project aims to study Java source code defect detection methods for defects like null pointer reference exception, XSS (Transform), and Structured Query Language (SQL) injection. Also, the project uses open-source Javalang to translate the Java source code, conducts a deep search on the AST to obtain the empty syntax feature library, and converts the Java source code into a dependency graph. The feature vector is then used as the learning target for the neural network. Four types of Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), Bi-directional Long Short-Term Memory (BiLSTM), and Attention Mechanism + Bidirectional LSTM, are used to investigate various code defects, including blank pointer reference exception, XSS, and SQL injection defects. Experimental results show that the attention mechanism in two-dimensional BLSTM is the most effective for object recognition, verifying the correctness of the method.

Summary The interpretation of faults within a geological basin or reservoir from seismic data is a time-consuming, and often manual task associated with high uncertainties. Recently, numerous approaches using machine learning, especially various types of convolutional neural networks, have been presented to automate the process of identifying fault planes within seismic images, which have been shown to outperform traditional fault detection techniques. While these proposed methods show good performance, many of these approaches do not allow investigation of the associated uncertainties that arise in the fault identification process. In this study, we present an application of Bayesian deep convolutional neural networks for identifying faults within seismic datasets. Using an approximate Bayesian inference method a Bayesian deep neural network was trained on a large dataset of synthetic faulted seismic images. The model is then applied to a benchmark dataset and a real data case from NW shelf Australia to identify fault planes, and to investigate the associated uncertainty in the predictive distribution.

Accelerometers have been widely used in almost every area of science and engineering. They are supposed to be physical instruments for measuring the acceleration of a moving object. Although huge technological advance is made in hardware of accelerometers over more than one century, accelerometers have been persistently designed and fabricated mechanically under the framework of forward problems of damped mass–spring systems. We show that accelerometers are essentially inverse ill-posed source problems from the mathematical point of view, implying that small measurement errors of equivalent displacements are inherently amplified significantly such that accelerations output from accelerometers can become extremely noisy, numerically incorrect and physically meaningless. The ill-posedness of accelerometers has been always implicitly circumvented approximately for more than one century. As a result, accelerometers theoretically can only produce approximate outputs of acceleration. Here we present the concept of computerized accelerometers, because inverse ill-posed problems, as in the case of accelerometers, cannot be rigorously solved mechanically. The acceleration can only rigorously be reconstructed computationally as a regularized solution to the inverse ill-posed source problem of acceleration from equivalent displacements. The concept of computerized accelerometers theoretically warrants precise measurement of acceleration without approximation, is valid for nonlinear time-dependent damping as well and provides a turning point for accelerometers to become a fully rigorous computerized physical instrument. Simulated examples have confirmed that least squares reconstruction of acceleration can be too extremely noisy to be physically meaningful and shown that elastic acceleration is incorrect by more than 100%.

The article discusses the problem of determining the learning outcomes for educational programs of the university, including at the program level and the level of individual disciplines included in the program. The goal is to substantiate the understanding of the definition of learning outcomes at a university as an inverse ill-posed problem within the framework of the theory of ill-posed problems, which will allow further reengineering of the educational process at a university based on models of inverse and ill-posed problems. The article identifies the types of uncertainty inherent in education, which affect the possibility of an incorrect decision. The publication substantiates a new understanding of the educational process focused on learning outcomes at a university, as a process of finding a solution to an inverse incorrect problem. In this case, the learning outcomes formulated for a specific educational program are considered as an available solution to the inverse problem. The research carried out by the authors made it possible to construct a mathematical model of the educational process at the university and identify its components. The authors demonstrate that the optimal way to design the educational process is to design it as an inverse problem, and the list of disciplines, their content and the corresponding learning outcomes are determined and developed at subsequent stages, focusing on the goals of the program.

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical path-length sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A virus), HAdV (adenovirus), and ZIKV (Zika virus). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic-processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates. The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically.

Defect classification of wind turbine blades is a popular topic in non-destructive health testing of complex components. This paper studies the wind turbine blade non-destructive testing (NDT) defect classification method based on a wavelet packet transform-convolutional neural network (WPT-CNN). Samples of wind turbine blades with different defect types were made and the datasets were collected using ultrasonic non-destructive inspection. The performance of two types of convolutional neural network (CNN) structure in predicting defects in wind turbine blades was compared. The results indicate that deep convolutional neural networks (DCNNs) are effective for defect detection in wind turbine blades, demonstrating greater feasibility and effectiveness in automatic classification based on deep learning models.

Historical database training method for neural networks

The unfolding problem in particle physics is to make inferences about the true particle spectrum based on smeared observations from a detector. This is an ill-posed inverse problem, where small changes in the smeared distribution can lead to large fluctuations in the unfolded distribution. The forward operator is the response matrix which models the detector response. In practice, the forward operator is rarely known analytically and is instead estimated using Monte Carlo simulation. This raises the question of how to best estimate the response matrix and what impact this estimation has on the unfolded solutions. In most analyses at the LHC, response matrix estimation is done by binning the true and smeared events and counting the propagation of events between the bins. However, this approach can result in a noisy estimate of the response matrix, especially with a small Monte Carlo sample size. Unexpectedly, we also find that the noise in the estimated response matrix can inadvertently regularize the problem. As an alternative, we propose to estimate the response matrix through the use of conditional density estimation of the response kernel in the unbinned setting followed by binning this estimator. Using simulation studies, we investigate the performance of the two approaches.

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.