Investigation of regularized techniques for boundary knot method

The boundary knot method (BKM) is a simple boundary-type meshless method. Due to the use of non-singular general solutions rather than singular fundamental solutions, BKM does not need to consider the artificial boundary. Therefore, this method has the merits of purely meshless, easy to program, high solution accuracy and so on. In this paper, we investigate the effectiveness of the BKM for solving Helmholtz-type problems under various conditions through a series of novel numerical experiments. The results demonstrate that the BKM is efficient and achieves high computational accuracy for problems with smooth or continuous boundary conditions. However, when applied to discontinuous boundary problems, the method exhibits significant numerical instability, potentially leading to substantial deviations in the computed results. Finally, three potential improvement strategies are proposed to mitigate this limitation.

Fast multipole accelerated boundary knot method for inhomogeneous Helmholtz problems

Boundary knot method for heat conduction in nonlinear functionally graded material

Three-dimensional transient heat conduction analysis by boundary knot method

Boundary Knot Method for Convection-diffusion Problems

BackgroundMicroarray technology is widely used in cancer diagnosis. Successfully identifying gene biomarkers will significantly help to classify different cancer types and improve the prediction accuracy. The regularization approach is one of the effective methods for gene selection in microarray data, which generally contain a large number of genes and have a small number of samples. In recent years, various approaches have been developed for gene selection of microarray data. Generally, they are divided into three categories: filter, wrapper and embedded methods. Regularization methods are an important embedded technique and perform both continuous shrinkage and automatic gene selection simultaneously. Recently, there is growing interest in applying the regularization techniques in gene selection. The popular regularization technique is Lasso (L1), and many L1 type regularization terms have been proposed in the recent years. Theoretically, the Lq type regularization with the lower value of q would lead to better solutions with more sparsity. Moreover, the L1/2 regularization can be taken as a representative of Lq (0 <q < 1) regularizations and has been demonstrated many attractive properties.ResultsIn this work, we investigate a sparse logistic regression with the L1/2 penalty for gene selection in cancer classification problems, and propose a coordinate descent algorithm with a new univariate half thresholding operator to solve the L1/2 penalized logistic regression. Experimental results on artificial and microarray data demonstrate the effectiveness of our proposed approach compared with other regularization methods. Especially, for 4 publicly available gene expression datasets, the L1/2 regularization method achieved its success using only about 2 to 14 predictors (genes), compared to about 6 to 38 genes for ordinary L1 and elastic net regularization approaches.ConclusionsFrom our evaluations, it is clear that the sparse logistic regression with the L1/2 penalty achieves higher classification accuracy than those of ordinary L1 and elastic net regularization approaches, while fewer but informative genes are selected. This is an important consideration for screening and diagnostic applications, where the goal is often to develop an accurate test using as few features as possible in order to control cost. Therefore, the sparse logistic regression with the L1/2 penalty is effective technique for gene selection in real classification problems.

The most critical concern in machine learning is how to make an algorithm that performs well both on training data and new data. No free lunch theorem implies that each specific task needs its own tailored machine learning algorithm to be designed. A set of strategies and preferences are built into learning machines to tune them for the problem at hand. These strategies and preferences, with the core concern of generalization improvement, are collectively known as regularization. In deep learning, because of a considerable number of parameters, a great many forms of regularization methods are available to the deep learning community. Developing more effective regularization strategies has been the subject of significant research efforts in recent years. However, it is difficult for developers to choose the most suitable strategy for their problem at hand, because there is no comparative study regarding the performance of different strategies. In this paper, at the first step, the most effective regularization methods and their variants are presented and analyzed in a systematic approach. At the second step, comparative research on regularization techniques is presented in which the testing errors and computational costs are evaluated in a convolutional neural network, using CIFAR-10 ( https://www.cs.toronto.edu/~kriz/cifar.html ) dataset. In the end, different regularization methods are compared in terms of accuracy of the network, the number of epochs for the network to be trained and the number of operations per input sample. Also, the results are discussed and interpreted based on the employed strategy. The experiment results showed that weight decay and data augmentation regularizations have little computational side effects so can be used in most applications. In the case of enough computational resources, Dropout family methods are rational to be used. Moreover, in the case of abundant computational resources, batch normalization family and ensemble methods are reasonable strategies to be employed.

For interior tomography based on truncated Hilbert transform (THT), the recently proposed truncated singular value decomposition (TSVD) reconstruction method uses a regularizati on parameter given directly. In this paper, a method of choosing the regularization parameter is presented based on L-curve to get an optimal regularization parameter in theoretical sense. Furthermore, we develop a Tikhonov regula rization method in comparison to TSVD. Simulation results indicate that both of the two regularization methods with the optimal regularization parameters have good performances on the image quality for both cases of noise-free and noisy projections. Keywords: Interior Tomography, Truncated Hilbert Transform, L-curve, Truncated Singular Value Decomposition, Tikhonov Regularization 1. INTRODUCTION It had been believed that the solution to the interior tomography is not unique fo r a long time. Recently, inspired by the concept of the differential backprojection (DBP) proposed by Noo et al [1] and Pan et al [2], a novel solution for the interior problem was proposed with numerical results demonstrat ed that the interior problem can be uniquely and stably solved if a sub-region inside a region of interest (ROI) is known [3]-[5]. Similar results were also independently proposed by other researchers [6]-[8]. In these papers, image reconstruction was reduced to the inversion of THT via the projection onto convex set (POCS) method, which is iterativ e and computationally expensive. To over above limitations, TSVD as a regularization method has been proposed for inversing the THT, which runs two orders of magnitude faster than the iterative method [9]. However the regularization parame ter in this method is given directly without theoretical analysis. The selection of the regularization parameter that controls the trade-off between fidelity to the data and constraint on the reconstruction result is a primary problem in the regularization method. In this paper, we present a method of selecting the optimal regularization based on the L-curve for TSVD method for solving the interior tomography based on truncated Hilbert transform. In this method, the optimal regularization parameter is computed theoretically using the maximum curvature of the L-curve. Moreover, we develop a Tikhonov regularization method in comparison to TSVD. Root Mean Sq uare Error (RMSE) of reconstr ucted image with respect to different regularization parameters validates our selecting regularization parameter method. The paper is organized as follows: section 2 gives Mathematical formulation, TSVD, Tikhonov regularization, the L-curve. Section 3 exhibits the simulation results. Section 4 is conclusion.

This article presents a local knot method (LKM) to solve inverse Cauchy problems of Helmholtz equations in arbitrary 2D and 3D domains. The Moore–Penrose pseudoinverse using the truncated singular value decomposition is employed in the local approximation of supporting domain instead of the moving least squares method. The developed approach is a semi‐analytical and local radial basis function collocation method using the non‐singular general solution as the basis function. Like the traditional boundary knot method, the LKM is simple, accurate and easy‐to‐program in solving inverse Cauchy problems associated with Helmholtz equations. Unlike the boundary knot method, the new scheme can directly reconstruct the unknowns both inside the physical domain and along its boundary by solving a sparse linear system, and can achieve a satisfactory solution. Numerical experiments, involving the complicated geometry and the high noise level, confirm the accuracy and reliability of the proposed method for solving inverse Cauchy problems of Helmholtz equations.

This research investigates Elastic Net and demonstrates its effectiveness as a regularization technique for digital predistortion (DPD). Elastic Net is compared to the popular and recent techniques including a standard Least Squares (LS) approach, Ridge Regression, and Least Absolute Shrinkage and Selection Operator (LASSO). Performance of these regularization techniques is compared using adjacent channel power ratio (ACPR), normalized mean square error (NMSE), and number of coefficients. Elastic Net consistently performed better compared to linearization with other regularization methods and achieved over 3dB better NMSE and ACPR performance on average. An experimental hardware testbed with multiple Doherty power amplifiers was used to validate the use of Elastic Net for DPD.

Regularization is an essential element of virtually all kernel methods for nonparametric regression problems. A critical factor in the effectiveness of a given kernel method is the type of regularization that is employed. This article compares and contrasts members from a general class of regularization techniques, which notably includes ridge regression and principal component regression. We derive an explicit finite-sample risk bound for regularization-based estimators that simultaneously accounts for (i) the structure of the ambient function space, (ii) the regularity of the true regression function, and (iii) the adaptability (or qualification) of the regularization. A simple consequence of this upper bound is that the risk of the regularization-based estimators matches the minimax rate in a variety of settings. The general bound also illustrates how some regularization techniques are more adaptable than others to favorable regularity properties that the true regression function may possess. This, in particular, demonstrates a striking difference between kernel ridge regression and kernel principal component regression. Our theoretical results are supported by numerical experiments.

The Newsletter is a key element in further enhancing the value of the journal to the inverse problems community. So why not be a part of this exciting forum by sending to our Bristol office material suitable for inclusion under any of the categories mentioned above. Your contributions will be very welcome. Book review Introduction to Inverse Problems in Imaging M Bertero and P Boccacci 1998 Bristol: Institute of Physics 362 pp ISBN 0-7503-0435-9 (pbk) £25.00, $49.00 This book shows that several problems in imaging are in fact linear inverse problems. In general inverse problems are ill-posed in the sense that small perturbations in the (measured) data have a significant influence on the output. Consequently, numerical methods developed in a general setting of inverse problems which cure the ill-posedness (such methods are called regularization methods) can be applied to solve problems in imaging. This is an excellent textbook on the principle of linear inverse problems, methods of their numerical solution and practical applications in imaging. It introduces basic ideas and methods for the solution of inverse and ill-posed problems while avoiding the mathematics of functional treatment of operator equations. This goal is achieved by a bottom-up presentation: the authors focus on the two problems of image deconvolution and tomography (bottom) and develop regularization theory specifically for these two problems. Most other books which are concerned with the numerical solution of inverse problems present a top-down approach. That is, a general regularization theory (top) is developed which is then applied to particular problems. The distinguished presentation makes the book very useful to students in applied mathematics, practitioners in engineering science and image processing. This book has the best prerequisites to establish a close link between the mathematical fields of imaging and inverse problems. Mainly the book focuses on two topics: image deconvolution and linear imaging systems. 1. In image deconvolution the goal is to restore a space invariant blurred image. This part of the book contains mathematical tools in image deconvolution, examples of space invariant imaging systems, a comparison of image deconvolution techniques and low-pass filtering techniques, the analysis of the ill-posedness of the deconvolution problem, regularization methods for deconvolution such as constrained least squares regularization, Tikhonov regularization, iterative regularization and statistical methods. Since the authors focus on particular problems they are able to analyse various methods for deconvolution on a very concrete level. As an example I would like to mention that the authors analyse Fourier methods for deconvolution. Thus they provide an optimal starting point to the theory of inverse problems for people working in the area of signal and image processing. 2. The second part of the book considers linear imaging systems that are not of a convolution type such as tomography problems, diffraction problems and inverse scattering problems. Efficient numerical methods for the solution of this class of problems such as singular value decomposition, Tikhonov regularization methods and Fourier-based methods are presented. O Scherzer Universität Linz

Determination of a molecular weight distribution from equilibrium sedimentation has been significantly improved by incorporating linear programming into Tikhonov's regularization method. The conventional way of computation failed because the Fujita equations which relate the distribution function f(m) to the experimental data are improperly posed problems in the Hadamard sense. Their inverse solution necessary for determination of a molecular weight distribution is unstable. As a result such a solution leads to an erratic curve instead of the desired f(m). This erratic behavior is significantly improved by applying Tikhonov's regularization method. However, a certain amount of oscillation in the f(m) curve still remains uncorrected. Linear programming, which was originally introduced to eliminate undesired negative f(m) values, also served as an additional smoothing factor and improved the calculated f(m) curve. Good results were obtained in cases of a unimodal, a bimodal, and a trimodal distribution.

The availability and quality of experimental data pose challenges for identifying the role of individual proteins in signal transduction pathways. To address this issue, this paper formulates and solves an inverse problem to determine the dynamics of transcription factors, i.e., one important class of proteins, from fluorescence intensity measurements of green fluorescent protein (GFP) reporter systems. In the presented approach, a model describing transcription and translation of GFP is discretized and concentrations of transcription factor are estimated at discrete time points. Unlike previous works, this approach has no restrictions with regard to a particular shape of the profiles. However, the resulting inverse problem is ill-conditioned and requires the use of regularization techniques. Two regularization methods - truncated singular value decomposition and Tikhonov regularization - are investigated in this work and the characteristics of the results obtained are discussed in detail.

Abstract. Accurate quantification of long-term trends in stratospheric ozone can be challenging due to their sensitivity to natural variability, the quality of the observational datasets, and non-linear changes in forcing processes as well as the statistical methodologies. Multivariate linear regression (MLR) is the most commonly used tool for ozone trend analysis; however, the complex coupling in many atmospheric processes can make it prone to the issue of over-fitting when using the conventional ordinary-least-squares (OLS) approach. To overcome this issue, here we adopt a regularized (ridge) regression method to estimate ozone trends and quantify the influence of individual processes. We use the Stratospheric Water and OzOne Satellite Homogenized (SWOOSH) merged dataset (v2.7) to derive stratospheric ozone profile trends for the period 1984–2020. Besides SWOOSH, we also analyse a machine-learning-based satellite-corrected gap-free global stratospheric ozone profile dataset from a chemical transport model (ML-TOMCAT) and output from a chemical transport model (TOMCAT) simulation forced with European Centre for Medium-Range Weather Forecasts (ECMWF) ERA5 reanalysis. For 1984–1997, we observe smaller negative trends in the SWOOSH stratospheric ozone profile using ridge regression compared to OLS. Except for the tropical lower stratosphere, the largest differences arise in the mid-latitude lowermost stratosphere (&gt;4 % per decade difference at 100 hPa). From 1998 and the onset of ozone recovery in the upper stratosphere, the positive trends estimated using the ridge regression model (∼1 % per decade near 2 hPa) are smaller than those using OLS (∼2 % per decade). In the lower stratosphere, post-1998 negative trends with large uncertainties are observed and ridge-based trend estimates are somewhat smaller and less variable in magnitude compared to the OLS regression. Aside from the tropical lower stratosphere, the largest difference is around 2 % per decade at 100 hPa (with ∼3 % per decade uncertainties for individual trends) in northern mid-latitudes. For both time periods the SWOOSH data produce large negative trends in the tropical lower stratosphere with a correspondingly large difference between the two trend methods. In both cases the ridge method produces a smaller trend. The regression coefficients from both OLS and ridge models, which represent ozone variations associated with natural processes (e.g. the quasi-biennial oscillation, solar variability, El Niño–Southern Oscillation, Arctic Oscillation, Antarctic Oscillation, and Eliassen–Palm flux), highlight the dominance of dynamical processes in controlling lower-stratospheric ozone concentrations. Ridge regression generally yields smaller regression coefficients due to correlated explanatory variables, and care must be exercised when comparing fit coefficients and their statistical significance across different regression methods. Comparing the ML-TOMCAT-based trend estimates with the ERA5-forced model simulation, we find ML-TOMCAT shows significant improvements with much better consistency with the SWOOSH dataset, despite the ML-TOMCAT training period overlapping with SWOOSH only for the Microwave Limb Sounder (MLS) measurement period. The largest inconsistencies with respect to SWOOSH-based trends post-1998 appear in the lower stratosphere where the ERA5-forced model simulation shows positive trends for both the tropics and the mid-latitudes. The large differences between satellite-based data and the ERA5-forced model simulation confirm significant uncertainties in ozone trend estimates, especially in the lower stratosphere, underscoring the need for caution when interpreting results obtained with different regression methods and datasets.