Deep learning tomographic reconstruction through hierarchical decomposition of domain transforms

Traditionally, analytical methods have been used to solve imaging problems such as image restoration, inpainting, and superresolution (SR). In recent years, the fields of machine and deep learning have gained a lot of momentum in solving such imaging problems, often surpassing the performance provided by analytical approaches. Unlike analytical methods for which the problem is explicitly defined and domain-knowledge carefully engineered into the solution, deep neural networks (DNNs) do not benefit from such prior knowledge and instead make use of large data sets to learn the unknown solution to the inverse problem. In this article, we review deep-learning techniques for solving such inverse problems in imaging. More specifically, we review the popular neural network architectures used for imaging tasks, offering some insight as to how these deep-learning tools can solve the inverse problem. Furthermore, we address some fundamental questions, such as how deeplearning and analytical methods can be combined to provide better solutions to the inverse problem in addition to providing a discussion on the current limitations and future directions of the use of deep learning for solving inverse problem in imaging.

Ultrasound computed tomography is an inexpensive and radiation-free medical imaging technique used to quantify the tissue acoustic properties for advanced clinical diagnosis. Image reconstruction in ultrasound tomography is often modeled as an optimization scheme solved by iterative methods like full-waveform inversion. These iterative methods are computationally expensive, while the optimization problem is ill-posed and nonlinear. To address this problem, we propose to use deep learning to overcome the computational burden and ill-posedness, and achieve near real-time image reconstruction in ultrasound tomography. We aim to directly learn the mapping from the recorded time-series sensor data to a spatial image of acoustical properties. To accomplish this, we develop a deep learning model using two cascaded convolutional neural networks with an encoder–decoder architecture. We achieve a good representation by first extracting the intermediate mapping-knowledge and later utilizing this knowledge to reconstruct the image. This approach is evaluated on synthetic phantoms where simulated ultrasound data are acquired from a ring of transducers surrounding the region of interest. The measurement data is acquired by forward modeling the wave equation using the k-wave toolbox. Our simulation results demonstrate that our proposed deep-learning method is robust to noise and significantly outperforms the state-of-the-art traditional iterative method both quantitatively and qualitatively. Furthermore, our model takes substantially less computational time than the conventional full-wave inversion method.

12 - Direct image reconstruction in electrical tomography and its applications

The paper considers methods for optimizing training samples for deep learning algorithms through the use of random point processes, such as Matern of the first and second types, Gibbs, Gaussian, and Poisson processes. An approach to reducing training data without sacrificing informativeness is proposed, enabling a decrease in computational costs and mitigating the risk of overfitting. A novel method of representing images as random point processes is introduced, allowing a transition from the pixel based representation of an image to a model more suitable for analysis with point process techniques. This transition to a more compact empirical form facilitates subsequent analysis and modeling. In addition, it enables the use of statistical tools to uncover patterns hidden within images. The effectiveness of random point processes in shaping the feature space, analyzing coverage, and structuring the training dataset is demonstrated. The study also considers the impact of different types of point processes on data balance and their ability to reduce redundancy within the sample. Particular attention is given to the issue of data representativeness, as it directly affects the stability and generalization capability of deep learning models. Algorithms for converting images into point processes and their application for class balancing, data thinning, and enhancing the representativeness of samples are presented. The evaluation of classification accuracy, conducted using ResNet models, highlights the advantages of applying point processes over random data thinning. The results confirm the effectiveness of point processes for optimizing large-scale training datasets and improving the accuracy of deep learning. Furthermore, the findings indicate that these methods may play a key role in developing more lightweight and efficient neural network models. The study outlines promising directions for future research in adaptive optimization of training samples, where random point processes may serve as the foundation for new approaches to data preparation in neural network training.

Nowadays, the application of deep learning is more and more widely, and children’s face recognition has made great progress. However, in the existing framework of children’s face recognition based on deep learning, many tasks (including face recognition, authentication and attribute classification) are relatively independent, and its overall algorithm is inefficient and time-consuming. In order to solve this problem, this paper proposes a deep convolution network based on keras framework. Taking children’s face recognition, authentication and attributes as the parameters of the network, the whole deep convolution network can be trained step by step. The network can complete three tasks at the same time without further steps. The network model is trained and learned on the training samples. It is an efficient neural network model. The network structure scheme of this method can improve the classification accuracy It has a certain positive impact. The purpose of this paper is to construct a network model structure, which can make the children face recognition effect better and still obtain good performance under the limited data support.

Many learning problems in computer vision can be posed as structured prediction problems, where the input and output instances are structured objects such as trees, graphs or strings rather than, single labels {+1, -1} or scalars. Kernel methods such as Structured Support Vector Machines, Twin Gaussian Processes (TGP), Structured Gaussian Processes, and vector-valued Reproducing Kernel Hilbert Spaces (RKHS), offer powerful ways to perform learning and inference over these domains. Positive definite kernel functions allow us to quantitatively capture similarity between a pair of instances over these arbitrary domains. A poor choice of the kernel function, which decides the RKHS feature space, often results in poor performance. Automatic kernel selection methods have been developed, but have focused only on kernels on the input domain (i.e.'one-way'). In this work, we propose a novel and efficient algorithm for learning kernel functions simultaneously, on both input and output domains. We introduce the idea of learning polynomial kernel transformations, and call this method Simultaneous Twin Kernel Learning (STKL). STKL can learn arbitrary, but continuous kernel functions, including 'one-way' kernel learning as a special case. We formulate this problem for learning covariances kernels of Twin Gaussian Processes. Our experimental evaluation using learned kernels on synthetic and several real-world datasets demonstrate consistent improvement in performance of TGP's.

Introduction: Protein conformational switches are often constructed by fusing an input domain, which recognizes a target ligand, to an output domain that establishes a biological response. Prior designs have employed binding-induced folding of the input domain to drive a conformational change in the output domain. Adding a second input domain can in principle harvest additional binding energy for performing useful work. It is not obvious, however, how to fuse two binding domains to a single output domain such that folding of both binding domains combine to effect conformational change in the output domain.Methods: Here, we converted the ribonuclease barnase (Bn) to a switchable enzyme by duplicating a C-terminal portion of its sequence and appending it to its N-terminus, thereby establishing a native fold (OFF state) and a circularly permuted fold (ON state) that competed for the shared core in a mutually exclusive fashion. Two copies of FK506 binding protein (FKBP), both made unstable by the V24A mutation and one that had been circularly permuted, were inserted into the engineered barnase at the junctions between the shared and duplicated sequences.Results: Rapamycin-induced folding of FK506 binding protein stretched and unfolded the native fold of barnase via the mutually exclusive folding effect, and rapamycin-induced folding of permuted FK506 binding protein stabilized the permuted fold of barnase by the loop-closure entropy principle. These folding events complemented each other to turn on RNase function. The cytotoxic switching mechanism was validated in yeast and human cells, and in vitro with purified protein.Discussion: Thermodynamic modeling and experimental results revealed that the dual action of loop-closure entropy and mutually exclusive folding is analogous to an engine transmission in which loop-closure entropy acts as the low gear, providing efficient switching at low ligand concentrations, and mutually exclusive folding acts as the high gear to allow the switch to reach its maximum response at high ligand concentrations.

A primary issue in black-box testing is how to generate adequate test cases from input domain of the system under test on the basis of user's requirement specification. However, for some types of systems including embedded systems, developing test cases from output domain is more suitable than developing from input domain, especially, when the output domain is smaller. This approach ensures better reliability of the system under test. In this paper, the authors present a new approach to automate the generation of test cases from output domain of a pilot project "Temperature Monitoring and Controlling of Nuclear Reactor System" (TMCNRS) which is an embedded system developed using modified Cleanroom Software Engineering methodology. An Automated Test Case Generator (ATCG) that uses Genetic algorithms (GAs) extensively and generates test cases from output domain is proposed. The ATCG generates test cases which are useful to conduct pseudo - exhaustive testing to detect single, double and several multimode faults in the system. The generator considers most of the combinations of outputs, and finds the corresponding inputs while optimizing the number of test cases generated. In order to investigate the effectiveness of this approach, test cases were generated by ATCG and the tests were conducted on the target embedded system at a minimum cost and time. Experimental results show that this approach is very promising.

Image reconstruction for Electrical Resistance Tomography (ERT) is an ill-posed nonlinear inverse problem. Considering the influence of the sparse measurement data on the quality of the reconstructed image, the l1-regularized least-squares program (l1 regularized LSP) is introduced to solve the inverse problem in this paper. To meet the need of high speed in ERT, the fast iterative shrinkage-thresholding algorithm (FISTA) is employed for image reconstruction in our work. Simulation results of the FISTA and l1_ls algorithm show that the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularized LSP is superior to the l2 regularization method, especially in avoiding the over-smoothing of the reconstructed image. In addition, to improve the convergence speed and imaging quality in FISTA algorithm, the initial guess is calculated with the conjugate gradient method. Comparative simulation results demonstrate the feasibility of FISTA in ERT system and its advantage over the l1_ls regularization method.

Image reconstruction in optoacoustic tomography (OAT) is a trending learning task highly dependent on measured physical magnitudes present at sensing time. A large number of different settings and also the presence of uncertainties or partial knowledge of parameters can lead to reconstruction algorithms that are specifically tailored and designed to a particular configuration, which could not be the one that will ultimately be faced in a final practical situation. Being able to learn reconstruction algorithms that are robust to different environments (e.g., the different OAT image reconstruction settings) or invariant to such environments is highly valuable because it allows us to focus on what truly matters for the application at hand and discard what are considered spurious features. In this work, we explore the use of deep learning algorithms based on learning invariant and robust representations for the OAT inverse problem. In particular, we consider the application of the ANDMask scheme due to its easy adaptation to the OAT problem. Numerical experiments are conducted showing that when out-of-distribution generalization (against variations in parameters such as the location of the sensors) is imposed, there is no degradation of the performance and, in some cases, it is even possible to achieve improvements with respect to standard deep learning approaches where invariance robustness is not explicitly considered.

Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases

The Newsletter is a key element in further enhancing the value of the journal to the inverse problems community. So why not be a part of this exciting forum by sending to our Bristol office material suitable for inclusion under any of the categories mentioned above. Your contributions will be very welcome. Book review Introduction to Inverse Problems in Imaging M Bertero and P Boccacci 1998 Bristol: Institute of Physics 362 pp ISBN 0-7503-0435-9 (pbk) £25.00, $49.00 This book shows that several problems in imaging are in fact linear inverse problems. In general inverse problems are ill-posed in the sense that small perturbations in the (measured) data have a significant influence on the output. Consequently, numerical methods developed in a general setting of inverse problems which cure the ill-posedness (such methods are called regularization methods) can be applied to solve problems in imaging. This is an excellent textbook on the principle of linear inverse problems, methods of their numerical solution and practical applications in imaging. It introduces basic ideas and methods for the solution of inverse and ill-posed problems while avoiding the mathematics of functional treatment of operator equations. This goal is achieved by a bottom-up presentation: the authors focus on the two problems of image deconvolution and tomography (bottom) and develop regularization theory specifically for these two problems. Most other books which are concerned with the numerical solution of inverse problems present a top-down approach. That is, a general regularization theory (top) is developed which is then applied to particular problems. The distinguished presentation makes the book very useful to students in applied mathematics, practitioners in engineering science and image processing. This book has the best prerequisites to establish a close link between the mathematical fields of imaging and inverse problems. Mainly the book focuses on two topics: image deconvolution and linear imaging systems. 1. In image deconvolution the goal is to restore a space invariant blurred image. This part of the book contains mathematical tools in image deconvolution, examples of space invariant imaging systems, a comparison of image deconvolution techniques and low-pass filtering techniques, the analysis of the ill-posedness of the deconvolution problem, regularization methods for deconvolution such as constrained least squares regularization, Tikhonov regularization, iterative regularization and statistical methods. Since the authors focus on particular problems they are able to analyse various methods for deconvolution on a very concrete level. As an example I would like to mention that the authors analyse Fourier methods for deconvolution. Thus they provide an optimal starting point to the theory of inverse problems for people working in the area of signal and image processing. 2. The second part of the book considers linear imaging systems that are not of a convolution type such as tomography problems, diffraction problems and inverse scattering problems. Efficient numerical methods for the solution of this class of problems such as singular value decomposition, Tikhonov regularization methods and Fourier-based methods are presented. O Scherzer Universität Linz

Modelling of neural networks is still a very interesting and important field in the area of computing models. If input domains of neural networks are data structures represented by graphs and output domain is expected in a similar form, it is necessary to consider it in the process of neural network modelling. We propose four models of extended Self Organizing Maps (SOM) that can be applied to graph data structures as input and output domains together with learning algorithms. Extensions of the SOM model are based on the idea to remember information of connections in data structures (using some context neurons).

Artificial intelligence, including deep learning, is currently revolutionising the field of medical imaging, with far reaching implications for almost every facet of diagnostic imaging, including patient radiation safety. This paper introduces basic concepts in deep learning and provides an overview of its recent history and its application in tomographic reconstruction as well as other applications in medical imaging to reduce patient radiation dose, as well as a brief description of previous tomographic reconstruction techniques. This review also describes the commonly used deep learning techniques as applied to tomographic reconstruction and draws parallels to current reconstruction techniques. Finally, this paper reviews some of the estimated dose reductions in CT and positron emission tomography in the recent literature enabled by deep learning, as well as some of the potential problems that may be encountered such as the obscuration of pathology, and highlights the need for additional clinical reader studies from the imaging community.

The purpose of this study is to evaluate and compare the performance of modern deep learning techniques for automatically recognizing and segmenting multiple organ regions on 3D CT images. CT image segmentation is one of the important task in medical image analysis and is still very challenging. Deep learning approaches have demonstrated the capability of scene recognition and semantic segmentation on nature images and have been used to address segmentation problems of medical images. Although several works showed promising results of CT image segmentation by using deep learning approaches, there is no comprehensive evaluation of segmentation performance of the deep learning on segmenting multiple organs on different portions of CT scans. In this paper, we evaluated and compared the segmentation performance of two different deep learning approaches that used 2D- and 3D deep convolutional neural networks (CNN) without- and with a pre-processing step. A conventional approach that presents the state-of-the-art performance of CT image segmentation without deep learning was also used for comparison. A dataset that includes 240 CT images scanned on different portions of human bodies was used for performance evaluation. The maximum number of 17 types of organ regions in each CT scan were segmented automatically and compared to the human annotations by using ratio of intersection over union (IU) as the criterion. The experimental results demonstrated the IUs of the segmentation results had a mean value of 79% and 67% by averaging 17 types of organs that segmented by a 3D- and 2D deep CNN, respectively. All the results of the deep learning approaches showed a better accuracy and robustness than the conventional segmentation method that used probabilistic atlas and graph-cut methods. The effectiveness and the usefulness of deep learning approaches were demonstrated for solving multiple organs segmentation problem on 3D CT images.