Pixel-wise deep learning for improving image reconstruction in photoacoustic tomography

Photoacoustic Imaging (also known as thermoacoustic or optoacoustic imaging) is a novel imaging method which combines the advantages of Diffuse Optical Imaging (high contrast) and Ultrasonic Imaging (high spatial resolution). A short laser pulse excites the sample. The absorbed energy causes a thermoelastic expansion and thereby launches a broadband ultrasonic wave (photoacoustic signal). This way one can measure the optical contrast of a sample with ultrasonic resolution. For collecting photoacoustic signals our group introduced so called integrating detectors a few years ago. Such integrating detectors integrate the pressure in one or two dimensions -a line or a plane detector, respectively. Thereby the three dimensional imaging problem is reduced to a two or a one dimensional problem for the projections and a two or three dimensional inverse radon transform as a second step to get the three dimensional initial pressure distribution. The integrating detectors are mainly optical detectors and thus can provide a high bandwidth up to several 100 MHz. Using these detectors the resolution is often limited by the acoustic attenuation in the sample itself, because attenuation increases with higher frequencies. Stoke's equation describes the attenuation of photoacoustic generated waves in liquids very well, which results in an increase of the acoustic attenuation with the square of the frequency. For fat tissue an absorption coefficient which is approximately linear proportional to frequency is reported. Presented measurements give an exponential power law dependency with an exponent between 1.31 and 1.36 in fat tissue near the skin of a pig. These equations describing frequency dependent acoustic attenuation have been solved in the past by decomposing the pressure wave into plane waves damped in space, described by the complex part of the wave number equal to the attenuation coefficient. One main result of this paper is that for Photoacoustic Tomography another description seems to be very useful: like for a standing wave in a resonator the wave number is real but the frequency is complex. The complex part of the frequency is the damping in time. Both descriptions are equivalent, but with the complex frequency description acoustic attenuation can be included in all k-space methods well known in Photoacoustic Tomography just by introducing a factor describing the exponential decay in time.

Photoacoustic tomography (PAT) is a non-ionizing imaging modality capable of acquiring high contrast and resolution images of optical absorption at depths greater than traditional optical imaging techniques. Practical considerations with instrumentation and geometry limit the number of available acoustic sensors and their “view” of the imaging target, which result in image reconstruction artifacts degrading image quality. Iterative reconstruction methods can be used to reduce artifacts but are computationally expensive. In this work, we propose a novel deep learning approach termed pixel-wise deep learning (Pixel-DL) that first employs pixel-wise interpolation governed by the physics of photoacoustic wave propagation and then uses a convolution neural network to reconstruct an image. Simulated photoacoustic data from synthetic, mouse-brain, lung, and fundus vasculature phantoms were used for training and testing. Results demonstrated that Pixel-DL achieved comparable or better performance to iterative methods and consistently outperformed other CNN-based approaches for correcting artifacts. Pixel-DL is a computationally efficient approach that enables for real-time PAT rendering and improved image reconstruction quality for limited-view and sparse PAT.

There are occasions, perhaps due to hardware constraints, or to speed-up data acquisition, when it is helpful to be able to reconstruct a photoacoustic image from an under-sampled or incomplete data set. Here, we will show how Deep Learning can be used to improve image reconstruction in such cases. Deep Learning is a type of machine learning in which a multi-layered neural network is trained from a set of examples to perform a task. Convolutional Neural Networks (CNNs), a type of deep neural network in which one or more layers perform convolutions, have seen spectacular success in recent years in tasks as diverse as image classification, language processing and game playing. In this work, a series of CNNs were trained to perform the steps of an iterative, gradient-based, image reconstruction algorithm from under-sampled data. This has two advantages: first, the iterative reconstruction is accelerated by learning more efficient updates for each iterate; second, the CNNs effectively learn a prior from the training data set, meaning that it is not necessary to make potentially unrealistic regularising assumptions about the image sparsity or smoothness, for instance. In addition, we show an example in which the CNNs learn to remove artifacts that arise when a slow but accurate acoustic model is replaced by a fast but approximate model. Reconstructions from simulated as well as in vivo data will be shown.

Photoacoustic imaging is a new imaging technology in recent years, which combines the advantages of high resolution and rich contrast of optical imaging with the advantages of high penetration depth of acoustic imaging. Photoacoustic imaging has been widely used in biomedical fields, such as brain imaging, tumor detection and so on. The signal-tonoise ratio (SNR) of image signals in photoacoustic imaging is generally low due to the limitation of laser pulse energy, electromagnetic interference in the external environment and system noise. In order to solve the problem of low SNR of photoacoustic images, we use feedforward denoising convolutional neural network to further process the obtained images, so as to obtain higher SNR images and improve image quality. We use Python language to manage the referenced Python external library through Anaconda, and build a feedforward noise-reducing convolutional neural network on Pycharm platform. We first processed and segmated a training set containing 400 images, and then used it for network training. Finally, we tested it with a series of cerebrovascular photoacoustic microscopy images. The results show that the peak signal-to-noise ratio (PSNR) of the image increases significantly before and after denoising. The experimental results verify that the feed-forward noise reduction convolutional neural network can effectively improve the quality of photoacoustic microscopic images, which provides a good foundation for the subsequent biomedical research.

Accelerated photoacoustic tomography (PAT) reconstruction is important for real-time photoacoustic imaging (PAI) applications. PAT requires a reconstruction algorithm to reconstruct the detected photoacoustic signal in order to obtain the detected image of the tissue, which is usually an inverse problem. Different from the typical method for solving the inverse problems that defines a model and chooses an inference procedure, we propose to use the Recurrent Inference Machines (RIM) as a framework for PAT reconstruction. Our model performs an accelerated iterative reconstruction, and directly learns to solve the inverse problem in PAT using the information from a forward model that is based on k-space methods. As shown in experiments, our method achieves faster high-resolution PAT reconstruction, and outperforms another method based on deep neural network in some respects.

Optoacoustic tomography (OAT), also known as photoacoustic tomography, is a rapidly emerging hybrid imaging technique that possesses great potential for a wide range of biomedical imaging applications. In OAT, a laser is employed to illuminate the tissue of interest and acoustic signals are produced via the photoacoustic effect. From these data, an estimate of the distribution of the absorbed optical energy density within the tissue is reconstructed, referred to as the object function. This quantity is defined, in part, by the distribution of light fluence within the tissue that is established by the laser source. When performing three-dimensional imaging of large objects, such as a female human breast, it can be difficult to achieve a relatively uniform coverage of light fluence within the volume of interest when the position of the laser source is fixed. To circumvent this, researchers have proposed illumination schemes in which the relative position of the laser source and ultrasound probe is fixed, and both are rotated together to acquire a tomographic dataset. A problem with this rotating-illumination scheme is that the tomographic data are inconsistent; namely, the acoustic data recorded at each tomographic view angle (i.e., probe position) are produced by a distinct object function. In this work, the impact of this data inconsistency on image reconstruction accuracy is investigated systematically. This is accomplished by use of computer-simulation studies and application of mathematical results from the theory of microlocal analysis. These studies specify the set of image discontinuities that can be stably reconstructed with a nonstationary optical illumination setup. The study also includes a comparison of the ability of iterative and analytic image reconstruction methods to mitigate artifacts attributable to the data inconsistency.

Delay-and-sum (DAS) beamformers, when applied to photoacoustic (PA) image reconstruction, produce strong sidelobes due to the absence of transmit focusing. Consequently, DAS PA images are often severely degraded by strong off-axis clutter. For preclinical in vivo cardiac PA imaging, the presence of these noise artifacts hampers the detectability and interpretation of PA signals from the myocardial wall, crucial for studying blood-dominated cardiac pathological information and to complement functional information derived from ultrasound imaging. In this article, we present PA subaperture processing (PSAP), an adaptive beamforming method, to mitigate these image degrading effects. In PSAP, a pair of DAS reconstructed images is formed by splitting the received channel data into two complementary nonoverlapping subapertures. Then, a weighting matrix is derived by analyzing the correlation between subaperture beamformed images and multiplied with the full-aperture DAS PA image to reduce sidelobes and incoherent clutter. We validated PSAP using numerical simulation studies using point target, diffuse inclusion and microvasculature imaging, and in vivo feasibility studies on five healthy murine models. Qualitative and quantitative analysis demonstrate improvements in PAI image quality with PSAP compared to DAS and coherence factor weighted DAS (DAS CF ). PSAP demonstrated improved target detectability with a higher generalized contrast-to-noise (gCNR) ratio in vasculature simulations where PSAP produces 19.61% and 19.53% higher gCNRs than DAS and DAS CF , respectively. Furthermore, PSAP provided higher image contrast quantified using contrast ratio (CR) (e.g., PSAP produces 89.26% and 11.90% higher CR than DAS and DAS CF in vasculature simulations) and improved clutter suppression.

Filtered backprojection (FBP) algorithms are commonly employed for image reconstruction in optoacoustic tomography (OAT). A limitation of FBP algorithms is that they require the measured acoustic data to be densely sampled, which necessitates expensive ultrasound arrays that possess a large number of elements or increased data-acquisition times if mechnical scanning is employed. Additionally, FBP algorithms are based on idealized imaging models that do not accurately model the response of the transducers and fail to exploit the statistical characteristics of noisy measurement data to minimize noise levels in the reconstructed images. Iterative image reconstruction algorithms can circumvent these difficulties. However, to date, iterative reconstruction algorithms have not been successfully applied to three-dimensional (3D) OAT. In this work we investigate the use of an iterative image reconstruction method in 3D OAT. The large computational burden of 3D iterative image reconstruction is circumvented by implementing the reconstrution algorithm with graphics processing units (GPUs). The ability of the reconstruction algorithm to mitigate artifacts due to incomplete data is demonstrated.

In recent several years, motivated by the need to reduce radiation doses in CT exams, all of the major CT manufacturers have commercialized different iterative image reconstruction techniques and these innovative techniques were used in clinical routines with increasing popularity. However, due to the intrinsic nonlinearity of these new techniques, the well accepted quantitative image quality assessment metrics such as spatial resolution and contrast to noise ratio are not sufficient to provide the needed quantitative metrics for assessing image quality and for guiding the CT scan protocol optimization. This symposium aims at providing a thorough update to AAPM community on what we have understood in the past for linear CT imaging system, what are the new challenges and opportunities offered by the nonlinear iterative image reconstruction, and what would be the future directions in quantitative image quality assessment that the AAPM community can work together to address the challenges and to adapt the nonlinear image reconstruction methods to routine clinical practice to improve patient care. Three invited speakers will lead the discussions in this symposium.Dr. Ke Li from the University of Wisconsin‐Madison will be presenting the new challenges introduced in model‐based iterative reconstruction (MBIR) method with a focus on how the nonlinear nature of the MBIR reconstruction poses challenges in quantitative image quality metrics such as spatial resolution, noise power spectrum, and the limitations of CNR in protocol optimization which eventually leads to the need of task‐based detectability index as the metric.Dr. Shuai Leng from Mayo Clinic will then present how the CNR metric is insufficient for nonlinear reconstruction algorithms, and how task based model observers can be used to more accurately quantify image quality, and to optimize imaging system and scanning protocols for nonlinear iterative image reconstruction algorithms based on specific imaging task. Practical considerations regarding the application of task based image quality metrics will also be discussed.Dr. Ehsan Samei from Duke University will start the symposium by summarizing a to‐be‐released AAPM Task Group report (TG 233: Performance Evaluation of Computed Tomography Systems). He will cover what has been understood and accepted for linear CT imaging system, what are the new challenges in CT image quality assessment introduced by the nonlinear iterative reconstruction (IR) techniques, and summarize both taskneutral and task‐based metrologies developed in TG 233 to partially address these challenges, and foresee additional challenges in the future. Learning Objectives: Understand the challenges in quantitative image quality assessment introduced by nonlinear iterative reconstruction techniques Understand the challenges and methods in noise performance assessment for nonlinear iterative reconstruction techniques Understand the challenges and methods in in spatial resolution assessment for nonlinear iterative reconstruction techniques Understand the challenges and methods in in task‐based observer performance assessment for nonlinear iterative reconstruction techniquesFunding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; S. Leng, R01 EB071095; U01 EB017185; E. Samei, Research grant, Siemens; Research grant, GE; K. Li, Funding from NIH, DOD and AAPM

Photoacoustic tomography (PAT) is an effective optical biomedical imaging method which is characterized with noninonizing and noninvasive, presenting good soft tissue contrast with excellent spatial resolution. To build a multi-dimensional breast PAT image, more ultrasound sensors are needed, which brings difficulties to data acquisition. The time complexity for multi-dimensional breast PAT image reconstruction also rises tremendously. Compressive sensing (CS) theory breaks the restriction of Nyquist sampling theorem and is capable to rebuild signals with fewer measurements. In this contribution, we propose an effective optimization method for multi-dimensional breast PAT, which combines the theory of CS and an unevenly, adaptively distributing data acquisition algorithm. With this method, the quality of our reconstructed breast PAT images are better than those using existing multi-dimensional breast PAT system. To build breast PAT images with the same quality, the required number of ultrasound transducers is decreased by using our proposed method. We have verified our method on simulation data and achieved expected results in both two dimensional and three dimensional PAT image reconstruction. In the future, our method can be applied to various aspects of biomedical PAT imaging such as early stage tumor detection and in vivo imaging monitoring.

By accurately modeling the physics of photon transport into the projection and backprojection operations, iterative SPECT reconstruction methods can reduce the degrading effects of scatter, attenuation and the non-stationary spatial resolution of the camera. Unfortunately, iterative reconstruction methods have required very long computation times, predominately due to the complexity involved in modeling these degrading effects into the projection and backprojection operations. In this study, we describe an approach which allows SPECT iterative reconstruction algorithms to be implemented with a reduction in the number of computations needed. The idea is to pre-process the measured projection data to compensate for scatter and attenuation, as well as to transform the projection data to those which would have been obtained with a stationary system resolution. Results of simulation studies indicate that preprocessing the measured projection data reduces the number of computations needed to perform the projection and backprojection operations, and yields reconstructions which differ minimally from those obtained using the slower standard iterative approach of modeling both photon attenuation and nonstationary blurring in the projection and backprojection steps.

BackgroundIterative computed tomography (CT) image reconstruction shows high potential for the preservation of image quality in diagnostic CT while reducing patients’ exposure; it has become available for low-dose CT (LD-CT) in high-end hybrid imaging systems (e.g. single-photon emission computed tomography [SPECT]-CT).PurposeTo examine the effect of an iterative CT reconstruction algorithm on image quality, image noise, detectability, and the reader’s confidence for LD-CT data by a subjective assessment.Material and MethodsThe LD-CT data were validated for 40 patients examined by an abdominal hybrid SPECT-CT (U = 120 kV, I = 40 mA, pitch = 1.375). LD-CT was reconstructed using either filtered back projection (FBP) or an iterative image reconstruction algorithm (Adaptive Statistical Iterative Reconstruction [ASIR]®) with different parameters (ASIR levels 50% and 100%). The data were validated by two independent blinded readers using a scoring system for image quality, image noise, detectability, and reader confidence, for a predefined set of 16 anatomic substructures.ResultsThe image quality was significantly improved by iterative reconstruction of the LD-CT data compared with FBP (P ≤ 0.0001). While detectability increased in only 2/16 structures (P ≤ 0.03), the reader’s confidence increased significantly due to iterative reconstruction (P ≤ 0.002). Meanwhile, at the ASIR level of 100%, the detectability in bone structure was highly reduced (P = 0.003).ConclusionAn ASIR level of 50% represents a good compromise in abdominal LD-CT image reconstruction. The specific ASIR level improved image quality (reduced image noise) and reader confidence, while preserving detectability of bone structure.

Photoacoustic imaging is a non-destructive method to obtain information about the distribution of optically absorbing structures inside a semitransparent medium. It is based on thermoelastic generation of ultrasonic waves by the absorption of a short laser pulse inside the sample. From the ultrasonic waves measured outside the object, the interior distribution of absorbed energy is reconstructed. The ultrasonic waves, which transport information from the interior to the surface of the sample, are scattered or absorbed to a certain extent by dissipative processes. The scope of this work is to quantify the information loss which is equal to the entropy production during these dissipative processes and thereby to give a principle limit for the spatial resolution which can be gained in photoacoustic imaging. This theoretical limit is compared to experimental data. In this book chapter stateof-the-art methods for modeling ultrasonic wave propagation in the case of attenuating media are described. From these models strategies for compensating ultrasound attenuation are derived which may be combined with well-known reconstruction algorithms from the non-attenuating case for photoacoustic imaging. Section 2 gives a short description of photoacoustic imaging, especially photoacoustic tomography, and the available image reconstruction algorithms to reconstruct the interior structure from the detected ultrasound signal at the sample surface. Beside small point-like detectors also large detectors, so called integrating detectors are used for photoacoustic tomography. The latter ones require different image reconstruction algorithms. Spatial resolution is an essential issue for any imaging method. Therefore we describe the influencing factors of the resolution in photoacoustic tomography. Section 3 is dedicated to acoustic attenuation. The spatial resolution in photoacoustic imaging is limited by the acoustic bandwidth. To resolve small objects shorter wavelengths with higher frequencies are necessary. For such high frequencies, however, the acoustic attenuation increases. This effect is usually ignored in photoacoustic image reconstruction but as small objects or structures generate high frequency components it limits the minimum detectable size, hence the resolution. Several models for acoustic attenuation, especially used for ultrasound propagation in biological tissue, are compared with experimental data.

Contrast-based photoacoustic imaging is an emerging imaging technology that gold nanoparticles (a kind of photoacoustic contrast agent with a size of 100 nm) are injected into blood vessels for photoacoustic imaging. Image reconstruction is a critical step in photoacoustic imaging, due to the fact that limitations and incompleteness of photoacoustic imaging data of image reconstruction are equivalent to the linear inverse problems. The photoacoustic data is transferred to IOT based on the platform of MATLAB and is quickly processed through the regularization method of Krylov space (the widely used method is LSQR). In order to solve the problem of Krylov space uncertainty, Krylov space dimension selection, and the minimum mismatch of the LSQR objective function (the limitation of LSQR algorithm) of LSQR algorithm, a KRLODPLSMR-GCV3DC algorithm is proposed to solve the above problems in this paper.In this study, the raw photoacoustic data was obtained by injecting gold nanoparticles into capillaries and subcutaneous veins of the palm, and the KRLODPLSMR-GCV3DC algorithm was verified to solve the limitations of LSQR algorithm through four groups of experiments.

Photoacoustic (PA) imaging technology is expected to be applied to clinical assessment for peripheral vascularity. We started a clinical evaluation with the prototype PA imaging system we recently developed. Prototype PA imaging system was composed with in-house Q-switched Alexandrite laser system which emits short-pulsed laser with 750 nm wavelength, handheld ultrasound transducer where illumination optics were integrated and signal processing for PA image reconstruction implemented in the clinical ultrasound (US) system. For the purpose of quantitative assessment of PA images, an image analyzing function has been developed and applied to clinical PA images. In this analyzing function, vascularity derived from PA signal intensity ranged for prescribed threshold was defined as a numerical index of vessel fulfillment and calculated for the prescribed region of interest (ROI). Skin surface was automatically detected by utilizing B-mode image acquired simultaneously with PA image. Skinsurface position is utilized to place the ROI objectively while avoiding unwanted signals such as artifacts which were imposed due to melanin pigment in the epidermal layer which absorbs laser emission and generates strong PA signals. Multiple images were available to support the scanned image set for 3D viewing. PA images for several fingers of patients with systemic sclerosis (SSc) were quantitatively assessed. Since the artifact region is trimmed off in PA images, the visibility of vessels with rather low PA signal intensity on the 3D projection image was enhanced and the reliability of the quantitative analysis was improved.