Masked cross-domain self-supervised deep learning framework for photoacoustic computed tomography reconstruction

Monitoring critical physiological processes in murine models like tumor vascular perfusion and its response to prospective anti-cancer treatments is a significant potential use case of dynamic photoacoustic computed tomography (PACT). Previously reported studies of dynamic PACT are based on a frame-by-frame image reconstruction (FBFIR) procedure in which full-view measurement data are assumed to be rapidly acquired. However, many commercial three-dimensional PACT imagers acquire measurements at each tomographic view rotating the object in discrete steps. The time to collect the full-view data is limited by the rotation speed of the object holder and the laser repetition rate. Therefore, FBFIR techniques are not applicable, and there is a critical need for accurate and efficient spatiotemporal image reconstruction (STIR) techniques that can account for spatiotemporal redundancies in the object’s features. To address this, we propose a low-rank matrix estimation-based STIR technique in which the sought-after dynamic image is approximated using a semiseparable approximation in space and time. To validate the proposed method, we also develop a virtual imaging framework for PACT employing dynamic numerical mouse phantoms with a physiologically realistic respiratory motion and perfusion model. The studies demonstrated that the proposed method accurately recovers the tumor vascular perfusion and object motion.

Conventional photoacoustic computed tomography (PACT) image reconstruction methods assume that the object and surrounding medium are described by a constant speed-of-sound (SOS) value. In order to accurately recover fine structures, SOS heterogeneities should be quantified and compensated for during PACT reconstruction. To address this problem, several groups have proposed hybrid systems that combine PACT with ultrasound computed tomography (USCT). In such systems, a SOS map is reconstructed first via USCT. Consequently, this SOS map is employed to inform the PACT reconstruction method. Additionally, the SOS map can provide structural information regarding tissue, which is complementary to the functional information from the PACT image. We propose a paradigm shift in the way that images are reconstructed in hybrid PACT-USCT imaging. Inspired by our observation that information about the SOS distribution is encoded in PACT measurements, we propose to jointly reconstruct the absorbed optical energy density and SOS distributions from a combined set of USCT and PACT measurements, thereby reducing the two reconstruction problems into one. This innovative approach has several advantages over conventional approaches in which PACT and USCT images are reconstructed independently: (1) Variations in the SOS will automatically be accounted for, optimizing PACT image quality; (2) The reconstructed PACT and USCT images will possess minimal systematic artifacts because errors in the imaging models will be optimally balanced during the joint reconstruction; (3) Due to the exploitation of information regarding the SOS distribution in the full-view PACT data, our approach will permit high-resolution reconstruction of the SOS distribution from sparse array data.

Photoacoustic computed tomography (PACT) is a promising imaging modality that combines the advantages of optical contrast with ultrasound detection. Utilizing ultrasound transducers with larger surface areas can improve detection sensitivity. However, when computationally efficient analytic reconstruction methods that neglect the spatial impulse responses (SIRs) of the transducer are employed, the spatial resolution of the reconstructed images will be compromised. Although optimization-based reconstruction methods can explicitly account for SIR effects, their computational cost is generally high, particularly in three-dimensional (3D) applications. To address the need for accurate but rapid 3D PACT image reconstruction, this study presents a framework for establishing a learned SIR compensation method that operates in the data domain. The learned compensation method maps SIR-corrupted PACT measurement data to compensated data that would have been recorded by idealized point-like transducers. Subsequently, the compensated data can be used with a computationally efficient reconstruction method that neglects SIR effects. Two variants of the learned compensation model are investigated that employ a U-Net model and a specifically designed, physics-inspired model, referred to as Deconv-Net. A fast and analytical training data generation procedure is also a component of the presented framework. The framework is rigorously validated in virtual imaging studies, demonstrating resolution improvement and robustness to noise variations, object complexity, and sound speed heterogeneity. When applied to in-vivo breast imaging data, the learned compensation models revealed fine structures that had been obscured by SIR-induced artifacts. To our knowledge, this is the first demonstration of learned SIR compensation in 3D PACT imaging.

Transcranial photoacoustic computed tomography (PACT) faces challenges due to skull-induced aberrations in measured pressure data. Although optimization-based image reconstruction methods have been developed to mitigate these aberrations, they require precise knowledge of skull properties that are generally unknown. Parameterized joint reconstruction (JR) methods improve computational tractability by estimating simplified skull models but only partially capture skull heterogeneity, potentially compromising modeling accuracy. Therefore, there remains a need for a reconstruction method that accounts for uncertainties in a fully heterogeneous skull model to enable accurate PACT image reconstruction in humans. In this study, a JR method for 3D transcranial PACT is developed to concurrently estimate the initial pressure distribution and CT-informed heterogeneous viscoelastic properties of the skull. The proposed approach leverages adjunct CT data and a linear relationship between CT voxel values and acoustic properties to parameterize a fully heterogeneous skull model. A gradientfree joint reconstruction method is used to estimate subject-dependent bone property parameters concurrently with the 3D initial pressure distribution. The method was systematically evaluated using realistic, full-scale 3D numerical head phantoms through in-silico studies. Results demonstrate that incorporating CT-informed heterogeneous skull modeling significantly improved image quality compared to model-based methods that assume a homogeneous skull model or a heterogeneous skull model with nominal parameter values. These findings highlight the importance of accurate skull modeling for robust transcranial PACT imaging.

Photoacoustic computed tomography (PACT) is a promising non-invasive imaging technique for both life science and clinical implementations. To achieve fast imaging speed, modern PACT systems have equipped arrays that have hundreds to thousands of ultrasound transducer (UST) elements, and the element number continues to increase. However, large number of UST elements with parallel data acquisition could generate a massive data size, making it very challenging to realize fast image reconstruction. Although several research groups have developed GPU-accelerated method for PACT, there lacks an explicit and feasible step-by-step description of GPU-based algorithms for various hardware platforms. In this study, we propose a comprehensive framework for developing GPU-accelerated PACT image reconstruction (GPU-accelerated photoacoustic computed tomography), to help the research community to grasp this advanced image reconstruction method. We leverage widely accessible open-source parallel computing tools, including Python multiprocessing-based parallelism, Taichi Lang for Python, CUDA, and possible other backends. We demonstrate that our framework promotes significant performance of PACT reconstruction, enabling faster analysis and real-time applications. Besides, we also described how to realize parallel computing on various hardware configurations, including multicore CPU, single GPU, and multiple GPUs platform. Notably, our framework can achieve an effective rate of times when reconstructing extremely large-scale three-dimensional PACT images on a dual-GPU platform compared to a 24-core workstation CPU. In this paper, we share example codes via GitHub. Our approach allows for easy adoption and adaptation by the research community, fostering implementations of PACT for both life science and medicine.

Photoacoustic computed tomography (PACT) is an advanced biomedical imaging modality that integrates optical and ultrasound imaging techniques. A central goal in PACT is to estimate the initial pressure rise from raw photoacoustic data (sinogram), a task complicated by the need for accurate inverse modeling, sensitivity to noise, and hardware limitations. Conventional reconstruction methods typically rely on multi-step processing and require careful parameter tuning to achieve high-quality results. While deep learning has proven effective in addressing challenges like noise suppression and computational efficiency, its use in PACT has largely been limited to image-domain or sensor data (sinogram)-domain enhancement, rather than direct image reconstruction from sensor data (sinograms) - a direction more extensively explored in computed tomography (CT) and magnetic resonance imaging (MRI). In this study, we propose a novel, data-driven methodology that formulates PACT image reconstruction as a supervised learning problem. Our approach employs a hybrid fully connected network followed by a Swin-UNETR to learn a direct mapping from raw sinograms (sensor data) to reconstruct images, effectively eliminating the need for conventional, handcrafted reconstruction steps. The proposed method achieved high image quality, with peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of 33.878 dB and 0.975, respectively.

Image segmentation is important in improving the diagnostic capability of ultrasound computed tomography (USCT) and photoacoustic computed tomography (PACT), as it can be included in the image reconstruction process to improve image quality and quantification abilities. Segmenting the imaged object out of the background using image domain methods is easily complicated by low contrast, noise, and artifacts in the reconstructed image. Here, we introduce a new signal domain object segmentation method for USCT and PACT which does not require image reconstruction beforehand and is automatic, robust, computationally efficient, accurate, and straightforward. We first establish the relationship between the time-of-flight of the received first arrival waves and the object's boundary which is described by ellipse equations. Then, we show that the ellipses are tangent to the boundary. By looking for tangent points on the common tangent of neighboring ellipses, the boundary can be approximated with high fidelity. Imaging experiments of human fingers and mice cross-sections showed that our method provided equivalent or better segmentations than the optimal ones by active contours. In summary, our method greatly reduces the overall complexity of object segmentation and shows great potential in eliminating user dependency without sacrificing segmentation accuracy. The method can be further seamlessly incorporated into algorithms for other processing purposes in USCT and PACT, such as high-quality image reconstruction.

Photoacoustic computed tomography (PACT) has emerged as a unique and promising technology for multiscale biomedical imaging. To fully realize its potential for various preclinical and clinical applications, development of systems with high imaging speed, reasonable cost, and manageable data flow are needed. Sparse-sampling PACT with advanced reconstruction algorithms, such as compressed-sensing reconstruction, has shown potential as a solution to this challenge. However, most such algorithms require iterative reconstruction and thus intense computation, which may lead to excessively long image reconstruction times. Here, we developed a principal component analysis (PCA)-based PACT (PCA-PACT) that can rapidly reconstruct high-quality, three-dimensional (3-D) PACT images with sparsely sampled data without requiring an iterative process. In vivo images of the vasculature of a human hand were obtained, thus validating the PCA-PACT method. The results showed that, compared with the back-projection (BP) method, PCA-PACT required ∼50% fewer measurements and ∼40% less time for image reconstruction, and the imaging quality was almost the same as that for BP with full sampling. In addition, compared with compressed sensing-based PACT, PCA-PACT had approximately sevenfold faster imaging speed with higher imaging accuracy. This work suggests a promising approach for low-cost, 3-D, rapid PACT for various biomedical applications.

Due to their low cost, hand-held convenience, wide selection of bandwidths, and ultrasound imaging capability, linear ultrasonic transducer arrays have been widely studied for photoacoustic computed tomography (PACT). As linear-array PACT suffers from a limited view, full-view imaging requires either the transducer or the object to be rotated. So far, both the central frequencies and bandwidth of linear transducer arrays applied in full-view PACT are low, limiting the spatial resolutions of the reconstructed images. Here, we present a multiview high-frequency PACT imaging system implemented with a commercial 40-MHz central frequency linear transducer array. By rotating the object through multiple angles with respect to the linear transducer array, we acquired full-view photoacoustic pressure measurements. Further, to quantify the unipolar initial pressures and overcome the limitations of the single-view Hilbert transformation, we developed a multiview Hilbert transformation method. The in-plane spatial resolution of this full-view linear-array PACT was quantified to be isotropically 60 μm within a 10×10 mm² field of view. The system was demonstrated by imaging both a leaf skeleton and a zebrafish in vivo.

In photoacoustic computed tomography (PACT) with short-pulsed laser excitation, wideband acoustic signals are generated in biological tissues with frequencies related to the effective shapes and sizes of the optically absorbing targets. Low-frequency photoacoustic signal components correspond to slowly varying spatial features and are often omitted during imaging due to the limited detection bandwidth of the ultrasound transducer, or during image reconstruction as undesired background that degrades image contrast. Here we demonstrate that low-frequency photoacoustic signals, in fact, contain functional and molecular information, and can be used to enhance structural visibility, improve quantitative accuracy, and reduce spare-sampling artifacts. We provide an in-depth theoretical analysis of low-frequency signals in PACT, and experimentally evaluate their impact on several representative PACT applications, such as mapping temperature in photothermal treatment, measuring blood oxygenation in a hypoxia challenge, and detecting photoswitchable molecular probes in deep organs. Our results strongly suggest that low-frequency signals are important for functional and molecular PACT.

Deep Learning for scaling large-aperture photoacoustic computed tomography : From single fingers to the human hand.

Photoacoustic computed tomography (PACT) holds great promise for transcranial brain imaging. However, the strong reflection, scattering, and attenuation of acoustic waves by the skull present significant challenges for image reconstruction. In this talk, we will review our recent progress on transcranial PACT image reconstruction. Our contributions include the following: (1) development of a methodology to establish a discrete transcranial PACT image model by use of adjunct X-ray CT data; (2) development of image reconstruction methods that can compensate for speed-of-sound and density variations within the skull; and (3) a detailed investigation of the role of shear waves in transcranial PACT. Computer-simulated and experimental data are employed to demonstrate the feasibility of transcranial PACT.

This paper proposes a novel self-supervised learning framework that uses model reinforcement, REference-free LAtent map eXtraction with MOdel REinforcement (RELAX-MORE), for accelerated quantitative MRI (qMRI) reconstruction. The proposed method uses an optimization algorithm to unroll an iterative model-based qMRI reconstruction into a deep learning framework, enabling accelerated MR parameter maps that are highly accurate and robust. Unlike conventional deep learning methods which require large amounts of training data, RELAX-MORE is a subject-specific method that can be trained on single-subject data through self-supervised learning, making it accessible and practically applicable to many qMRI studies. Using quantitative mapping as an example, the proposed method was applied to the brain, knee and phantom data. The proposed method generates high-quality MR parameter maps that correct for image artifacts, removes noise, and recovers image features in regions of imperfect image conditions. Compared with other state-of-the-art conventional and deep learning methods, RELAX-MORE significantly improves efficiency, accuracy, robustness, and generalizability for rapid MR parameter mapping. This work demonstrates the feasibility of a new self-supervised learning method for rapid MR parameter mapping, that is readily adaptable to the clinical translation of qMRI.

The influences of finite aperture size in photoacoustic computed tomography

Based on the photoacoustic (PA) effect, PA tomography directly measures specific optical absorption, i.e., absorbed optical energy per unit volume. We recently developed a full-ring ultrasonic transducer array-based photoacoustic computed tomography (PACT) system for small-animal whole-body imaging. The system has a full-view detection angle and high in-plane resolution (∼100 μm). However, due to the bandpass frequency response of the piezoelectric transducer elements and the limited elevational detection coverage of the full-ring transducer array, the reconstructed images present bipolar (i.e., both positive and negative) pixel values, which cause ambiguities in image interpretation for physicians and biologists. We propose a multiview Hilbert transformation method to recover the unipolar initial pressure for full-ring PACT. The effectiveness of the proposed algorithm was first validated by numerical simulations and then demonstrated with ex vivo mouse brain structural imaging and in vivo mouse whole-body imaging.