Deep learning in photoacoustic imaging: a review

Photoacoustic (PA) imaging technology is expected to be applied to clinical assessment for peripheral vascularity. We started a clinical evaluation with the prototype PA imaging system we recently developed. Prototype PA imaging system was composed with in-house Q-switched Alexandrite laser system which emits short-pulsed laser with 750 nm wavelength, handheld ultrasound transducer where illumination optics were integrated and signal processing for PA image reconstruction implemented in the clinical ultrasound (US) system. For the purpose of quantitative assessment of PA images, an image analyzing function has been developed and applied to clinical PA images. In this analyzing function, vascularity derived from PA signal intensity ranged for prescribed threshold was defined as a numerical index of vessel fulfillment and calculated for the prescribed region of interest (ROI). Skin surface was automatically detected by utilizing B-mode image acquired simultaneously with PA image. Skinsurface position is utilized to place the ROI objectively while avoiding unwanted signals such as artifacts which were imposed due to melanin pigment in the epidermal layer which absorbs laser emission and generates strong PA signals. Multiple images were available to support the scanned image set for 3D viewing. PA images for several fingers of patients with systemic sclerosis (SSc) were quantitatively assessed. Since the artifact region is trimmed off in PA images, the visibility of vessels with rather low PA signal intensity on the 3D projection image was enhanced and the reliability of the quantitative analysis was improved.

A three-dimensional modeling method for quantitative photoacoustic breast imaging with handheld probe

Compared to piezoelectric based photoacoustic (PA) scanners, the planar Fabry-Perot (FP) scanner has several advantages. It can provide small element size with high sensitivity, a smooth broadband frequency response, and is transparent to excitation light. This enables the FP scanner to provide excellent high-resolution in vivo PA images of soft tissue to depths up to approximately 10 mm. However, unlike piezoelectric scanners, the FP scanner in its current form cannot provide a pulse-echo ultrasound (US) as well as a PA image, which is useful because of the additional tissue contrast it provides. To address this, a dual mode FP scanner-based system that, for the first time, can acquire co-registered 3D PA and US images has been developed. In order to provide an optical US generation capability, the FP ultrasound sensor was coated with a novel Gold-Nanoparticle-PDMS composite which was excited with nanosecond laser pulses to generate plane wave US pulses. By modifying the FP sensor in this way, it now acts as an US transmitter as well as a receiver. The coating is highly absorbing at the US generation wavelength (>95%) but transparent at the PA excitation wavelength, the latter to allow the system to also operate in PA imaging mode as before. The generated US pulses exhibited peak pressures in the MPa range, which is comparable to the output of conventional piezoelectric based medical US scanners. The pulses had a broad bandwidth (>40 MHz) and the emitted wavefront was planar to within λ/10 at 10 MHz. PA and pulse-echo US signals were mapped in turn by the FP scanner over centimetre scale areas with a step size of 100 μm and an element size of 64 μm. The -3dB bandwidth of the FP sensor was 30 MHz. Reconstruction methods using a k-space formulation recovered co-registered 3D PA and US images. The system’s lateral spatial resolution was evaluated by imaging a line target at depths up to 10 mm and ranged between 50 and 120 μm for both modes. Arbitrarily shaped 3D objects were imaged to demonstrate the volumetric US imaging capability of the scanner. Tissue mimicking phantoms, with impedance mismatches representative of soft tissues, and ex vivo tissue samples were imaged with the system as well as a conventional clinical US scanner for comparison. Finally, the system obtained promising high-resolution 3D dual mode PA-US images for a variety of phantoms with contrast based on both optical absorption and acoustic impedance. This novel all-optical system has the potential to add complementary morphological contrast to photoacoustic vascular images which could aid the clinical assessment of superficial tumours, lymph node disease and other conditions.

Seeing through the Skin: Photoacoustic Tomography of Skin Vasculature and Beyond

Photoacoustic (PA) imaging holds great promise for preclinical research and clinical practice. However, most studies rely on the laser wavelength in the first near-infrared (NIR) window (NIR-I, 650-950 nm), while few studies have been exploited in the second NIR window (NIR-II, 1000-1700 nm), mainly due to the lack of NIR-II absorbing contrast agents. We herein report the synthesis of a broadband absorbing PA contrast agent based on semiconducting polymer nanoparticles (SPN-II) and apply it for PA imaging in NIR-II window. SPN-II can absorb in both NIR-I and NIR-II regions, providing the feasibility to directly compare PA imaging at 750 nm with that at 1064 nm. Because of the weaker background PA signals from biological tissues in NIR-II window, the signal-to-noise ratio (SNR) of SPN-II resulted PA images at 1064 nm can be 1.4-times higher than that at 750 nm when comparing at the imaging depth of 3 cm. The proof-of-concept application of NIR-II PA imaging is demonstrated in in vivo imaging of brain vasculature in living rats, which showed 1.5-times higher SNR as compared with NIR-I PA imaging. Our study not only introduces the first broadband absorbing organic contrast agent that is applicable for PA imaging in both NIR-I and NIR-II windows but also reveals the advantages of NIR-II over NIR-I in PA imaging.

Photoacoustic imaging is a new imaging technology in recent years, which combines the advantages of high resolution and rich contrast of optical imaging with the advantages of high penetration depth of acoustic imaging. Photoacoustic imaging has been widely used in biomedical fields, such as brain imaging, tumor detection and so on. The signal-tonoise ratio (SNR) of image signals in photoacoustic imaging is generally low due to the limitation of laser pulse energy, electromagnetic interference in the external environment and system noise. In order to solve the problem of low SNR of photoacoustic images, we use feedforward denoising convolutional neural network to further process the obtained images, so as to obtain higher SNR images and improve image quality. We use Python language to manage the referenced Python external library through Anaconda, and build a feedforward noise-reducing convolutional neural network on Pycharm platform. We first processed and segmated a training set containing 400 images, and then used it for network training. Finally, we tested it with a series of cerebrovascular photoacoustic microscopy images. The results show that the peak signal-to-noise ratio (PSNR) of the image increases significantly before and after denoising. The experimental results verify that the feed-forward noise reduction convolutional neural network can effectively improve the quality of photoacoustic microscopic images, which provides a good foundation for the subsequent biomedical research.

As a molecular imaging modality, photoacoustic (PA) imaging has been in the spotlight because it can provide an optical contrast image of physiological information and a relatively deep imaging depth. However, its sensitivity is limited despite the use of exogenous contrast agents due to the background PA signals generated from nontargeted absorbers, such as blood and boundaries between different biological tissues. In addition, clutter artifacts generated in both in-plane and out-of-plane imaging region degrade the sensitivity of PA imaging. We propose a method to eliminate the nontargeted PA signals. For this study, we used a dual-modal ultrasound (US)-PA contrast agent that is capable of generating both the backscattered US and PA signals in response to the transmitted US and irradiated light, respectively. The US images of the contrast agents are used to construct a masking image that contains the location information about the target site and is applied to the PA image acquired after contrast agent injection. In vitro and in vivo experimental results demonstrated that the masking image constructed using the US images makes it possible to completely remove nontargeted PA signals. The proposed method can be used to enhance the clear visualization of the target area in PA images.

Chapter 13 - Photoacoustic Molecular Imaging: Principles and Practice

Photoacoustic (PA) imaging is an emerging technology that could map the functional contrasts in deep biological tissues in high resolution by “listening” to the laser induced thermoelastic waves. Almost all of the current studies in PA imaging are focused on the intensity of the PA signals as an indication of the optical absorbance of the biological tissues. Our group has for the first time demonstrated that the frequency domain power distribution of the broadband PA signals encode the texture information within the regions-of-interest (ROI). Following the similar method of ultrasound spectral analysis (USSA), photoacoustic spectrum analysis (PASA) could evaluate the relative concentrations and, more importantly, the dimensions of microstructures of the optically absorbing materials in biological tissues, including lipid, collagen, water and hemoglobin. By providing valuable insights into tissue pathology, PASA should benefit basic research and clinical management of many diseases, and may help achieve eventual “noninvasive biopsy”. In this work, taking advantage of the optical absorption contrasts contributed by lipid and hemoglobin at 1200-nm and 532-nm wavelengths respectively, we investigated the capability of PASA in identifying histological changes corresponding to fat accumulation livers through the study on ex vivo and in situ mouse models. The PA signals from the mouse livers were acquired using our PA and US dual-modality imaging system, and analyzed in the frequency domain. After quantifying the power spectrum by fitting it to a first order model, three spectral parameters, including the intercept, the midband fit and the slope, were extracted and used to differentiate fatty livers from normal livers. The comparison between the PASA parameters from the normal and the fatty livers supports our hypotheses that PASA can quantitatively identify the microstructure changes in liver tissues for differentiating normal and fatty livers.

Brachytherapy seed therapy is an increasingly common way to treat prostate cancer through localized radiation. The current standard of care relies on transrectal ultrasound (TRUS) for imaging guidance during the seed placement procedure. As visualization of individual metallic seeds tends to be difficult or inaccurate under TRUS guidance, guide needles are generally tracked to infer seed placement. In an effort to improve seed visualization and placement accuracy, the use of photoacoustic (PA) imaging, which is highly sensitive to metallic objects in soft tissue, was investigated for this clinical application. The PA imaging properties of bare (i.e., embedded in pure gelatin) and tissue-embedded (at depths of up to 13 mm) seeds were investigated with a multi-wavelength (750 to 1090 nm) PA imaging technique. Results indicate that, much like ultrasonic (US) imaging, an angular dependence (i.e., seed orientation relative to imaging transducer) of the PA signal exists. Despite this shortcoming, however, PA imaging offers improved contrast, over US imaging, of a seed in prostate tissue if sufficient local fluence is achieved. Additionally, although the PA signal of a bare seed is greatest for lower laser wavelengths (e.g., 750 nm), the scattering that results from tissue tends to favor the use of higher wavelengths (e.g., 1064 nm, which is the primary wavelength of Nd:YAG lasers) when the seed is located in tissue. A combined PA and US imaging approach (i.e., PAUS imaging) shows strong potential to visualize both the seed and the surrounding anatomical environment of the prostate during brachytherapy seed placement procedures.

Introduction: There is an urgent clinical need to develop noninvasive biomarkers that early detect tumor responses to anti-angiogenic therapy. Treatment response in ovarian cancer patients is assessed with transvaginal ultrasound imaging of tumor size and CA125 screening. We hypothesized that combining photoacoustic (PA) imaging to non-invasively visualize tumor vascular architecture, with serum diagnostics using endothelial biomarkers, would yield a sensitive and readily translatable approach for monitoring response to anti-angiogenic therapy. Methods: Ovarian tumors were established in nude mice by orthotopic injection of OV2008 cells at day 0. At day 10 mice were randomized into groups of at least n = 10. Baseline PA imaging and submandibular blood draw were performed, then repeated 24h after dosing with a peptibody that prevents the interaction of secreted angiopoietin 1 (Ang1; agonist) and 2 (Ang2; antagonist) with the receptor tyrosine kinase Tie2, or vehicle, on days 13, 16 and 19. Mice were then sacrificed and tumors excised for histology. Results: PA imaging uses endogenous hemoglobin light absorption to generate contrast. At 797nm, we found a 1.5-fold increase in hemoglobin-weighted signal in vehicle mice by day 19, while treated mice remained close to the baseline (n=9; p=0.008). The oxyhemoglobin-weighted PA signal (837nm/797nm) was elevated by 20% in treated mice but decreased 15% in vehicle mice (n=9; p<0.01). This indicated treated mice had significantly lower tumor vessel density but improved function of residual vessels. To gain insight into the mechanism of drug action, we measured host serum angiopoietins over the treatment time course. Ang1 in vehicle mice increased by 1.5-fold (n=10; p<0.0001) and was significantly higher than treated mice by day 19 (n=10; p=0.03). Ang2 was identical between treatment groups and healthy control mice (n=10; p=0.65). Elevated Ang1 in the circulation of vehicle mice is consistent with excess Ang1 promoting vascular remodeling and plasticity in tumors, while normalization in treated mice allows vessel stabilization. Immunofluorescence (IF) confirmed this by indicating that CD31 positive vessel density was 30% lower in treated mice but their tumor vessel coverage of desmin-positive pericytes was 2.5-fold higher. Conclusions: We have shown for the first time that PA imaging and serum angiopoietin levels measure complementary parameters that indicate response to angiopoietin blockade in ovarian cancer. PA imaging noninvasively detected drug-induced vessel regression and normalization, giving a functional readout of response. This modality can be readily combined with transvaginal ultrasound. The drug action was mediated by normalization of mAng1 levels, indicated by serum measures and IF. Importantly, this work supports the combined use of in vitro and in vivo diagnostics to monitor response to anti-angiogenic therapy in clinical trials. Citation Format: Sarah E. Bohndiek, Laura Sasportas, Steven Machtaler, Jesse V. Jokerst, Sharon Hori, Sanjiv S. Gambhir. Molecular photoacoustic imaging and serum diagnostics rapidly detect response to angiopoietin 1 and 2 blockade in ovarian cancer. [abstract]. In: Proceedings of the 105th Annual Meeting of the American Association for Cancer Research; 2014 Apr 5-9; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2014;74(19 Suppl):Abstract nr 2047. doi:10.1158/1538-7445.AM2014-2047

<p>The tumour vasculature supplies the tumour with oxygen and nutrition that are vital for the survival and growth of cancer cells. Vascular disrupting agents (VDAs) are a category of cancer therapeutic drugs that target the endothelial cells lining the vessels and damage tumour vasculature. Photoacoustic (PA) imaging has high sensitivity in detecting vascular changes due to the high optical absorption of hemoglobin. PA quantitative analysis can be used to further extract structural and biochemical changes of hemoglobin through ultrasound spectral analysis and multi-spectral PA imaging. I proposed the use of simulations, in-vitro, and in-vivo PA imaging to assess the efficacy of VDAs in cancer therapy.</p> <p>PA signals acquired from single and a collection of red blood cells (RBCs) were examined before and after acid sphingomyelinase exposure (SMase is a signaling molecule excreted due to endothelial cells damage) to investigate changes at the cellular level. Bleeding of tumour vasculature was simulated using a fractal-based model of bifurcating cylinders and the diffusion of blood to the tumour interstitium. Wavelength selection and fluence matching approaches were proposed to improve chromophore quantification in real time for PA imaging. In-vivo experiments were conducted to examine how PA imaging can be used to monitor the effect of the VDA 5,6-dimethylxanthenone-4-acetic acid (DMXAA). PA ultrasound frequency analysis and multi-spectral PA imaging were used to detect structural and biochemical changes to the tumour vasculature.</p> <p>PA quantitative analysis of the in-vivo data demonstrates a decrease in the total hemoglobin 72 hrs post DMXAA injection and an increase in Hb 24 hrs post DMXAA injection. The changes in the chromophores’ composition are due to changes in the environment as the RBCs extravasate to the surrounding tissues as seen from the in-vitro experiments. A decrease in the average spectral slope at 24 hrs post injection was measured. The decrease in the spectral slope was correlated to an increase in the effective vessel size due to bleeding, which was also supported by the vascular tree simulations. The combination of simulations, in-vitro and in-vivo studies demonstrate the capability of PA signal analysis in monitoring VDA at early time points.</p>

The lateral imaging characteristic of an acoustic lens is similar to the well-known object-image relationship of an optical lens. In this paper, axial signal analysis is carried on and reconstruction methods for a sample's axial photoacoustic (PA) image are researched based on the acoustic-lens PA imaging system. The integral and envelope methods are both proposed to reconstruct a sample's axial PA image, which has only been achieved approximately by the PA signals' peak values obtained by a peak-holding circuit in previous reports. And the envelope method is proposed for the first time. Hilbert transform was used to obtain the envelope of the sample's PA signals and axial PA images. Simulation results show that the integral method is appropriate for low frequency samples, and that the envelope reconstruction method can provide low frequency information of the PA images of samples. So for a sample, it would be best to combine the two methods together.

An inexpensive noncontact photoacoustic (PA) imaging system using a low-power continuous wave laser and a kilohertz-range microphone has been developed. The system operates in both optical and PA imaging modes and is designed to be compatible with conventional optical microscopes. Aqueous coupling fluids are not required for the detection of the PA signals; air is used as the coupling medium. The main component of the PA system is a custom designed PA imaging sensor that consists of an air-filled sample chamber and a resonator chamber that isolates a standard kilohertz frequency microphone from the input laser. A sample to be examined is placed on the glass substrate inside the chamber. A laser focused to a small spot by a 40 × objective onto the substrate enables generation of PA signals from the sample. Raster scanning the laser over the sample with micrometer-sized steps enables high-resolution PA images to be generated. A lateral resolution of 1.37 ?? ? m was achieved in this proof of concept study, which can be further improved using a higher numerical aperture objective. The application of the system was investigated on a red blood cell, with a noise-equivalent detection sensitivity of 43,887 hemoglobin molecules ( 72.88 × 10 ? 21 ?? mol or 72.88 zeptomol). The minimum pressure detectable limit of the system was 19.1 ?? ? Pa . This inexpensive, compact noncontact PA sensor is easily integrated with existing commercial optical microscopes, enabling optical and PA imaging of the same sample. Applications include forensic measurements, blood coagulation tests, and monitoring the penetration of drugs into human membrane.

Understanding the tumor microenvironment is critical to characterizing how cancers operate and predicting their response to treatment. We describe a novel, high-resolution coregistered photoacoustic (PA) and pulse echo (PE) ultrasound system used to image the tumor microenvironment. Compared to traditional optical systems, the platform provides complementary contrast and important depth information. Three mice are implanted with a dorsal skin flap window chamber and injected with PC-3 prostate tumor cells transfected with green fluorescent protein. The ensuing tumor invasion is mapped during three weeks or more using simultaneous PA and PE imaging at 25 MHz, combined with optical and fluorescent techniques. Pulse echo imaging provides details of tumor structure and the surrounding environment with 100-μm(3) resolution. Tumor size increases dramatically with an average volumetric growth rate of 5.35 mm(3)/day, correlating well with 2-D fluorescent imaging (R = 0.97, p < 0.01). Photoacoustic imaging is able to track the underlying vascular network and identify hemorrhaging, while PA spectroscopy helps classify blood vessels according to their optical absorption spectrum, suggesting variation in blood oxygen saturation. Photoacoustic and PE imaging are safe, translational modalities that provide enhanced depth resolution and complementary contrast to track the tumor microenvironment, evaluate new cancer therapies, and develop molecular contrast agents in vivo.