UPAMNet: A unified network with deep knowledge priors for photoacoustic microscopy

Mosaic imaging is a computer vision process that is used for merging multiple overlapping imaging patches into a wide-field-of-view image. To achieve a wide-field-of-view photoacoustic microscopy (PAM) image, the limitations of the scan range of PAM require a merging process, such as marking the location of patches or merging overlapping areas between adjacent images. By using the mosaic imaging process, PAM shows a larger field view of targets and preserves the quality of the spatial resolution. As an essential process in mosaic imaging, various feature generation methods have been used to estimate pairs of image locations. In this study, various feature generation algorithms were applied and analyzed using a high-resolution mouse ear PAM image dataset to achieve and optimize a mosaic imaging process for wide-field PAM imaging. We compared the performance of traditional and deep learning feature generation algorithms by estimating the processing time, the number of matches, good matching ratio, and matching efficiency. The analytic results indicate the successful implementation of wide-field PAM images, realized by applying suitable methods to the mosaic PAM imaging process.

Photoacoustic microscopy (PAM) enables structural and functional imaging at a microscopic level, playing a vital role in biomedical imaging. However, due to the limitations imposed by its scanning mechanism, PAM faces a trade‐off between spatial resolution and imaging speed. While deep learning has improved PAM imaging speed, the image reconstruction performance of existing methods under high downsampling ratios still needs improvement. To address these limitations, we propose UPAM‐KAN for undersampled PAM image reconstruction. This model is based on U‐Net and integrates dedicated Kolmogorov–Arnold Network layers into the tokenized intermediate representations to construct its backbone network. Furthermore, three feature fusion extraction modules are proposed to enhance details, extract multi‐scale features, and fuse shallow and deep features, respectively. Compared with the leading methods in undersampled PAM image reconstruction, UPAM‐KAN achieves significant SSIM improvements of 8.322% and 2.692% on the Leaf Vein and Mouse Cerebrovascular datasets with only 1.4% of fully sampled pixels, respectively. Moreover, for functional reconstruction, the model pre‐trained on public datasets achieves a 3.275% SSIM improvement in oxygen saturation at the 1/4 undersampling ratio. These results demonstrate that UPAM‐KAN efficiently reconstructs both structural and functional information, offering insights for high‐speed PAM imaging, dynamic vascular imaging and tissue functional monitoring.

Choroidal neovascularization (CNV) is a hallmark of advanced age-related macular degeneration and is a leading cause of irreversible vision loss and blindness in the world. There is a lack of imaging modalities that can detect CNV early. Photoacoustic microscopy (PAM) combines acoustic and optical imaging to create high-resolution images that can delineate microvasculature non-invasively and permit for visualization of CNV. However, to distinguish CNV from native microvasculature, molecular contrast agents can be applied to increase the sensitivity of PAM imaging. In this study, functionalized indocyanine green (ICG) with RGD ligands (ICG-RGD) was synthesized and applied to in vivo imaging of CNV in the rabbit retina using a custom-built high-resolution PAM imaging system. RGD ligands selective binds to integrins present in neovascularization. The CNV model was created in three New Zealand White rabbits via subretinal injection of human vascular endothelial grow factor (VEGF-165). Three-dimensional in vivo PAM images of CNV were acquired before and after intravenous injection of 400 µL ICG-RGD at a concentration of 2.5 mg/mL over a period of 14 days. In addition, color fundus photography and fluorescence imaging were also acquired. This enables visualization of the newly developed CNV as well as monitoring the changes in CNV over time. With the administration of ICG-RGD, PAM signal amplitudes enhanced up to 15.7-fold at day 1 post-injection and persisted up to 5 days. The PAM results were also confirmed by fluorescence imaging. Biosafety studies show no systemic toxicity of the ICG-RGD. These studies demonstrate the potential of molecular PAM imaging for the characterization of microvasculature pathologies.

Retinal safety evaluation of photoacoustic microscopy

Detection of metastasis of mediastinal lymph nodes in lung cancer patients with an artificial intelligence model.

Visualizing ocular vasculature is important in clinical ophthalmology because ocular circulation abnormalities are early signs of ocular diseases. Photoacoustic microscopy (PAM) images the ocular vasculature without using exogenous contrast agents, avoiding associated side effects. Moreover, 3D PAM images can be useful in understanding vessel-related eye disease. However, the complex structure of the multi-layered vessels still present challenges in evaluating ocular vasculature. In this study, we demonstrate a new method to evaluate blood circulation in the eye by combining in vivo PAM imaging and an ocular surface estimation method based on a machine learning algorithm: a random sample consensus algorithm. By using the developed estimation method, we were able to visualize the PA ocular vascular image intuitively and demonstrate layer-by-layer analysis of injured ocular vasculature. We believe that our method can provide more accurate evaluations of the eye circulation in ophthalmic applications.

Image denoising faces significant challenges, arising from the sources of noise. Specifically, Gaussian, impulse, salt, pepper, and speckle noise are complicated sources of noise in imaging. Convolutional neural network (CNN) has increasingly received attention in image denoising task. Several CNN methods for denoising images have been studied. These methods used different datasets for evaluation. In this paper, we offer an elaborate study on different CNN techniques used in image denoising. Different CNN methods for image denoising were categorized and analyzed. Popular datasets used for evaluating CNN image denoising methods were investigated. Several CNN image denoising papers were selected for review and analysis. Motivations and principles of CNN methods were outlined. Some state-of-the-arts CNN image denoising methods were depicted in graphical forms, while other methods were elaborately explained. We proposed a review of image denoising with CNN. Previous and recent papers on image denoising with CNN were selected. Potential challenges and directions for future research were equally fully explicated.

OBJECTIVERoughly 6.5 million diabetic patients in the United States suffer from chronic wounds that exhibit impaired healing response1. These wounds often do not respond to standard treatment due to insufficient oxygenation in the wound bed2. Given the heterogeneity of oxygen levels in wounds and the importance of oxygen in tissue repair, it is compelling to develop a reliable way to non‐invasively monitor oxygen levels within chronic wounds over time. Previously, we demonstrated the ability of topically‐delivered BF2nbm(I)PLA boron dye nanoparticles (BNPs) that phosphoresce differentially at varying oxygen levels, to provide a spatial map of oxygen levels throughout the wound bed of a murine full‐thickness cutaneous wound3. The objective of this work is to validate the ability of BNP imaging to detect impaired tissue oxygenation during wound healing in diabetic individuals compared to healthy individuals.METHODSWe have developed a novel model of wound healing in the murine ear which allows measurements of oxygen levels in both tissue and blood vessels using BNP imaging and photoacoustic microscopy (PAM), respectively. PAM non‐invasively measures structural and hemodynamic changes, including vessel diameter, hemoglobin concentration, blood flow speed, and hemoglobin oxygen saturation levels in individual blood vessels down to the capillary level during microvascular remodeling within wound beds4. In this ear wound model, the epidermis and a portion of the dermis, including the center feeder arteriole and its main branches, are excised from the center of the ear lobe. The wound is covered with Tegaderm bandage to prevent drying, and the mouse is fitted with an Elizabethan collar to prevent it from removing the bandage. At regular intervals until the wound heals, BNPs are topically applied to the wound and excited by UV light for two minutes during which the fluorescence (F) and phosphorescence (P) emission signals are recorded using a camera. A custom Matlab program calculates the ratio of F:P emission intensities at each pixel in the wound bed image, which provides a relative mapping of oxygen levels throughout the wound. Following BNP imaging, PAM imaging is performed in order to calculate the percent sO2 in each blood vessel in/around the wound bed. BNP and PAM measurements are compared to one another for wounds in both healthy and diabetic mice.RESULTSDiabetic db/db mice exhibit muted wound oxygenation increases over time compared to non‐diabetic control mice. BNP fluorescence:phosphorescence emission ratios from wound beds emulate sO2 trends generated via PAM, confirming the ability of BNPs to assess tissue oxygenation over time in both diabetic and healthy wounds.CONCLUSIONSBNPs can be used to monitor tissue oxygenation over time in diabetic and healthy wounds. Given their low cost and potential for clinical monitoring of these wounds in humans, BNPs warrant further study for non‐invasively monitoring wound healing over time.Support or Funding InformationUVA School of Medicine Pinn Scholars ProgramThis abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

Non-human primates (NHPs) are crucial models for studies of neuronal activity. Emerging photoacoustic imaging modalities offer excellent tools for studying NHP brains with high sensitivity and high spatial resolution. In this research, a photoacoustic microscopy (PAM) device was used to provide a label-free quantitative characterization of cerebral hemodynamic changes due to peripheral mechanical stimulation. A 5 × 5 mm area within the somatosensory cortex region of an adult squirrel monkey was imaged. A deep, fully connected neural network was characterized and applied to the PAM images of the cortex to enhance the vessel structures after mechanical stimulation on the forelimb digits. The quality of the PAM images was improved significantly with a neural network while preserving the hemodynamic responses. The functional responses to the mechanical stimulation were characterized based on the improved PAM images. This study demonstrates capability of PAM combined with machine learning for functional imaging of the NHP brain.

Image denoising and super-resolution play vital roles in imaging systems, greatly reducing the preprocessing cost of many AI techniques for object detection, segmentation, and tracking. Various advancements have been accomplished in this field, but progress is still needed. In this paper, we have proposed a novel technique named the Enhanced Learning Enriched Features (ELEF) mechanism using a deep convolutional neural network, which makes significant improvements to existing techniques. ELEF consists of two major processes: (1) Denoising, which removes the noise from images; and (2) Super-resolution, which improves the clarity and details of images. Features are learned through deep CNN and not through traditional algorithms so that we can better refine and enhance images. To effectively capture features, the network architecture adopted Dual Attention Units (DUs), which align with the Multi-Scale Residual Block (MSRB) for robust feature extraction, working sidewise with the feature-matching Selective Kernel Extraction (SKF). In addition, resolution mismatching cases are processed in detail to produce high-quality images. The effectiveness of the ELEF model is highlighted by the performance metrics, achieving a Peak Signal-to-Noise Ratio (PSNR) of 42.99 and a Structural Similarity Index (SSIM) of 0.9889, which indicates the ability to carry out the desired high-quality image restoration and enhancement.

This study aims to investigate the feasibility of using photoacoustic microscopy for the diagnosis of prostatitis. We induced inflammation in rats to establish a model of prostatitis using Freund’s Complete Adjuvant (FCA). Prostate tissues from both the model and control groups were extracted and processed into histological sections. We explored the photoacoustic microscopy imaging results of unstained sections, consistent with the histological detection results using HE staining. Inflammation was found to enhance the photoacoustic signals. Subsequently, we conducted photoacoustic microscopy imaging on all samples, and the detection results were nearly consistent with the diagnoses made by medical professionals. Finally, we quantified the collected photoacoustic signals to classify the severity of prostatitis.

Angiogenesis induced by growth factor administration, which can augment the blood supply in regenerative applications, has drawn wide attention in medical research. Longitudinal monitoring of vascular structure and development in vivo is important for understanding and evaluating the dynamics of involved biological processes. In this work, a dual-modality imaging system consisting of photoacoustic microscopy (PAM) and optical coherence tomography (OCT) was applied for noninvasive in vivo imaging of angiogenesis in a murine model. Fibrin scaffolds, with and without basic fibroblast growth factor (bFGF), were implanted in a flexible imaging window and longitudinally observed over 9 days. Imaging was conducted at 3, 5, 7, and 9 days after implantation to monitor vascularization in and around the scaffold. Several morphometric parameters were derived from the PAM images, including vessel area density (VAD), total vessel length (TVL), and vessel mean diameter (VMD). On days 7 and 9, mice receiving bFGF-laden fibrin gels exhibited significantly larger VAD and TVL compared to mice with fibrin-only gels. In addition, VMD significantly decreased in +bFGF mice versus fibrin-only mice on days 7 and 9. Blood vessel density, evaluated using immunohistochemical staining of explanted gels and underlying tissue on day 9, corroborated the findings from the PAM images. Overall, the experimental results highlight the utility of a dual-modality imaging system in longitudinally monitoring of vasculature in vivo with high resolution and sensitivity, thereby providing an effective tool to study angiogenesis.

ABSTRACTNowadays, blind image denoising with deep convolutional neural network (CNN) is one of the research hotspots in the field of image denoising. Relying on the convolutional operation and respective field, CNN is excellent in processing local information. However, this also brings the problems of lack of cross‐domain interaction and process of global feature information. We incorporate it with transformer's self‐attention mechanism and propose a cross‐channel enhanced CNN, namely CCNet, for image denoising. CCNet consists of three parts: the backbone encoder (BE), the cross‐channel enhancer (CCE), and the backbone decoder (BD). The BE and BD are constructed using the multiscale symmetric network U‐Net and use the residual connection block (RCB) as the basic block for image feature extraction and reconstruction. CCE introduces transposed attention, serving as a complement of BE for cross‐channel modeling. Meanwhile, we propose a unique gated fusion block (GFB) to fuse the information of these two modules and further feature learning. To improve training, we use random cropping, shuffling, and mixed noise strategies to expand the noise distribution learned by the model, increasing its noise adaptability. Extensive experiments on grayscale images, color images, and real noisy images demonstrate CCNet's strong performance in these tasks.

Photoacoustic imaging is a new imaging technology in recent years, which combines the advantages of high resolution and rich contrast of optical imaging with the advantages of high penetration depth of acoustic imaging. Photoacoustic imaging has been widely used in biomedical fields, such as brain imaging, tumor detection and so on. The signal-tonoise ratio (SNR) of image signals in photoacoustic imaging is generally low due to the limitation of laser pulse energy, electromagnetic interference in the external environment and system noise. In order to solve the problem of low SNR of photoacoustic images, we use feedforward denoising convolutional neural network to further process the obtained images, so as to obtain higher SNR images and improve image quality. We use Python language to manage the referenced Python external library through Anaconda, and build a feedforward noise-reducing convolutional neural network on Pycharm platform. We first processed and segmated a training set containing 400 images, and then used it for network training. Finally, we tested it with a series of cerebrovascular photoacoustic microscopy images. The results show that the peak signal-to-noise ratio (PSNR) of the image increases significantly before and after denoising. The experimental results verify that the feed-forward noise reduction convolutional neural network can effectively improve the quality of photoacoustic microscopic images, which provides a good foundation for the subsequent biomedical research.

Identifying complete response (CR) after rectal cancer preoperative treatment is critical to deciding subsequent management. Imaging techniques, including endorectal ultrasound and MRI, have been investigated but have low negative predictive values. By imaging post-treatment vascular normalization using photoacoustic microscopy, we hypothesize that co-registered ultrasound and photoacoustic imaging will better identify complete responders. In this study, we used in vivo data from 21 patients to develop a robust deep learning model (US-PAM DenseNet) based on co-registered dual-modality ultrasound (US) and photoacoustic microscopy (PAM) images and individualized normal reference images. We tested the model's accuracy in differentiating malignant from non-cancer tissue. Compared to models based on US alone (classification accuracy 82.9 ± 1.3%, AUC 0.917(95%CI: 0.897-0.937)), the addition of PAM and normal reference images improved the model performance significantly (accuracy 92.4 ± 0.6%, AUC 0.968(95%CI: 0.960-0.976)) without increasing model complexity. Additionally, while US models could not reliably differentiate images of cancer from those of normalized tissue with complete treatment response, US-PAM DenseNet made accurate predictions from these images. For use in the clinical settings, US-PAM DenseNet was extended to classify entire US-PAM B-scans through sequential ROI classification. Finally, to help focus surgical evaluation in real time, we computed attention heat maps from the model predictions to highlight suspicious cancer regions. We conclude that US-PAM DenseNet could improve the clinical care of rectal cancer patients by identifying complete responders with higher accuracy than current imaging techniques.