Model-informed deep-learning photoacoustic reconstruction for low-element linear array

.Significance: Photoacoustic (PA) imaging can provide structural, functional, and molecular information for preclinical and clinical studies. For PA imaging (PAI), non-ideal signal detection deteriorates image quality, and quantitative PAI (QPAI) remains challenging due to the unknown light fluence spectra in deep tissue. In recent years, deep learning (DL) has shown outstanding performance when implemented in PAI, with applications in image reconstruction, quantification, and understanding.Aim: We provide (i) a comprehensive overview of the DL techniques that have been applied in PAI, (ii) references for designing DL models for various PAI tasks, and (iii) a summary of the future challenges and opportunities.Approach: Papers published before November 2020 in the area of applying DL in PAI were reviewed. We categorized them into three types: image understanding, reconstruction of the initial pressure distribution, and QPAI.Results: When applied in PAI, DL can effectively process images, improve reconstruction quality, fuse information, and assist quantitative analysis.Conclusion: DL has become a powerful tool in PAI. With the development of DL theory and technology, it will continue to boost the performance and facilitate the clinical translation of PAI.

Recent advancements in deep learning techniques tailored for autonomous driving in Indian road conditions are crucial for revolutionizing transportation systems. Indian roads present unique challenges, including unpredictable traffic patterns, diverse road infrastructures, and challenging weather conditions. Deep learning is pivotal in addressing these challenges, focusing on perception, decision-making, and control. Various architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Deep Reinforcement Learning (DRL), are analyzed for their efficacy in autonomous driving. Sensor fusion techniques enhance perception capabilities, such as lidar, radar, and camera data integration. Additionally, advancements in semantic segmentation and object detection algorithms improve scene understanding and obstacle recognition. Collaborative efforts between academia, industry, and government agencies are essential for accelerating the deployment of autonomous driving technology in Indian road conditions. In conclusion, leveraging cutting-edge deep learning methodologies enables safe and efficient navigation of autonomous vehicles, ushering in a transformative future in transportation systems.

Combined Photoacoustic (PA) and Ultrasound (US) imaging systems are finding more preclinical and clinical applications. However, majority of the commercial systems use expensive pulsed lasers. In most small animal studies and clinical applications like arthritis screening of finger joints, there is a need for tomographic imaging. In this work, we present PA and US tomographic imaging using a commercial imaging system with LED arrays as illumination source. We employ multiangle spatial compounding of PA and US images using a probe with a linear array transducer and four LED arrays, to form dual-mode tomographic images. Using phantom experiments, the proposed approach is validated and thoroughly tested. Further, the potential of the system is demonstrated by imaging knee joint and abdominal region of a mouse. This proposed approach has several advantages. First, the resolution and signal to noise ratio (SNR) are improved with the compounding of images from multiple angles. The resolution improvement owes to better axial resolution compared to lateral and high SNR with averaging. Secondly, the limited view artifacts and loss of information from the use of a linear array US probe is tackled. The US tomographic images of the mouse-knee RA model show structural details of the joint and blood vessels were visible in the tomographic PA images. The whole animal images enabled improved functional and structural information. An affordable PA/US tomographic imaging system with potential in clinical arthritis-screening and small animal imaging is presented.

Attention-driven complementary information fusion network for sparse photoacoustic image reconstruction.

Emotion detection, hence, is an important step toward making human-computer interaction a more enhanced process, where systems are made capable of identifying and responding to the emotional state of users. In fact, multimodal emotion detection systems in which both auditory and visual information are fused are emerging, and these approaches toward expressive emotional states are complementary and robust. Multimodal systems enhance the quality of interacting and, through many applications, can diagnose emotional disorders, monitor automotive safety, and improve human-robot interactions. In nature, the high-dimensional space and dynamic threats have resulted in obtaining low accuracy and high computational cost in applying the traditional models based on single-modality data. On the other hand, multimodal systems explore the synergy between audio and visual data, giving better performance and higher accuracy in inferring subtle emotional expressions. The latest improvement was done on these systems using recent advancements in transfer learning and deep learning techniques.That being said, this research Proposal devises a multimodal emotion recognition system integrating speech and face information through transfer learning for improved accuracy and robustness. Serving this purpose, the objectives of this research entail the effective comparison among different transfer-learning strategies, including the impact of pre-trained models in speech-based emotion recognition, and to introduce the role of voice activity detection in the process. Advanced neural network architectures like Spatial Transformer Networks and bidirectional LSTM in facial emotion recognition will also be tested. Early and late fusion strategies will also be used to find the best strategy for combining speech and facial data.This research will target several challenges that involve the complexity of data, balancing of the model performance-robustness balance, computational limitations, and standardization of evaluations in developing a working and robust emotion recognition system to enhance digital interaction and apply in practical areas. The goal is to create a system that oversteps the limitation of single-modality models through state-of-the-art advances in deep learning, as well as front-line improvements in transfer learning, in the manner of emotion detection performance.

Photoacoustic (PA) imaging is an emerging imaging technique for many clinical applications. One of the challenges posed by clinical translation is that imaging systems often rely on a finite-aperture transducer rather than a full tomography system. This results in imaging artifacts arising from an underdetermined reconstruction of the initial pressure distribution (IPD). Furthermore, clinical applications often require deep imaging, resulting in a low-signal-to-noise ratio for the acquired signal because of strong light attenuation in tissue. Conventional approaches to reconstruct the IPD, such as back projection and time-reversal, do not adequately suppress the artifacts and noise. We propose a sparsity-based optimization approach that improves the reconstruction of IPD in PA imaging with a linear array ultrasound transducer. In simulation studies, the forward model matrix was measured from k-Wave simulations, and the approach was applied to reconstruct simulated point objects and the Shepp-Logan phantom. The results were compared with the conventional back projection, time-reversal approach, frequency-domain reconstruction, and the iterative least-squares approaches. In experimental studies, the forward model of our imaging system is directly measured by scanning a graphite point source through the imaging field of view. Experimental images of graphite inclusions in tissue-mimicking phantoms are reconstructed and compared with the back projection and iterative least-squares approaches. Overall these results show that our proposed optimization approach can leverage the sparsity of the PA images to improve the reconstruction of the IPD and outperform the existing popular reconstruction approaches.

Quantization-aware training for low precision photonic neural networks

Breast cancer is the most prevalent cancer globally and presents a significant threat to human health. The low survival rate for advanced stages highlights the importance of early and accurate diagnosis. Pathology images, providing insights into cancer's cellular aggressiveness, are crucial for breast cancer diagnosis. Advances in deep learning have transformed automated medical image diagnosis. However, traditional deep learning methods, reliant on large labeled datasets, face challenges with the typically small size of medical image datasets. To overcome this, our study combines deep transfer learning (DTL) with convolutional neural networks(CNN). We utilize and enhance a ResNet50 model pretrained on a large dataset. Additionally, we apply four different deep transfer learning methods to feature extraction and classification of the BreakHis dataset, aiming to determine if breast cancer is benign or malignant. Ultimately, our system achieved the best average accuracy rates under four different magnifications: 98.57%, 96.83%, 97.76%, and 97.16%. These results demonstrate the accuracy and potential of deep transfer learning in classifying breast cancer histopathological images and its promising application in tumor prediction.

Integrative System of Fast Photoacoustic Imaging

Tomographic photoacoustic and ultrasound imaging is essential for isotropic spatial resolution and to obtain a full view of the target tissue. However, tomographic systems with pulsed laser sources and custom made transducer arrays are expensive. Additionally, there are other factors that limit the wide use of photoacoustic and ultrasound tomographic systems which include the size of the tomographic systems that use pulsed laser and the laser safety issues. A cost-effective, compact and safe photoacoustic and ultrasound tomographic system can find several imaging applications both in clinics and small animal labs. LED-based photoacoustic imaging has shown the potential to bring down the cost, enable faster imaging with high pulse repetition rate and is safer when compared to pulsed lasers. The conventional US system can be adopted for photoacoustic imaging by adding a light source to it. Hence, linear transducer arrays are preferred as they are cheaper and allow faster imaging. The combination of LED-based illumination and linear transducer array-based tomographic imaging can be a cost-effective alternative to current tomographic imaging, especially in point-of-care applications.

Photoacoustic imaging is a new imaging technology in recent years, which combines the advantages of high resolution and rich contrast of optical imaging with the advantages of high penetration depth of acoustic imaging. Photoacoustic imaging has been widely used in biomedical fields, such as brain imaging, tumor detection and so on. The signal-tonoise ratio (SNR) of image signals in photoacoustic imaging is generally low due to the limitation of laser pulse energy, electromagnetic interference in the external environment and system noise. In order to solve the problem of low SNR of photoacoustic images, we use feedforward denoising convolutional neural network to further process the obtained images, so as to obtain higher SNR images and improve image quality. We use Python language to manage the referenced Python external library through Anaconda, and build a feedforward noise-reducing convolutional neural network on Pycharm platform. We first processed and segmated a training set containing 400 images, and then used it for network training. Finally, we tested it with a series of cerebrovascular photoacoustic microscopy images. The results show that the peak signal-to-noise ratio (PSNR) of the image increases significantly before and after denoising. The experimental results verify that the feed-forward noise reduction convolutional neural network can effectively improve the quality of photoacoustic microscopic images, which provides a good foundation for the subsequent biomedical research.

The rapid growth of genomic data presents new challenges and opportunities in the field of bioinformatics. Deep learning has emerged as a powerful tool for analyzing large and complex genomic datasets, offering unprecedented capabilities for identifying patterns, predicting gene functions, and understanding genetic variations. This article explores the latest advancements in deep learning techniques applied to genomic data analysis. It discusses key approaches such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and autoencoders, and their applications in various aspects of genomics, including gene expression analysis, variant calling, and drug discovery. The article also highlights the integration of deep learning with other emerging technologies, such as CRISPR, and its potential to revolutionize personalized medicine and genetic research..

Deep learning belongs to the field of artificial intelligence, where machines perform tasks that typically require some kind of human intelligence. Deep learning tries to achieve this by drawing inspiration from the learning of a human brain. Similar to the basic structure of a brain, which consists of (billions of) neurons and connections between them, a deep learning algorithm consists of an artificial neural network, which resembles the biological brain structure. Mimicking the learning process of humans with their senses, deep learning networks are fed with (sensory) data, like texts, images, videos or sounds. These networks outperform the state-of-the-art methods in different tasks and, because of this, the whole field saw an exponential growth during the last years. This growth resulted in way over 10,000 publications per year in the last years. For example, the search engine PubMed alone, which covers only a sub-set of all publications in the medical field, provides already over 11,000 results in Q3 2020 for the search term ‘deep learning’, and around 90% of these results are from the last three years. Consequently, a complete overview over the field of deep learning is already impossible to obtain and, in the near future, it will potentially become difficult to obtain an overview over a subfield. However, there are several review articles about deep learning, which are focused on specific scientific fields or applications, for example deep learning advances in computer vision or in specific tasks like object detection. With these surveys as a foundation, the aim of this contribution is to provide a first high-level, categorized meta-survey of selected reviews on deep learning across different scientific disciplines and outline the research impact that they already have during a short period of time. The categories (computer vision, language processing, medical informatics and additional works) have been chosen according to the underlying data sources (image, language, medical, mixed). In addition, we review the common architectures, methods, pros, cons, evaluations, challenges and future directions for every sub-category.

Optoacoustic images from rather large tissue samples, such as the human extremities, the breast, or large organs, are preferably obtained in reflection mode. In the past it has been assumed that irradiating the tissue directly below or even better through an acoustic receiver results in an optimum image contrast. Our theoretical and experimental results however show that when a linear array transducer is used, this is not always true. The optimum location of irradiation depends on the depth of the tissue structures to be imaged and on various sources of image background, namely random optical absorption in the bulk tissue surrounding the region of interest, reconstruction artifacts, and acoustic backscattering. It turns out that the influence of absorption in the bulk tissue becomes minimal when irradiating close to the transducer aperture, the opposite however is the case for image artefact background. Its influence becomes minimal if the fluence in the tissue is homogeneously distributed obtained for an irradiation far away from the transducer. Echo background, which results from backscattered optoacoustic transients, additionally limits the imaging depth in reflection mode optoacoustic imaging. Therefore, the irradiation geometry when using a linear array transducer has to be adapted to the depth of the imaged structures.

Transfontanelle ultrasound imaging (TFUI) is the conventional approach for diagnosing brain injury in neonates. Despite being the first stage imaging modality, TFUI lacks accuracy in determining the injury at an early stage due to degraded sensitivity and specificity. Therefore, a modality like photoacoustic imaging that combines the advantages of both acoustic and optical imaging can overcome the existing TFUI limitations. Even though a variety of transducers have been used in TFUI, it is essential to identify the transducer specification that is optimal for transfontanelle imaging using the photoacoustic technique. In this study, we evaluated the performance of 6 commercially available ultrasound transducer arrays to identify the optimal characteristics for transfontanelle photoacoustic imaging. We focused on commercially available linear and phased array transducer probes with center frequencies ranging from 2.5MHz to 8.5MHz which covers the entire spectrum of the transducer arrays used for brain imaging. The probes were tested on both in vitro and ex vivo brain tissue, and their performance in terms of transducer resolution, size, penetration depth, sensitivity, signal to noise ratio, signal amplification and reconstructed image quality were evaluated. The analysis of selected transducers in these areas allowed us to determine the optimal transducer for transfontanelle imaging, based on vasculature depth and blood density in tissue using ex vivo sheep brain. The outcome of this evaluation identified the two most suitable ultrasound transducer probes for transfontanelle photoacoustic imaging.