RISING: A new framework for model-based few-view CT image reconstruction with deep learning.

Within the project “Small animal proton Irradiator for Research in Molecular Image-guided radiation-Oncology” (SIRMIO) we have designed an in-beam PET scanner for preclinical application. The system is based on a novel spherical geometry, and in order to fully exploit its potential we are developing an integrated computational framework for simulation, image reconstruction and range verification. The software comprises a full Monte Carlo engine to simulate the proton treatment with related detector response, and an image reconstruction tool for simulated and experimental data. The platform is designed to integrate robust analytical reconstruction algorithms and new statistical approaches based on deep learning. The core of the framework is based on MEGAlib (The Medium Energy Gamma-ray Astronomy software library). The physical simulation is based on GEANT4. The machine learning method for the event classification is implemented with the ROOT based Toolkit for Multivariate Data Analysis (TMVA). The first prototype of the SIRMIO irradiation platform foresees a fixed beam, thus requiring the movement of the mouse for scanned beam delivery. Hence, we have extended the MEGAlib image reconstruction algorithm based on maximum-likelihood expectation-maximization (ML-EM) to correct for geometrical efficiency and attenuation taking into account the mouse motion. The goal is to be able to discriminate proton range shifts of ~ 0.5 mm. Moreover, we are augmenting the image reconstruction framework with a new approach based on machine learning, which aims at using all photon events collected during irradiation (dominated by prompt gamma) to retrieve on-the-fly the range of the beam, to complement the PET information.

We propose Deep Learning (DL) as a framework for simultaneous waveform estimation and image reconstruction in passive synthetic aperture radar (SAR). We consider a passive imaging scenario where the scene is illuminated by source of opportunity with known position, but transmitting an unknown waveform. Image reconstruction is then posed as a machine learning task for which a recurrent neural network (RNN) can be constructed by unfolding the iterations of a proximal gradient descent algorithm. We separate the unknown waveform from the known components of the forward model and parameterize the network operator by the waveform coefficients. The image estimate generated by the RNN is then mapped back to the data space by a linear decoder stage, so that the waveform coefficients are refined in an unsupervised manner using passive SAR measurements as training data. As a result, network becomes a recurrent auto-encoder. In this architecture, forward propagation recovers scene reflectivity, and backpropagation solves for waveform coefficients. The non-convex training routine is initialized using the partially known SAR forward model, which helps ensure that the optimization problem converges to a satisfactory optimal point.

Ultrafast X-ray tomographic imaging of multiphase flow in bubble columns - Part 1: Image processing and reconstruction comparison

This work aims to combine compressed sensing reconstruction with a deep learning framework to leverage their individual strengths and enable 123-view moderate-dose sparse-view reconstruction for diagnostic CT imaging systems. Specifically, linear FBP reconstruction was applied to reconstruct sparse data, followed by a trained U-Net to remove artifacts resulting from undersampling. This methodology has been exploited in many other deep learning applications for CT imaging. However, this approach may be subjected to the generalizability issue in which the reconstruction can either remove real lesions or add lesions that may not exist. This will be demonstrated in our results. In our proposed work, the output of the network is used as the prior image for a prior image constrained compressed sensing (PICCS) reconstruction. This step helps to ensure the reconstructed image is consistent with the measured data. Finally, the PICCS reconstructed image is further cleaned up by a trained light duty U-Net to improve noise texture and reduce noise to generate the final reconstructed image. Both simulation data and human subject data were used to validate the proposed image reconstruction framework. In simulation studies, it is demonstrated that the final output corrected the distorted structures in the deep learning-only reconstruction with respect to the shape, size and contrast of the structures. The final images also appeared streak-free with more natural noise texture when compared with the PICCS-only reconstruction. In human subject validation, the false positive lesion-like structures in the deep learning prior image were eliminated in the final output.

Medical image processing is often limited by the computational cost of the involved algorithms. Whereas dedicated computing devices (GPUs in particular) exist and do provide significant efficiency boosts, they have an extra cost of use in terms of housekeeping tasks (device selection and initialization, data streaming, synchronization with the CPU, and others), which may hinder developers from using them. This paper describes an OpenCL-based framework that is capable of handling dedicated computing devices seamlessly and that allows the developer to concentrate on image processing tasks. The framework handles automatically device discovery and initialization, data transfers to and from the device and the file system and kernel loading and compiling. Data structures need to be defined only once independently of the computing device; code is unique, consequently, for every device, including the host CPU. Pinned memory/buffer mapping is used to achieve maximum performance in data transfers. Code fragments included in the paper show how the computing device is almost immediately and effortlessly available to the users algorithms, so they can focus on productive work. Code required for device selection and initialization, data loading and streaming and kernel compilation is minimal and systematic. Algorithms can be thought of as mathematical operators (called processes), with input, output and parameters, and they may be chained one after another easily and efficiently. Also for efficiency, processes can have their initialization work split from their core workload, so process chains and loops do not incur in performance penalties. Algorithm code is independent of the device type targeted.

Fully automated on-scanner aortic four-dimensional flow magnetic resonance imaging processing and hemodynamic analysis

Motivation: The low clinical feasibility of time-consuming multi-shot interleaved acquisitions for IVIM-DWI. Goal(s): To achieve effective and efficient image reconstruction and biomarker estimation from highly-accelerated four-shot IVIM-DWI data in the brain. Approach: An end-to-end deep learning (DL)-based joint image reconstruction and biomarker estimation framework was proposed for highly-accelerated IVIM-DWI. It consists of a fully-supervised multi-b-value joint extraction and reconstruction module, and a self-supervised physics-informed estimation module. Results: Our framework permits high-quality reconstruction of IVIM-DWI and estimation of biomarker maps with minimized residual artifacts, improved geometric fidelity and a significant reduction of acquisition time, surpassing other conventional reconstruction methods. Impact: Our proposed DL-based technique is capable of precisely reconstructing IVIM-DWI and producing IVIM-related biomarker maps within a clinically feasible acquisition time, potentially improving the quantitative evaluation and analysis of IVIM-DWI-based assessment of cerebrovascular disease.

Recent advancements in deep learning techniques tailored for autonomous driving in Indian road conditions are crucial for revolutionizing transportation systems. Indian roads present unique challenges, including unpredictable traffic patterns, diverse road infrastructures, and challenging weather conditions. Deep learning is pivotal in addressing these challenges, focusing on perception, decision-making, and control. Various architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Deep Reinforcement Learning (DRL), are analyzed for their efficacy in autonomous driving. Sensor fusion techniques enhance perception capabilities, such as lidar, radar, and camera data integration. Additionally, advancements in semantic segmentation and object detection algorithms improve scene understanding and obstacle recognition. Collaborative efforts between academia, industry, and government agencies are essential for accelerating the deployment of autonomous driving technology in Indian road conditions. In conclusion, leveraging cutting-edge deep learning methodologies enables safe and efficient navigation of autonomous vehicles, ushering in a transformative future in transportation systems.

Recent advances in deep learning have contributed greatly to the field of parallel MR imaging, where a reduced amount of k-space data are acquired to accelerate imaging time. In our previous work, we have proposed a deep learning method to reconstruct MR images directly from k-space data acquired with Cartesian trajectories. However, MRI utilizes various non-Cartesian trajectories, such as radial trajectories, with various numbers of multi-channel RF coils according to the purpose of an MRI scan. Thus, it is important for a reconstruction network to efficiently unfold aliasing artifacts due to undersampling and to combine multi-channel k-space data into single-channel data. In this work, a neural network named ‘ETER-net’ is utilized to reconstruct an MR image directly from k-space data acquired with Cartesian and non-Cartesian trajectories and multi-channel RF coils. In the proposed image reconstruction network, the domain transform network converts k-space data into a rough image, which is then refined in the following network to reconstruct a final image. We also analyze loss functions including adversarial and perceptual losses to improve the network performance. For experiments, we acquired k-space data at a 3T MRI scanner with Cartesian and radial trajectories to show the learning mechanism of the direct mapping relationship between the k-space and the corresponding image by the proposed network and to demonstrate the practical applications. According to our experiments, the proposed method showed satisfactory performance in reconstructing images from undersampled single- or multi-channel k-space data with reduced image artifacts. In conclusion, the proposed method is a deep-learning-based MR reconstruction network, which can be used as a unified solution for parallel MRI, where k-space data are acquired with various scanning trajectories.

Emotion detection, hence, is an important step toward making human-computer interaction a more enhanced process, where systems are made capable of identifying and responding to the emotional state of users. In fact, multimodal emotion detection systems in which both auditory and visual information are fused are emerging, and these approaches toward expressive emotional states are complementary and robust. Multimodal systems enhance the quality of interacting and, through many applications, can diagnose emotional disorders, monitor automotive safety, and improve human-robot interactions. In nature, the high-dimensional space and dynamic threats have resulted in obtaining low accuracy and high computational cost in applying the traditional models based on single-modality data. On the other hand, multimodal systems explore the synergy between audio and visual data, giving better performance and higher accuracy in inferring subtle emotional expressions. The latest improvement was done on these systems using recent advancements in transfer learning and deep learning techniques.That being said, this research Proposal devises a multimodal emotion recognition system integrating speech and face information through transfer learning for improved accuracy and robustness. Serving this purpose, the objectives of this research entail the effective comparison among different transfer-learning strategies, including the impact of pre-trained models in speech-based emotion recognition, and to introduce the role of voice activity detection in the process. Advanced neural network architectures like Spatial Transformer Networks and bidirectional LSTM in facial emotion recognition will also be tested. Early and late fusion strategies will also be used to find the best strategy for combining speech and facial data.This research will target several challenges that involve the complexity of data, balancing of the model performance-robustness balance, computational limitations, and standardization of evaluations in developing a working and robust emotion recognition system to enhance digital interaction and apply in practical areas. The goal is to create a system that oversteps the limitation of single-modality models through state-of-the-art advances in deep learning, as well as front-line improvements in transfer learning, in the manner of emotion detection performance.

Three-dimensional (3D) reconstruction of computed tomography (CT) and magnetic resonance imaging (MRI) images is an important diagnostic method, which is helpful for doctors to clearly recognize the 3D shape of the lesion and make the surgical plan. In the study of medical image reconstruction, most researchers use surface rendering or volume rendering method to construct 3D models from image sequences. The watertightness of the algorithm-reconstructed surface will be affected by the segmentation precision or the thickness of the CT layer. The articular surfaces at femoral ends are often used in biomechanical simulation experiments. The model may not conform to its original shape due to the manual repair of non-watertight surfaces. To solve this problem, a 3D reconstruction method of leg bones based on deep learning is proposed in this paper. By deforming the convex hull of the target, comparing with state-of-the-art methods, our method can stably generate a watertight model with higher reconstruction accuracy. In the situation of target transition structures getting fuzzy and the layer spacing increasing, the proposed method can maintain better reconstruction performance and appear higher robustness. Also, the chamfer loss is optimized based on the rotational shape of the leg bones, and the weight of the loss function can be assigned according to the geometric characteristics of the target. Experiment results show that the optimization method improves the accuracy of the model. Furthermore, our research provides a reference for the application of deep learning in medical image reconstruction.

Photon-counting detectors acquire spectral information for the scanned object by separating incoming photons into pre-defined energy windows based on their energies. Using this energy information in computed tomography (CT) imaging, also called spectral CT, can mitigate beam-hardening artifacts and allows to estimate two or more basis material maps from the measured counts data. An important step for realizing spectral CT with photon-counting detectors is calibrating the spectral response of the imaging system. In this work, we explore the use of KL-divergence approach for spectrum estimation to allow for auto-calibration of the spectral response of each of the detector pixels in the spectral CT image reconstruction. Specifically, we incorporate unknown spectral components in the spectral CT data model and formulate simultaneous image reconstruction and estimation of these spectral components into the framework of alternating minimization. A simulation study is carried out to show how the algorithm can be implemented on spectral CT data.

Markov random field (MRF) model has been widely used in Bayesian image reconstruction to reconstruct piecewise smooth images in the presence of noise, such as in low-dose X-ray computed tomography (LdCT). While it can preserve edge sharpness via edge-preserving potential function, its regional smoothing may sacrifice tissue image textures, which have been recognized as useful imaging biomarkers, and thus it compromises clinical tasks such as differentiating malignant vs. benign lesions, e.g., lung nodule or colon polyp. This study aims to shift the edge preserving regional noise smoothing paradigm to texture-preserving framework for LdCT image reconstruction while retaining the advantage of MRF’s neighborhood system on edge preservation. Specifically, we adapted the MRF model to incorporate the image textures of lung, bone, fat, muscle, etc. from previous full-dose CT scan as a priori knowledge for texture-preserving Bayesian reconstruction of current LdCT images. To show the feasibility of proposed reconstruction framework, experiments using clinical patient scans (with lung nodule or colon polyp) were conducted. The experimental outcomes showed noticeable gain by the a priori knowledge for LdCT image reconstruction with the well-known Haralick texture measures. Thus, it is conjectured that texture-preserving LdCT reconstruction has advantages over edge-preserving regional smoothing paradigm for texture-specific clinical applications.

Recently, deep learning based MR image reconstructions have shown outstanding performance. While there have been many direct mapping based methods by deep neural networks without taking advantage of known physical model of medical imaging modality, some groups investigated combining conventional model-based image reconstruction (MBIR) and learning based method to enhance performance and computation speed of MBIR. Here, we investigated learned denoiser-based approximate message passing (LDAMP) with undersampled MR measurements. LDAMP yielded favorable performance over BM3D-based AMP even though ground truth images were noisy and deep denoisers were trained only for Gaussian noise, not for undersampling artifacts. We further investigated the feasibility of using Stein’s unbiased risk estimator (SURE) to fine-tune deep denoisers with given undersampled MR measurement. Our proposed SURE based unsupervised fine-tuning method faithfully reconstructed images corresponding to the measurement and demonstrated the potential of enhancing the image quality of LDAMP results on real MRI dataset.

Quantization-aware training for low precision photonic neural networks