Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction

Objective. Multi-contrast magnetic resonance (MR) imaging super-resolution (SR) reconstruction is an effective solution for acquiring high-resolution MR images. It utilizes anatomical information from auxiliary contrast images to improve the quality of the target contrast images. However, existing studies have simply explored the relationships between auxiliary contrast and target contrast images but did not fully consider different anatomical information contained in multi-contrast images, resulting in texture details and artifacts unrelated to the target contrast images. Approach. To address these issues, we propose a dual contrast attention-guided multi-frequency fusion (DCAMF) network to reconstruct SR MR images from low-resolution MR images, which adaptively captures relevant anatomical information and processes the texture details and low-frequency information from multi-contrast images in parallel. Specifically, after the feature extraction, a feature selection module based on a dual contrast attention mechanism is proposed to focus on the texture details of the auxiliary contrast images and the low-frequency features of the target contrast images. Then, based on the characteristics of the selected features, a high- and low-frequency fusion decoder is constructed to fuse these features. In addition, a texture-enhancing module is embedded in the high-frequency fusion decoder, to highlight and refine the texture details of the auxiliary contrast and target contrast images. Finally, the high- and low-frequency fusion process is constrained by integrating a deeply-supervised mechanism into the DCAMF network. Main results. The experimental results show that the DCAMF outperforms other state-of-the-art methods. The peak signal-to-noise ratio and structural similarity of DCAMF are 39.02 dB and 0.9771 on the IXI dataset and 37.59 dB and 0.9770 on the BraTS2018 dataset, respectively. The image recovery is further validated in segmentation tasks. Significance. Our proposed SR model can enhance the quality of MR images. The results of the SR study provide a reliable basis for clinical diagnosis and subsequent image-guided treatment.

In magnetic resonance imaging (MRI), low-resolution (LR) images often hamper clinical diagnosis and research due to constraints in imaging conditions and technology limitations. Recent studies in super-resolution (SR) reconstruction of multi-contrast MRI have shown promise by leveraging the complementary information from different MRI contrasts. However, existing multi-contrast MRI SR techniques face several challenges: 1) a lack of pre-alignment precision can result in distorted reconstructions; 2) prevailing transformer network structures, with their smaller windows (e.g., 8 × 8), struggle to effectively capture long-range dependencies and lack the ability to interact between different windows; and 3) current methods are limited to fixed integer scaling (e.g., 2 ×, 3 ×, 4 ×), which limits flexibility and increases complexity in training and storage. To address these challenges, we propose a novel arbitrary-scale SR network for multi-contrast MRI. Specifically, our approach compensates for spatial misalignment between modalities through deformable registration module and employs permuted cross-attention transformer in MR images. In addition, we introduce a ref-scale ensemble implicit attention module that better integrates high-frequency information from reference images and enables arbitrary-scale upsampling. Extensive experiments on two publicly available MRI datasets validate the superiority of our method in multi-contrast MRI SR, demonstrating its significant potential in clinical applications.

Magnetic resonance imaging (MRI) has an important feature that it provides multiple images with different contrasts for complementary diagnostic information. However, a large amount of data is needed for multi-contrast images depiction, and thus, the scan is time-consuming. Many methods based on parallel magnetic resonance imaging (pMRI) and compressed sensing (CS) are applied to accelerate multi-contrast MR imaging. Nevertheless, the image reconstructed by sophisticated pMRI methods contains residual aliasing artifact that degrades the quality of the image when the acceleration factor is high. Other methods based on CS always suffer the regularization parameter-selecting problem. To address these issues, a new method is presented for joint multi-contrast image reconstruction and coil sensitivity estimation. The coil sensitivities can be shared during the reconstruction due to the identity of coil sensitivity profiles of different contrast images for imaging stationary tissues. The proposed method uses the coil sensitivities as sharable information during the reconstruction to improve the reconstruction quality. As a result, the residual aliasing artifact can be effectively removed in the reconstructed multi-contrast images even if the acceleration factor is high. Besides, as there is no regularization term in the proposed method, the troublesome regularization parameter selection in the CS can also be avoided. Results from multi-contrast in vivo experiments demonstrated that multi-contrast images can be jointly reconstructed by the proposed method with effective removal of the residual aliasing artifact at a high acceleration factor.

In magnetic resonance imaging (MRI), multi-contrast pulse sequences are routinely acquired, providing complementary information for medical diagnosis. Compared with the single-contrast MR image reconstruction, the multi-contrast MR imaging could further accelerate data acquisition and improve reconstruction quality by leveraging the complementary information of multi-contrast MR images. In this paper, we propose a model-driven graph transformer (MD-GraphFormer) for fast multi-contrast MR imaging, which incorporates the physical constraints of MRI and investigates the complementary information among multi-contrast MR images using graph structure and attention mechanism. The MD-GraphFormer consists of graph-attention-based interaction modules (GAB-IM) and multi-contrast data consistency modules (MC-DCM). GAB-IM learns and interacts the features of multi-contrast MR images over the graph with nodes representing MR contrasts. MC-DCM enforces the consistency between the reconstructed multi-contrast MR images and their corresponding measurements in k-space. Extensive experiments are conducted on the collected raw uMR and SMS brain MRI datasets under different sampling patterns and sampling rates. The results demonstrate that the proposed MD-GraphFormer outperforms the previous multi-contrast MRI reconstruction methods in multi-coil imaging settings.

In clinical exams, multi-contrast images from conventional MRI are scanned with the same field of view (FOV) for complementary diagnostic information, such as proton density- (PD-), T1- and T2-weighted images. Their sharable information can be utilized for more robust and accurate image reconstruction. In this work, we propose a novel model and an efficient algorithm for joint image reconstruction and coil sensitivity estimation in multi-contrast partially parallel imaging (PPI) in MRI. Our algorithm restores the multi-contrast images by minimizing an energy function consisting of an L2-norm fidelity term to reduce construction errors caused by motion, a regularization term of underlying images to preserve common anatomical features by using vectorial total variation (VTV) regularizer, and updating sensitivity maps by Tikhonov smoothness based on their physical property. We present the numerical results including T1- and T2-weighted MR images recovered from partially scanned k-space data and provide the comparisons between our results and those obtained from the related existing works. Our numerical results indicate that the proposed method using vectorial TV and penalties on sensitivities can be made promising and widely used for multi-contrast multi-channel MR image reconstruction.

To evaluate the feasibility of the Simultaneous Noncontrast Angiography and intraPlaque hemorrhage (SNAP) technique in identification of carotid plaque surface characteristics compared with the conventional multicontrast vessel wall imaging protocol. Thirty symptomatic patients with carotid plaque were recruited and underwent carotid artery magnetic resonance imaging (MRI) (3.0T) using a conventional multicontrast protocol and SNAP sequence. As an intrinsic multicontrast sequence, SNAP could generate a gray blood reference (Ref) image set, a black blood corrected real (CR) image set, and a bright blood MR angiography (MRA) image set. A bright blood SNAP Ref2 image was implemented by combining Ref and MRA images for facilitating plaque surface characteristics evaluation. The presence/absence of calcification (CA), juxtaluminal calcification (JCA), and ulceration was assessed. The agreement between SNAP and multicontrast vessel wall protocol in identifying CA, JCA, and ulceration was analyzed using Cohen's kappa analysis. The interreader and intrareader reproducibility of SNAP imaging in identifying plaque surface characteristics was also assessed. Good to excellent agreement was found between SNAP and conventional multicontrast protocol in identifying CA (κ = 0.74, 95% confidence interval [CI]: 0.54-0.93), JCA (κ = 0.81, 95% CI: 0.66-0.97), and ulceration (κ = 0.82, 95% CI: 0.65-0.99). In addition, excellent intrareader and interreader reproducibility was found for SNAP imaging in identification of CA, JCA, and ulceration. SNAP imaging showed excellent agreement with multicontrast imaging and high reproducibility in identification of both JCA and ulceration, suggesting that SNAP imaging may be a time-efficient, alternative tool in identification of plaque surface characteristics in carotid arteries. 4 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2018;47:634-639.

In the realm of medical diagnosis, recent strides in deep neural network-guided magnetic resonance imaging (MRI) restoration have shown promise. Nevertheless, persistent drawbacks overshadow these advancements. Challenges persist in balancing acquisition speed and image quality, while existing methods primarily focus on singular tasks like MRI reconstruction or super-resolution (SR), neglecting the interplay between these tasks. To tackle these challenges, this paper introduces the mutual co-attention network (MCAN) specifically designed to concurrently address both MRI reconstruction and SR tasks. Comprising multiple mutual cooperation attention blocks (MCABs) in succession, MCAN is tailored to maintain consistency between local physiological details and global anatomical structures. The intricately crafted MCAB includes a feature extraction block, a local attention block and a global attention block. Additionally, to ensure data fidelity without compromising acquired data, we propose the channel-wise data consistency block. Thorough experimentation on the IXI and fastMRI dataset showcases MCAN’s superiority over existing state-of-the-art methods. Both quantitative metrics and visual quality assessments validate the enhanced performance of MCAN in MRI restoration. The findings underscore MCAN’s potential in significantly advancing therapeutic applications. By mitigating the trade-off between acquisition speed and image quality while simultaneously addressing both MRI reconstruction and SR tasks, MCAN emerges as a promising solution in the domain of magnetic resonance image restoration.

As two different modalities of medical images, Magnetic Resonance (MR) and Computer Tomography (CT), provide mutually-complementary information to doctors in clinical applications. However, to obtain both images sometimes is cost-consuming and unavailable, particularly for special populations. For example, patients with metal implants are not suitable for MR scanning. Also, it is probably infeasible to acquire multi-contrast MR images during once clinical scanning. In this context, to synthesize needed MR images for patients whose CT images are available becomes valuable. To this end, we present a novel generative network, called CAE-ACGAN, which incorporates the advantages of Variational Auto-Encoder (VAE) and Generative Adversarial Network (GAN) with an auxiliary discriminative classifier network. We apply this network to synthesize multi-contrast MR images from single CT and conduct experiments on brain datasets. Our main contributions can be summarized as follows: 1)We alleviate the problems of images blurriness and mode collapse by integrating the advantages of VAE and GAN; 2) We solve the complicated cross-domain, multi-contrast MR synthesis task using the proposed network; 3) The technique of random-extraction-patches is used to lower the limit of insufficient training data, enabling to obtain promising results even with limited available data; 4) By comparing with other typical networks, we are able to yield nearer-real, higher-quality synthetic MR images, demonstrating the effectiveness and stability of our proposed network.

The human brain is a remarkably complex organ. Comprehension of the tissue properties of the brain and the multitude of changes in disease and ageing may lead to a greater appreciation and understanding of human biology. Combined, Magnetic Resonance Imaging (MRI) and image processing techniques are now capable of resolving very high-resolution images of the brain, including characterising discrete tissue types within small brain structures such as the hippocampus. The hippocampus is a small structure most commonly associated with its function in memory, and is comprised of subfields with distinct cellular compositions (or laminae). Computational techniques for delineating tissue types within the hippocampus incorporate sophisticated computer vision and medical imaging algorithms to better describe the range of changes occurring in these structures. In parallel, improved MR scanner hardware and sequence choices including leveraging multiple contrasts of the same tissue in vivo allows for complementary imaging information of the structure, leading to a more specific and reliable characterisation of brain tissue. Despite advances in the field, several competing algorithms and techniques are used for hippocampus subfield morphometry properties such as volume and shape.This thesis will explore computational techniques for characterising human brain tissue in vivo using multi-contrast, high resolution, ultra high-field MRI data. Specifically, this thesis will focus on characterising the tissue of the hippocampus as an exemplar for the heterogeneity of tissue classes, complex shape changes, and large, variable individual differences in anatomy between healthy and diseased/disordered subjects. This thesis will pay specific attention to characterising hippocampus tissue (in particular, image segmentation and volumetry) in a wide range of subjects and patients, which is often ignored in computational techniques for analysing MRI data.First, techniques for retrospectively ameliorating within-session motion in MRI in both healthy participants and patients with Motor Neurone Disease (MND) will be examined and discussed. It was found that incorporating a computational technique for image registration improved the reliability of image segmentation of the subfields of the hippocampus.Next, in order to better characterise the changes in anatomy over time using MRI, we constructed and evaluated a longitudinal pipeline for segmenting and examining the volume of hippocampal subfields using computational techniques in a healthy participant pool and patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI). To aid in this research, the Towards Optimising MRI ChAracterisation of Tissue (TOMCAT) data set was acquired using Ultra-High Field (7-Tesla) MRI. Thirdly, to better characterise the subfields of the hippocampus and in less time than traditional computational techniques, a deep learning approach to hippocampus subfield segmentation using multi-contrast, high-resolution MRI was developed and evaluated. We found that deep learning techniques may aid in speeding the process of hippocampus subfield segmentation and deliver accurate results.Finally, the clinical applications of the previous studies were explored. We detail automated methods for extracting precise measures describing hippocampal subfield morphology and compare these features in MND patients and healthy controls. Brain tissue can be successfully characterised in vivo using high resolution and multi-contrast MRI. Developing robust and reproducible methods that are useful in both research and clinical contexts is an important step in demystifying biological processes involved in ageing and contributing to disorder and disease. This thesis adds to and improves the broader knowledge base of computational techniques for characterising brain tissue and adds novel and enhanced methodology to the field of neuroimaging.

Automatic knee cartilage segmentation from multi-contrast MR images using support vector machine classification with spatial dependencies

Compressive sensing magnetic resonance (MR) imaging is aimed at achieving high-quality MR image reconstruction by undersampling K-space data. It is crucial to explore prior information since compressive sensing MR imaging relies heavily on some prior assumptions, such as signal's sparse property. In this study, in order to explore the prior information fully, an improved MR image reconstruction model based on compressive sensing theory is proposed, named reference image MR imaging with adaptive tight frame. In the proposed model, an adaptive tight frame is involved to explore the sparse prior information adapt to MR images and the similarity prior information to the target image. Meanwhile, improved adaptive weighting parameters are used to trade off the sparsity between the regions with much similarity and that of little similarity. In addition, the smoothing-based fast iterative shrinkage-threshold algorithm is utilised to tackle the optimisation problem so as to speed up imaging. The experimental results demonstrate that the proposed MR image reconstruction method outperforms some state-of-the-art methods in terms of quantitative results.

ObjectiveMild neurocognitive disorders (MND) affect a subset of HIV+ patients under effective combination antiretroviral therapy (cART). In this study, we used an innovative multi-contrast magnetic resonance imaging (MRI) approach at high-field to assess the presence of micro-structural brain alterations in MND+ patients.MethodsWe enrolled 17 MND+ and 19 MND− patients with undetectable HIV-1 RNA and 19 healthy controls (HC). MRI acquisitions at 3T included: MP2RAGE for T1 relaxation times, Magnetization Transfer (MT), T2* and Susceptibility Weighted Imaging (SWI) to probe micro-structural integrity and iron deposition in the brain. Statistical analysis used permutation-based tests and correction for family-wise error rate. Multiple regression analysis was performed between MRI data and (i) neuropsychological results (ii) HIV infection characteristics. A linear discriminant analysis (LDA) based on MRI data was performed between MND+ and MND− patients and cross-validated with a leave-one-out test.ResultsOur data revealed loss of structural integrity and micro-oedema in MND+ compared to HC in the global white and cortical gray matter, as well as in the thalamus and basal ganglia. Multiple regression analysis showed a significant influence of sub-cortical nuclei alterations on the executive index of MND+ patients (p = 0.04 he and R2 = 95.2). The LDA distinguished MND+ and MND− patients with a classification quality of 73% after cross-validation.ConclusionOur study shows micro-structural brain tissue alterations in MND+ patients under effective therapy and suggests that multi-contrast MRI at high field is a powerful approach to discriminate between HIV+ patients on cART with and without mild neurocognitive deficits.

Motivation: Variations in clinical magnetic resonance (MR) imaging, due to differences in hardware and acquisition parameters, often result in inconsistencies in image contrast and field-of-view, complicating analysis across imaging sites. Goal(s): To develop a method to impute missing regions of MR images of the brain due to a restricted field-of-view during harmonization. Approach: We expanded on an existing disentangling harmonization framework to differentiate between anatomical (foreground) and non-anatomical (background) regions. Results: Our approach successfully imputed the anatomy for restricted field-of-view MR images while simultaneously harmonizing contrast. Impact: Our results impact researchers handling diverse, inconsistent imaging datasets with variable field-of-view acquisitions. This approach enables the analysis of previously unusable data by imputing missing regions using multi-contrast information, making them suitable for meaningful clinical or research outcomes.

Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network

In this paper, we present a fast iterative magnetic resonance imaging (MRI) reconstruction algorithm taking advantage of the prevailing GPGPU programming paradigm. In clinical environment, MRI reconstruction is usually performed via fast Fourier transform (FFT). However, imaging artifacts (i.e. signal loss) resulting from susceptibility-induced magnetic field inhomogeneities degrade the quality of reconstructed images. These artifacts must be addressed using accurate modeling of the physics of the system coupled with iterative reconstruction. We have developed a reconstruction algorithm with improved image quality at the expense of computation time and hence an implementation on GPUs achieving significant speedup. In this work, we extend our previous work on GPU implementation by adding several new features. First, we enable Sensitivity Encoding for Fast MRI (SENSE) reconstruction (from data acquired using a multi-receiver coil array) which can reduce the acquisition time. Besides, we have implemented a GPU-based total variation regularization in our SENSE reconstruction framework. In this paper, we describe the different optimizations employed from levels of algorithm, program code structures, and specific architecture performance tuning, featuring both our MRI reconstruction algorithm and GPU hardware specifics. Results show that the current GPU implementation produces accurate image estimates while significantly accelerating the reconstruction.