Non-tracking-based 2D strain estimation in tagged MRI

Medical imaging systems are commonly assessed by use of objective image quality measures. Supervised deep learning methods have been investigated to implement numerical observers for task-based image quality assessment. However, labeling large amounts of experimental data to train deep neural networks is tedious, expensive, and prone to subjective errors. Computer-simulated image data can potentially be employed to circumvent these issues; however, it is often difficult to computationally model complicated anatomical structures, noise sources, and the response of real world imaging systems. Hence, simulated image data will generally possess physical and statistical differences from the experimental image data they seek to emulate. Within the context of machine learning, these differences between the sets of two images is referred to as domain shift. In this study, we propose and investigate the use of an adversarial domain adaptation method to mitigate the deleterious effects of domain shift between simulated and experimental image data for deep learning-based numerical observers (DL-NOs) that are trained on simulated images but applied to experimental ones. In the proposed method, a DL-NO will initially be trained on computer-simulated image data and subsequently adapted for use with experimental image data, without the need for any labeled experimental images. As a proof of concept, a binary signal detection task is considered. The success of this strategy as a function of the degree of domain shift present between the simulated and experimental image data is investigated.

Palpation has been widely used to detect hard tumorous tissues surrounded by softer normal tissues. The goal of ultrasound tissue elasticity imaging is to extract information regarding tissue stiffness that is closely related to pathology. For this tissue elasticity imaging, compression is applied first, and the amount of resulting tissue deformation or strain needs to be estimated. Traditionally, strain estimators aim to accurately derive tissue displacements between pre- and post-compression and compute strain from the displacements. However, the displacement can be as large as a thousand times of strain for typical compression levels used in ultrasound elasticity imaging. Error in displacement estimation leads to a large variance in strain, thus resulting in poor signal to noise ratio for the estimated strain. We have developed a novel strain estimator that directly estimates strain from the phase of temporal and spatial correlation instead of estimating small strain from large displacements. SNRe (signal to noise ratio of elastogram) and CNRe (contrast to noise ratio of elastogram) using the direct strain estimator are at least three times and six times larger than that using conventional displacement-based strain estimators, respectively. These results indicate that the direct strain estimator can significantly improve accuracy and lesion detectability in ultrasound elasticity imaging. In addition, the direct strain estimator is computationally efficient compared to conventional estimators, thus enabling the realtime implementation and clinical use of this new ultrasound imaging mode.

Duchenne muscular dystrophy (DMD) is a fatal inherited genetic disorder that results in progressive muscle weakness and ultimately loss of ambulation, respiratory failure and heart failure. Cardiac MRI (MRI) plays an increasingly important role in the diagnosis and clinical care of boys with DMD and associated cardiomyopathies. Conventional cardiac MRI biomarkers permit measurements of global cardiac function and presence of fibrosis, but changes in these measures are late manifestations. Emerging MRI biomarkers of myocardial function and structure include the estimation of rotational mechanics and regional strain using MRI tagging; T1-mapping; and T2-mapping, a marker of inflammation, edema and fat. These emerging biomarkers provide earlier insights into cardiac involvement in DMD, improving patient care and aiding the evaluation of emerging therapies.

Large amounts of data are generated continually in the real world. The objects in the data and the relationships between them have become increasingly complex. Graph is a powerful tool for representing these data and the complex relationships between them. To effectively describe the entities and connections in these data, the concept of a knowledge graph was proposed, and knowledge graph has become one of the bases for storing graph data, and the importance of it is self-evident. Genealogy data is a kind of graph data where nodes are used to represent the person in the genealogy data, and edges are used to represent the relationship between the person. Moreover, in a genealogy graph database, the critical relationship that is unique for an entity is defined. In contrast, the abnormal data derive from the existence of multiple critical relationships between entities. In a genealogy graph, abnormal data will cause the wrong relationships and redundant entities. To avoid such abnormal data in genealogy graph database, the abnormal data are categorized into four different types, and the corresponding processing methods are proposed for each type of abnormal data, respectively. The experiment results demonstrate that the processing method can effectively solve these abnormal data.

Ultrasound Myocardial Elastography (UME) and Tagged Magnetic Resonance Imaging (tMRI) are two imaging modalities that were developed in the recent years to quantitatively estimate the myocardial deformations. Tagged MRI is currently considered as the gold standard for myocardial strain mapping in vivo. However, despite the low SNR nature of ultrasound signals, echocardiography enjoys the widespread availability in the clinic, as well as its low cost and high temporal resolution. Comparing the strain estimation performances of the two techniques has been of great interests to the community. In order to assess the cardiac deformation across different imaging modalities, in this paper, we developed a semi-automatic intensity and gradient based registration framework that rigidly registers the 3D tagged MRIs with the 2D ultrasound images. Based on the two registered modalities, we conducted spatially and temporally more detailed quantitative strain comparison of the RF-based UME technique and tagged MRI. From the experimental results, we conclude that qualitatively the two modalities share similar overall trends. But error and variations in UME accumulate over time. Quantitatively tMRI is more robust and accurate than UME.

Three-dimensional vascular strain estimation is crucial for assessment of the location of high strain regions in the carotid artery (CA) and the identification of vulnerable plaque features. This study compares 2D vs. 3D displacement estimation in terms of radial and circumferential strain using simulated ultrasound images of a 3D atherosclerotic CA model at the bifurcation embedded in surrounding tissue. The 3D finite element model (FEM) of a patient-specific, pulsating atherosclerotic CA (pulse pressure 60 mmHg) was generated with ABAQUS FEM software. Global longitudinal motion was superimposed to the model. Radiofrequency (RF) ultrasound data were simulated in Field II by moving point scatterers (vessel wall) according to the deformation patterns of the model. A linear array transducer (f c = 9 MHz, pitch = 198 µm, 192 elements) was used which transmitted plane waves at 3 alternating angles (+19.5°, 0°, −19.5°) at a pulse repetition rate of 10 kHz. Simulations with 20 ms (systole) and 100 ms (diastole) inter-frame (IF) time were performed for 191 equally spaced (0.1 mm) longitudinal positions of the internal CA containing fatty and calcified areas. After delay-and-sum beamforming, IF axial displacements were estimated using a coarse-to-fine normalized cross-correlation method. The axial displacement at 0° was used as the vertical displacement component. Projection of the −19.5° and +19.5° axial displacements yielded the horizontal displacement component. A polar grid and the lumen center were determined in the end-diastolic frame of each longitudinal position and used to convert the tracked vertical and horizontal displacements into radial and circumferential displacements. Least squares strain estimation was performed to determine accumulated radial and circumferential strain. The performance of the 2D and 3D method was compared by calculating the root-mean-squared error (RMSE) of the estimated strains with respect to the reference strains obtained from the model. More accurate strain images were obtained using the 3D displacement estimation for the entire cardiac cycle. The 3D technique clearly outperforms the 2D technique in phases with high IF longitudinal motion.

The images captured by vision-based tactile sensors carry information about high-resolution tactile fields, such as the distribution of the contact forces applied to their soft sensing surface. However, extracting the information encoded in the images is challenging and often addressed with learning-based approaches, which generally require a large amount of training data. This article proposes a strategy to generate tactile images in simulation for a vision-based tactile sensor based on an internal camera that tracks the motion of spherical particles within a soft material. The deformation of the material is simulated in a finite element environment under a diverse set of contact conditions, and spherical particles are projected to a simulated image. Features extracted from the images are mapped to the three-dimensional contact force distribution, with the ground truth also obtained using finite-element simulations, with an artificial neural network that is therefore entirely trained on synthetic data avoiding the need for real-world data collection. The resulting model exhibits high accuracy when evaluated on real-world tactile images, is transferable across multiple tactile sensors without further training, and is suitable for efficient real-time inference.

C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks

Bridging the simulation-to-real gap of depth images for deep reinforcement learning

Myocardial deformation and strain can be investigated using suitably encodedcine MRI that admits disambiguation of material motion. Practical limitationscurrently restrict the analysis to in-plane motion in cross-sections of the heart(2D + time), but the proposed method readily generalizes to 3D + time. We proposea new, promising methodology, which departs from a multiscale algorithm thatexploits local scale selection so as to obtain a robust estimate for the velocitygradient tensor field. Time evolution of the deformation tensor is governed by afirst-order ordinary differential equation, which is completely determined by thisvelocity gradient tensor field. We solve this matrix-ODE analytically and presentresults obtained from healthy volunteers as well as from patient data. The proposedmethod requires only off-the-shelf algorithms and is readily applicable to planar orvolumetric tagging MRI sampled on arbitrary coordinate grids.

For heart diagnostics, ultrasound is generally the modality of choice due to its high temporal and spatial resolution, availability, and patient safety. Although 3D echocardiography captures the complex shape and motion of the heart with more precision than 2D, it suffers to a greater extent from poor resolution, noise, and limited field-of-view. Multi-perspective echocardiography has proven to significantly enhance both image quality and field-of-view. The greatest improvements occur when combining acquisitions from widely differing insonification angles, but this process is challenging because of substantial local structural and brightness variations and ultrasound's anisotropic nature. To handle these inconsistencies, a novel temporal and spatial registration algorithm designed is proposed. Temporal registration is achieved using low-frequency cardiac wall features and motion extracted via singular value decomposition of a spatio-temporal Casorati matrix, while spatial registration is performed using phase-only correlation of low-frequency data. The acquisitions are seamlessly fused using a 3D, oriented, wavelet transform including a near-field clutter algorithm. In vitro and in vivo testing highlights the benefits of this approach. Temporal alignment, validated against electrocardiograms, is precise, with an average error of just 2 ± 10 ms. Furthermore, our method outperforms a six-degree-of-freedom encoder-based probe tracker, reducing spatial registration error to 5 ± 3 mm from 19 ± 10 mm. The resulting longitudinal and radial strain measurements closely align with those obtained by tagged magnetic resonance imaging, demonstrating the accuracy and feasibility of this technique.

Anomaly detection of internal defects in underground engineering structures is critical. This paper proposes an unsupervised deep learning image-to-image translation method tailored for ground penetrating radar (GPR) images. The proposed model can translate real-world GPR images to simulated ones. In this manner, labeling real GPR images is not necessary, and only the target detection model trained on simulated GPR images is required to directly identify defects in real GPR images. The unsupervised deep learning network introduces geometry-consistency constraints into the CycleGAN, which largely prevents the problem of semantic distortion in translation. Validation of the proposed method was performed using GPR data collected in various scenarios using GPR of different center frequencies and manufacturers. Moreover, to verify its adaptability and feasibility for defect recognition, commonly used deep learning-based defect recognition methods, which were trained only on simulated GPR images, were used to detect the translated GPR images. The findings indicate that the type and location of internal defects in translated GPR images can be accurately identified using the proposed method.

Despeckling is a key preprocessing step for applications using PolSAR data in most cases. In this paper, a technique based on a nonlocal weighted linear minimum mean-squared error (NWLMMSE) filter is proposed for polarimetric synthetic aperture radar (PolSAR) speckle filtering. In the process of filtering a pixel by the LMMSE estimator, the idea of nonlocal means is employed to evaluate the weights of the samples in the estimator, based on the statistical equalities between the neighborhoods of the sample pixels and the processed pixel. The NWLMMSE estimator is then derived. In the preliminary processing, an effective step is taken to preclassify the pixels, aiming at preserving point targets and considering the similarity of the scattering mechanisms between pixels in the subsequent filter. A simulated image and two real-world PolSAR images are used for illustration, and the experiments show that this filter is effective in speckle reduction, while effectively preserving strong point targets, edges, and the polarimetric scattering mechanism.

Visual navigation models based on deep learning can learn effective policies when trained on large amounts of visual observations through reinforcement learning. Unfortunately, collecting the required experience in the real world requires the deployment of a robotic platform, which is expensive and time-consuming. To deal with this limitation, several simulation platforms have been proposed in order to train visual navigation policies on virtual environments efficiently. Despite the advantages they offer, simulators present a limited realism in terms of appearance and physical dynamics, leading to navigation policies that do not generalize in the real world. In this paper, we propose a tool based on the Habitat simulator which exploits real world images of the environment, together with sensor and actuator noise models, to produce more realistic navigation episodes. We perform a range of experiments to assess the ability of such policies to generalize using virtual and real-world images, as well as observations transformed with unsupervised domain adaptation approaches. We also assess the impact of sensor and actuation noise on the navigation performance and investigate whether it allows to learn more robust navigation policies. We show that our tool can effectively help to train and evaluate navigation policies on real-world observations without running navigation episodes in the real world.

Objective. Synthesize realistic and controllable respiratory motions in the extended cardiac-torso (XCAT) phantoms by developing a generative adversarial network (GAN)-based deep learning technique. Methods. A motion generation model was developed using bicycle-GAN with a novel 4D generator. Input with the end-of-inhale (EOI) phase images and a Gaussian perturbation, the model generates inter-phase deformable-vector-fields (DVFs), which were composed and applied to the input to generate 4D images. The model was trained and validated using 71 4D-CT images from lung cancer patients and then applied to the XCAT EOI images to generate 4D-XCAT with realistic respiratory motions. A separate respiratory motion amplitude control model was built using decision tree regression to predict the input perturbation needed for a specific motion amplitude, and this model was developed using 300 4D-XCAT generated from 6 XCAT phantom sizes with 50 different perturbations for each size. In both patient and phantom studies, Dice coefficients for lungs and lung volume variation during respiration were compared between the simulated images and reference images. The generated DVF was evaluated by deformation energy. DVFs and ventilation maps of the simulated 4D-CT were compared with the reference 4D-CTs using cross correlation and Spearman’s correlation. Comparison of DVFs and ventilation maps among the original 4D-XCAT, the generated 4D-XCAT, and reference patient 4D-CTs were made to show the improvement of motion realism by the model. The amplitude control error was calculated. Results. Comparing the simulated and reference 4D-CTs, the maximum deviation of lung volume during respiration was 5.8%, and the Dice coefficient reached at least 0.95 for lungs. The generated DVFs presented comparable deformation energy levels. The cross correlation of DVFs achieved 0.89 ± 0.10/0.86 ± 0.12/0.95 ± 0.04 along the x/y/z direction in the testing group. The cross correlation of ventilation maps derived achieved 0.80 ± 0.05/0.67 ± 0.09/0.68 ± 0.13, and the Spearman’s correlation achieved 0.70 ± 0.05/0, 60 ± 0.09/0.53 ± 0.01, respectively, in the training/validation/testing groups. The generated 4D-XCAT phantoms presented similar deformation energy as patient data while maintained the lung volumes of the original XCAT phantom (Dice = 0.95, maximum lung volume variation = 4%). The motion amplitude control models controlled the motion amplitude control error to be less than 0.5 mm. Conclusions. The results demonstrated the feasibility of synthesizing realistic controllable respiratory motion in the XCAT phantom using the proposed method. This crucial development enhances the value of XCAT phantoms for various 4D imaging and therapy studies.