WE‐C‐330A‐09: Dynamic, MultiModality Imaging: Temporal Precision and US Artifact Reduction

A system and method for converting 2-D ultrasound data into a 2-D display image. The system and method input a 2-D ultrasound data set, transform the 2-D ultrasound data set into a mathematical 3-D data set, and transform the mathematical 3-D data set into an altered 2-D data set. Another preferred embodiment of the invention involves generating a mathematical surface, such that the height of the mathematical surface is based on a first component of an ultrasound signal in a 2-D ultrasound data set; mapping the mathematical surface to points on a 2-D plane; assigning a color value to each point based on a second component of the ultrasound signal; and displaying the mapped mathematical surface with the assigned color values on a 2-D display device.

Three-dimensional (3D)/four-dimensional (4D) volume ultrasound is an established method in human medicine that offers various options for analysing and presenting ultrasound volume data. However, the successful application of the different 3D/4D imaging modalities in pregnant dogs and cats has not yet been reported in the literature. The main reasons for this are: (1) the high costs of 3D/4D ultrasound systems, (2) operation difficulties due to high breathing frequency in non-sedated animals and (3) the missing specific knowledge in veterinary medicine concerning how to perform high-quality volume scans. Automatically acquired ultrasound volume data sets were generated with two different ultrasound systems: the portable Voluson i and the stationary Voluson Expert 730. Different 3D/4D imaging modalities were tested in regard of their practicability in pregnancy monitoring in dogs and cats. Nine different volume imaging modalities were applied using the saved files. For the presentation of the static 3D volume data sets, we used the multiplanar, niche, surface, transparency, glass body, inversion, volume calculation and tomographic ultrasound imaging modes. For the dynamic 4D data, the surface and glass body modes were applied. By changing the human standard settings to the requirements of small animal anatomy, it was found that 3D/4D ultrasound has great potential for the characterization of pregnancy in queens and bitches. The 3D/4D technology offered advanced information about pregnancy status and birth prediction and improved the diagnostic confidence. By using standardized examination protocols, 3D/4D ultrasound will allow a reduction in examination time by generating even more relevant information. These benefits, combined with possible future cost reduction of commercial ultrasound systems, might lead to frequent utilization in routine pregnancy diagnostic and birth management in small animal practice.

Breast cancer diagnosis using three-dimensional ultrasound and pixel relation analysis.

3D ultrasound (US) can facilitate detailed prenatal examinations for fetal growth monitoring. To analyze a 3D US volume, it is fundamental to identify anatomical landmarks of the evaluated organs accurately. Typical deep learning methods usually regress the coordinates directly or involve heatmap-matching. However, these methods struggle to deal with volumes with large sizes and the highly-varying positions and orientations of fetuses. In this work, we exploit an object detection framework to detect landmarks in 3D fetal facial US volumes. By regressing multiple parameters of the landmark-centered bounding box (B-box) with a strict criteria, the proposed model is able to pinpoint the exact location of the targeted landmarks. Specifically, the model uses a 3D region proposal network (RPN) to generate 3D candidate regions, followed by several 3D classification branches to select the best candidate. It also adopts an IoU-balance loss to improve communications between branches that benefit the learning process. Furthermore, it leverage a distance-based graph prior to regularize the training and helps to reduce false positive predictions. The performance of the proposed framework is evaluated on a 3D US dataset to detect five key fetal facial landmarks. Results showed the proposed method outperforms some of the state-of-the-art methods in efficacy and efficiency.

Cross-modal Transfer Learning Based on an Improved CycleGAN Model for Accurate Kidney Segmentation in Ultrasound Images

Fast and accurate catheter detection in cardiac catheterization using harmless 3D ultrasound (US) can improve the efficiency and outcome of the intervention. However, the low image quality of US requires extra training for sonographers to localize the catheter. In this paper, we propose a catheter detection method based on a pre-trained VGG network, which exploits 3D information through re-organized cross-sections to segment the catheter by a shared fully convolutional network (FCN), which is called a Direction-Fused FCN (DF-FCN). Based on the segmented image of DF-FCN, the catheter can be localized by model fitting. Our experiments show that the proposed method can successfully detect an ablation catheter in a challenging ex-vivo 3D US dataset, which was collected on the porcine heart. Extensive analysis shows that the proposed method achieves a Dice score of 57.7%, which offers at least an 11.8% improvement when compared to state-of the-art instrument detection methods. Due to the improved segmentation performance by the DF-FCN, the catheter can be localized with an error of only 1.4 mm.

Recently, deep learning approaches have been successfully used for ultrasound (US) image artifact removal. However, paired high-quality images for supervised training are difficult to obtain in many practical situations. Inspired by the recent theory of unsupervised learning using optimal transport driven CycleGAN (OT-CycleGAN), here, we investigate the applicability of unsupervised deep learning for US artifact removal problems without matched reference data. Two types of OT-CycleGAN approaches are employed: one with the partial knowledge of the image degradation physics and the other with the lack of such knowledge. Various US artifact removal problems are then addressed using the two types of OT-CycleGAN. Experimental results for various unsupervised US artifact removal tasks confirmed that our unsupervised learning method delivers results comparable to supervised learning in many practical applications.

With the success of deep learning-based methods applied in medical image analysis, convolutional neural networks (CNNs) have been investigated for classifying liver disease from ultrasound (US) data. However, the scarcity of available large-scale labeled US data has hindered the success of CNNs for classifying liver disease from US data. In this work, we propose a novel generative adversarial network (GAN) architecture for realistic diseased and healthy liver US image synthesis. We adopt the concept of stacking to synthesize realistic liver US data. Quantitative and qualitative evaluation is performed on 550 in-vivo B-mode liver US images collected from 55 subjects. We also show that the synthesized images, together with real in vivo data, can be used to significantly improve the performance of traditional CNN architectures for Nonalcoholic fatty liver disease (NAFLD) classification.

As a crucial step in accessing the kidney in several minimally invasive interventions, percutaneous renal access (PRA) practicality and safety may be improved through the fusion of computed tomography (CT) and ultrasound (US) data. This work aims to assess the potential of a surface-based registration technique and establish an optimal US acquisition protocol to fuse two-dimensional (2D) US and CT data for image-guided PRA. Ten porcine kidney phantoms with fiducial markers were imaged using CT and three-dimensional (3D) US. Both images were manually segmented and aligned. In a virtual environment, 2D contours were extracted by slicing the 3D US kidney surfaces and using usual PRA US-guided views, while the 3D CT kidney surfaces were transformed to simulate positional variability. Surface-based registration was performed using two methods of the iterative closest point algorithm (point-to-point, ICP1; and point-to-plane, ICP2), while four acquisition variants were studied: (a) use of single-plane (transverse, SPT ; or longitudinal, SPL ) vs bi-plane views (BP); (b) use of different kidney's coverage ranges acquired by a probe's sweep; (c) influence of sweep movements; and (d) influence of the spacing between consecutive slices acquired for a specific coverage range. BP view showed the best performance (TRE=2.26mm) when ICP2 method, a wide kidney coverage range (20°, with slices spaced by 5°), and a large sweep along the central longitudinal view were used, showing a statistically similar performance (P=0.097) to a full 3D US surface registration (TRE=2.28mm). An optimal 2D US acquisition protocol was evaluated. Surface-based registration, using multiple slices and specific sweep movements and views, is here suggested as a valid strategy for intraoperative image fusion using CT and US data, having the potential to be applied to other image modalities and/or interventions.

Medical ultrasound (US) is a commonly used modality for image-guided procedures. Recent research systems providing an in situ visualization of 2D US images via an augmented reality (AR) head-mounted display (HMD) were shown to be advantageous over conventional imaging through reduced task completion times and improved accuracy. In this work, we continue in the direction of recent developments by describing the first AR HMD application visualizing real-time volumetric (3D) US in situ for guiding vascular punctures. We evaluated the application on a technical level as well as in a mixed-methods user study with a qualitative prestudy and a quantitative main study, simulating a vascular puncture. Participants completed the puncture task significantly faster when using 3D US AR mode compared to 2D US AR, with a decrease of 28.4% in time. However, no significant differences were observed regarding the success rate of vascular puncture (2D US AR-50% vs. 3D US AR-72%). On the technical side, the system offers a low latency of 49.90 ± 12.92 ms and a satisfactory frame rate of 60 Hz. Our work shows the feasibility of a system that visualizes real-time 3D US data via an AR HMD, and our experiments show, furthermore, that this may offer additional benefits in US-guided tasks (i.e., reduced task completion time) over 2D US images viewed in AR by offering a vividly volumetric visualization.

Recovery of respiratory motion and deformation of the liver using laparoscopic freehand 3D ultrasound system

3D ultrasound (US) to computed tomography (CT) registration is a topic of significant interest because it can potentially improve many minimally invasive procedures such as laparoscopic partial nephrectomy. Partial nephrectomy patients often receive preoperative CT angiography, which helps define the important structures of the kidney such as the vasculature. Intraoperatively, dynamic real-time imaging information can be captured using ultrasound and compared with the preoperative data. Providing accurate registration between the two modalities would enhance navigation and guidance for the surgeon. However, one of the major problems of developing and evaluating registration techniques is obtaining sufficiently accurate and realistic phantom data especially for soft tissue. We present a detailed procedure for constructing tissue phantoms using porcine kidneys, which incorporates contrast agent into the tissue such that the kidneys appear representative of in vivo human CT angiography. These phantoms are also imaged with US and resemble US images from human patients. We then perform registration on corresponding CT and US datasets using a simulation-based algorithm. The method simulates an US image from the CT, generating an intermediate modality that resembles ultrasound. This simulated US is then registered to the original US dataset. Embedded fiducial markers provide a gold standard for registration. Being able to test our registration method on realistic datasets facilitates the development of novel CT to US registration techniques such that we can generate an effective method for human studies.

Medical Ultrasound (US), despite its wide use, is characterized by artefacts and operator dependency. Those attributes hinder the gathering and utilization of US datasets for the training of deep neural networks used for computer-assisted intervention systems. Data augmentation is commonly used to enhance model generalization and performance. However, common data augmentation techniques, such as affine transformations do not align with the physics of US and, when used carelessly can lead to unrealistic US images. To this end, we propose a set of physics-inspired transformations, including deformation, reverb and signal-to-noise ratio, that we apply on US B-mode images for data augmentation. We evaluate our method on a new spine US dataset for the tasks of bone segmentation and classification.

Three-dimensional (3D) ultrasound data are acquired mostly using a dedicated mechanical probe that houses a 1D array transducer. This 1D transducer swivels back and forth continuously in the elevation direction (continuous scanning) for fast acquisition. When 3D ultrasound data are acquired via continuous scanning but the continuous motion of a transducer is not taken into account during reconstruction, the reconstructed volume contains error. In this study, we systematically analyzed this error, which is a complex function of many parameters. The error increases when the transducer angular speed (ω) increases. Also, it varies depending on the voxel location inside an acquired volume. The mean error is calculated by averaging the errors at all acquired voxel locations. With a 60-degree volume angle, a 60-degree sector angle, 12-cm scan depth and 48 transmit beams per slice, the mean error is 5.3 mm when ω is 0.6 degrees/ms. When ω is reduced to 0.1 degrees/ms, the mean error decreases to 0.81 mm. We also assessed the impact of this error on the reconstructed images of a 3D phantom using simulation. At high angular speeds, the error in reconstructed images becomes noticeable and results in missing parts, geometric distortion and lowered image quality.

The thickness of nuchal translucency (NT) is one of the most important parameters to be detected in prenatal screening during the first trimester of pregnancy. In this paper, an automatic method is proposed for the localization of the volume of interest (VOI) of NT in three-dimensional (3D) ultrasound data. With the size and position of the fetal head and the equation of the sagittal plane in the 3D ultrasound data as the prior knowledge, the method can be divided into three steps. First, a Hessian-based plate filter is proposed to enhance the possible plates in the volume. Second, several VOIs with large filter responses are selected as candidates. Those VOIs consisting of several connected planes are segmented according to the prior knowledge of the sagittal plane. Third, a set of shape features is proposed to select the NT VOI with the maximum probability. Automatic fetal head localization and sagittal plane detection can be achieved using our previously proposed methodology, which means fully automatic NT volume detection could be reached by using our method. The proposed method produces 86.36% localization accuracy with the manual selection as the ground truth.