GPU technology in the high frame rate ultrasound imaging era

Novel 3D GPU based numerical parallel diffusion algorithms in cylindrical coordinates for health care simulation

Photoacoustic tomography (PAT) offers structural and functional imaging of living biological tissue with highly sensitive optical absorption contrast and excellent spatial resolution comparable to medical ultrasound (US) imaging. We report the development of a fully integrated PAT and US dual-modality imaging system, which performs signal scanning, image reconstruction and display for both photoacoustic (PA) and US imaging all in a truly real-time manner. The backprojection (BP) algorithm for PA image reconstruction is optimized to reduce the computational cost and facilitate parallel computation on a state of the art graphics processing unit (GPU) card. For the first time, PAT and US imaging of the same object can be conducted simultaneously and continuously, at a real time frame rate, presently limited by the laser repetition rate of 10 Hz. Noninvasive PAT and US imaging of human peripheral joints in vivo were achieved, demonstrating the satisfactory image quality realized with this system. Another experiment, simultaneous PAT and US imaging of contrast agent flowing through an artificial vessel was conducted to verify the performance of this system for imaging fast biological events. The GPU based image reconstruction software code for this dual-modality system is open source and available for download from http://sourceforge.net/projects/pat realtime .

We investigated the performance of the conventional and high frame rate ultrasound imaging techniques for strain and elasticity imaging. High frame rate imaging is necessary to track fast deformation in elasticity imaging, but the ultrasound image quality is known to be compromised. Both numerical and experimental studies were performed using point targets and tissue mimicking phantoms. The results of our study confirm that the signal to noise ratio (SNR), contrast to noise ratio (CNR), and axial/lateral resolution of the displacement and strain images acquired using high frame rate ultrasound imaging are comparable with those obtained using the conventional imaging. Thus, the high frame rate ultrasound imaging can be reliably used for both static and dynamic elasticity imaging if the frame rate to capture the internal tissue motion is critical

Objective. External beam radiotherapy is aimed to precisely deliver a high radiation dose to malignancies, while optimally sparing surrounding healthy tissues. With the advent of increasingly complex treatment plans, the delivery should preferably be verified by quality assurance methods. Recently, online ultrasound imaging of vaporized radiosensitive nanodroplets was proposed as a promising tool for in vivo dosimetry in radiotherapy. Previously, the detection of sparse vaporization events was achieved by applying differential ultrasound (US) imaging followed by intensity thresholding using subjective parameter tuning, which is sensitive to image artifacts. Approach. A generalized deep learning solution (i.e. BubbleNet) is proposed to localize vaporized nanodroplets on differential US frames, while overcoming the aforementioned limitation. A 5-fold cross-validation was performed on a diversely composed 5747-frame training/validation dataset by manual segmentation. BubbleNet was then applied on a test dataset of 1536 differential US frames to evaluate dosimetric features. The intra-observer variability was determined by scoring the Dice similarity coefficient (DSC) on 150 frames segmented twice. Additionally, the BubbleNet generalization capability was tested on an external test dataset of 432 frames acquired by a phased array transducer at a much lower ultrasound frequency and reconstructed with unconventional pixel dimensions with respect to the training dataset. Main results. The median DSC in the 5-fold cross validation was equal to ∼0.88, which was in line with the intra-observer variability (=0.86). Next, BubbleNet was employed to detect vaporizations in differential US frames obtained during the irradiation of phantoms with a 154 MeV proton beam or a 6 MV photon beam. BubbleNet improved the bubble-count statistics by ∼30% compared to the earlier established intensity-weighted thresholding. The proton range was verified with a −0.8 mm accuracy. Significance. BubbleNet is a flexible tool to localize individual vaporized nanodroplets on experimentally acquired US images, which improves the sensitivity compared to former thresholding-weighted methods.

A system for real-time ultrasound (US) elastography will advance interventions for the diagnosis and treatment of cancer by advancing methods such as thermal monitoring of tissue ablation. A multi-stream graphics processing unit (GPU) based accelerated normalized cross-correlation (NCC) elastography, with a maximum frame rate of 78 frames per second, is presented in this paper. A study of NCC window size is undertaken to determine the effect on frame rate and the quality of output elastography images. This paper also presents a novel system for Online Tracked Ultrasound Elastography (O-TRuE), which extends prior work on an offline method. By tracking the US probe with an electromagnetic (EM) tracker, the system selects in-plane radio frequency (RF) data frames for generating high quality elastograms. A novel method for evaluating the quality of an elastography output stream is presented, suggesting that O-TRuE generates more stable elastograms than generated by untracked, free-hand palpation. Since EM tracking cannot be used in all systems, an integration of real-time elastography and the da Vinci Surgical System is presented and evaluated for elastography stream quality based on our metric. The da Vinci surgical robot is outfitted with a laparoscopic US probe, and palpation motions are autonomously generated by customized software. It is found that a stable output stream can be achieved, which is affected by both the frequency and amplitude of palpation. The GPU framework is validated using data from in-vivo pig liver ablation; the generated elastography images identify the ablated region, outlined more clearly than in the corresponding B-mode US images.

Prognostic methods enable operators and maintainers to predict the future performance for critical systems. However, these methods can be computationally expensive and may need to be performed each time new information about the system becomes available. In light of these computational requirements, we have investigated the application of graphics processing units (GPUs) as a computational platform for real-time prognostics. Recent advances in GPU technology have reduced cost and increased the computational capability of these highly parallel processing units, making them more attractive for the deployment of prognostic software. We present a survey of model-based prognostic algorithms with considerations for leveraging the parallel architecture of the GPU and a case study of GPU-accelerated battery prognostics with computational performance results.

This work aims to investigate the FPGA (Field-Programmable Gate Array) and GPU (Graphical Processing Unit) technology in image optimization research for an industrial frontier study. Using an experimental method, the research compared the efficiency of two technologies as implemented in some many image processing algorithms. NI CompactRIO platform for FPGA implementation and NVIDIA GeForce GTX 970 in GPU processing performed differently. As is well known, low-lag applications (camera synchronization, real-time data processing etc.) were very well suited for FPGAs. GPUs with architecture CUDA, on the other hand could be a thousand times faster than traditional CPUs in parallel data processing. Other challenges identified through analysis were FPGA design optimization and GPU resource wise utilization. The results give recommendation in terms of selecting technologies based on the features for image industrial processing applications

Quantitative assessment for characterization of breast lesion tissues using adaptively decomposed ultrasound RF images

Phase-change ultrasound contrast agents (PCCAs), have shown promises for ultrasound-mediated diagnostics and therapeutics. High-frame-rate ultrasound imaging with microbubbles significantly improved temporal resolution and contrast compared to conventional contrast-enhanced ultrasound imaging, offering opportunities to observe and better understand PCCA behavior after vaporisation. In this study, high-frame-rate ultrasound was used to image the PCCAs immediately after vaporisation in vitro with clinical ultrasound parameters. Compared with microbubbles, the vaporised PCCAs have shown different temporal signal features with high-frame-rate ultrasound imaging: a significant decrease in the first tens of milliseconds after acoustic vaporisation while no evident change over time with microbubble signal. Simultaneous optical microscopy was operated to observe behaviors of the vaporised PCCAs. Highframe-rate ultrasound imaging enables improved understanding of the acoustic behavior of the vaporised PCCA populations at high temporary resolution.

The safety and efficacy of proton therapy is currently hampered by range uncertainties. The combination of ultrasound imaging with injectable radiation-sensitive superheated nanodroplets was recently proposed for in vivo range verification. The proton range can be estimated from the distribution of nanodroplet vaporization events, which is stochastically related to the stopping distribution of protons, as nanodroplets are vaporized by protons reaching their maximal LET at the end of their range. Here, we aim to estimate the range estimation precision of this technique. As for any stochastic measurement, the precision will increase with the sample size, that is, the number of detected vaporizations. Thus, we first develop and validate a model to predict the number of vaporizations, which is then applied to estimate the range verification precision for a set of conditions (droplet size, droplet concentration, and proton beam parameters). Starting from the thermal spike theory, we derived a model that predicts the expected number of droplet vaporizations in an irradiated sample as a function of the droplet size, concentration, and number of protons. The model was validated by irradiating phantoms consisting of size-sorted perfluorobutane droplets dispersed in an aqueous matrix. The number of protons was counted with an ionization chamber, and the droplet vaporizations were recorded and counted individually using high frame rate ultrasound imaging. After validation, the range estimate precision was determined for different conditions using a Monte Carlo algorithm. A good agreement between theory and experiments was observed for the number of vaporizations, especially for large (5.8±2.2µm) and medium (3.5±1.1µm) sized droplets. The number of events was lower than expected in phantoms with small droplets (2.0±0.7µm), but still within the same order of magnitude. The inter-phantom variability was considerably larger (up to 30x) than predicted by the model. The validated model was then combined with Monte Carlo simulations, which predicted a theoretical range retrieval precision improving with the square-root of the number of vaporizations, and degrading at high beam energies due to range straggling. For single pencil beams with energies between 70 and 240 MeV, a range verification precision below 1% of the range required perfluorocarbon concentrations in the order of 0.3-2.4µM. We proposed and experimentally validated a model to provide a quick estimate of the number of vaporizations for a given set of conditions (droplet size, droplet concentration, and proton beam parameters). From this model, promising range verification performances were predicted for realistic perfluorocarbon concentrations. These findings are an incentive to move towards preclinical studies, which are critical to assess the achievable droplet distribution in and around the tumor, and hence the in vivo range verification precision.

Very high frame rate ultrasound imaging has enabled the development of novel imaging modalities such as the functional imaging of the brain, cardiac electrophysiology, and the quantitative real-time imaging of the intrinsic mechanical properties of tumors, to name a few. We present here the extension of Ultrafast Ultrasound Imaging in three dimensions based on the use of either diverging or plane waves emanating from a sparse virtual array located behind the probe. High contrast and resolution were achieved while maintaining imaging rates of thousands of volumes per second. A customized portable ultrasound system was developed to sample 1024 independent channels and to drive a 32X32 matrix-array probe. Its capability to track in 3D transient phenomena occurring in the millisecond range within a single ultrafast acquisition was demonstrated for 3-D Shear-Wave Imaging, 4-D Ultrafast Doppler Imaging, and 4D vector flow imaging. These results demonstrate the potential of 4-D Ultrafast Ultrasound Imaging for the volumetric real-time mapping of stiffness, tissue motion and flow in humans in vivo and promises new clinical quantitative applications of ultrasound with reduced intra- and inter-observer variability.

The synthetic aperture focusing technique (SAFT) has been proposed to improve the lateral resolution of ultrasound image outside the focal region of the transducer. Yet, SAFT would result in an unexpected artifact image at the region near the focal point of the transducer due to the use of a virtue source concept associated with limited numbers of scan lines. To further improve this issue, the present study incorporated the transducer’s depth-dependency beam characteristics into SAFT-based imaging procedure. The imaging process was implemented on a graphic processing unit (GPU) platform for improving the computational efficiency. Ultrasound images were acquired from a commercial phantom with various contrast regions using a 3.5 MHz single element transducer. The -20dB beam width of transducer as a function of axial depth were measured for better estimating the time delay of each scanning point. Two scanning intervals, 50 and 200 μm, were acquired for B-mode imaging. Subsequently, the linear interpolation was performed for correcting the error associated with the time delay estimator for the image at the focus region. Results demonstrated that SAFT-based images implemented with virtual source correction corresponding to scanning intervals of 50 and 200 μm both displayed darker regions near the depth of 23 mm. The SAFT-based images modified with the correction of -20dB beam width of the transducer may not only improve the artifact region but also better restore the shape and contrast regions in the deeper regions of phantom. In additional, the computational efficiency is verified able to be greatly improved by the implementation with GPU platform. These results verify that current modified SAFT-based imaging may greatly enhance ultrasound image quality.KeywordsModified synthetic aperture focusing techniqueVirtual sourceBeam widthTransducer characteristics

Since its inception, ultrasound has been recognized to be one of the most significant technologies in clinical imaging. The rapid and continuous development of ultrasound scanning technologies has added another challenge of rendering relatively large scale 3D volumes with high resolution. Moreover, some applications have required particular high quality ultrasound rendering at interactive frame rate for improved interpretation of the data. Based on the vast computing power of the modern Graphics Processing Units (GPUs), we present an efficient ultrasound reconstruction pipeline that is capable of rendering high quality and high resolution ultrasound images at interactive frame rates for large scale volumes. The reconstruction results of our rendering pipeline have been applied on a real fetal ultrasound data. The performance benchmarks of our pipeline demonstrate its capability of rendering high quality ultrasound images for 10243 volumes with resolutions of 20482 at interactive frame rates using a recent commodity GPU.

An ultrasound image is reconstructed from echo signals received by array elements of a transducer. The time of flight of the echo depends on the distance between the focus to the array elements. The received echo signals have to be delayed to make their wave fronts and phase coherent before summing the signals. In digital beamforming, the delays are not always located at the sampled points. Generally, the values of the delayed signals are estimated by the values of the nearest samples. This method is fast and easy, however inaccurate. There are other methods available for increasing the accuracy of the delayed signals and, consequently, the quality of the beamformed signals; for example, the in-phase (I)/quadrature (Q) interpolation, which is more time consuming but provides more accurate values than the nearest samples. This paper compares the signals after dynamic receive beamforming, in which the echo signals are delayed using two methods, the nearest sample method and the I/Q interpolation method. The comparisons of the visual qualities of the reconstructed images and the qualities of the beamformed signals are reported. Moreover, the computational speeds of these methods are also optimized by reorganizing the data processing flow and by applying the graphics processing unit (GPU). The use of single and double precision floating-point formats of the intermediate data is also considered. The speeds with and without these optimizations are also compared.

BackgroundThoracic lymph node (LN) evaluation is essential for the accurate diagnosis of lung cancer and deciding the appropriate course of treatment. Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is considered a standard method for mediastinal nodal staging. This study aims to build a deep convolutional neural network (CNN) for the automatic classification of metastatic malignancies involving thoracic LN, using EBUS-TBNA.MethodsPatients who underwent EBUS-TBNAs to assess the presence of malignancy in mediastinal LNs during a ten-month period at Severance Hospital, Seoul, Republic of Korea, were included in the study. Corresponding LN ultrasound images, pathology reports, demographic data, and clinical history were collected and analyzed.ResultsA total of 2,394 endobronchial ultrasound (EBUS) images of 1,459 benign LNs from 193 patients, and 935 malignant LNs from 177 patients, were collected. We employed the visual geometry group (VGG)-16 network to classify malignant LNs using only traditional cross-entropy for classification loss. The sensitivity, specificity, and accuracy of predicting malignancy were 69.7%, 74.3%, and 72.0%, respectively, and the overall area under the curve (AUC) was 0.782. We applied the new loss function to train the network and, using the modified VGG-16, the AUC improved to a value of 0.8. The sensitivity, specificity, and accuracy improved to 72.7%, 79.0%, and 75.8%, respectively. In addition, the proposed network can process 63 images per second on a single mainstream graphics processing unit (GPU) device, making it suitable for real-time analysis of EBUS images.ConclusionsDeep CNNs can effectively classify malignant LNs from EBUS images. Selecting LNs that require biopsy using real-time EBUS image analysis with deep learning is expected to shorten the EBUS-TBNA procedure time, increase lung cancer nodal staging accuracy, and improve patient safety.