OpenCL: a viable solution for high-performance medical image reconstruction?

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Faced with a backlog of nuclear waste and weapons plutonium, as well as an ever-increasing public concern about safety and environmental issues associated with conventional nuclear reactors, many countries are studying new, accelerator-driven technologies that hold the promise of providing safe and effective solutions to these problems. Proposed projects include accelerator transmutation of waste (ATW), accelerator-based conversion of plutonium (ABC), accelerator-driven energy production (ADEP), and accelerator production of tritium (APT). Also, next-generation spallation neutron sources based on similar technology will play a major role in materials science and biological science research. The design of accelerators for these projects will require a major advance in numerical modeling capability. For example, beam dynamics simulations with approximately 100 million particles will be needed to ensure that extremely stringent beam loss requirements (less than a nanoampere per meter) can be met. Compared with typical present-day modeling using 10,000–100,000 particles, this represents an increase of 3–4 orders of magnitude. High performance computing (HPC) platforms make it possible to perform such large scale simulations, which require 10’s of GBytes of memory. They also make it possible to perform smaller simulations in a matter of hours that would require months to run on a single processor workstation. This paper will describe how HPC platforms can be used to perform the numerically intensive beam dynamics simulations required for development of these new accelerator-driven technologies.

Scientific applications are often irregular and characterized by large computationally-intensive parallel loops. Dynamic loop scheduling (DLS) techniques improve the performance of computationally-intensive scientific applications via load balancing of their execution on high-performance computing (HPC) systems. Identifying the most suitable choices of data distribution strategies, system sizes, and DLS techniques which improve the performance of a given application, requires intensive assessment and a large number of exploratory native experiments (using real applications on real systems), which may not always be feasible or practical due to associated time and costs. In such cases, simulative experiments are more appropriate for studying the performance of applications. This motivates the question of How realistic are the simulations of executions of scientific applications using DLS on HPC platforms? In the present work, a methodology is devised to answer this question. It involves the experimental verification and analysis of the performance of DLS in scientific applications. The proposed methodology is employed for a computer vision application executing using four DLS techniques on two different HPC plat- forms, both via native and simulative experiments. The evaluation and analysis of the native and simulative results indicate that the accuracy of the simulative experiments is strongly influenced by the approach used to extract the computational effort of the application (FLOP- or time-based), the choice of application model representation into simulation (data or task parallel), and the available HPC subsystem models in the simulator (multi-core CPUs, memory hierarchy, and network topology). The minimum and the maximum percent errors achieved between the native and the simulative experiments are 0.95% and 8.03%, respectively.

An iterative image reconstruction is expected as a powerful dose reduction method in clinical CT examinations. Although many studies have been made on the iterative reconstruction, its clinical use in CT routine examinations is still challenging. An iterative reconstruction is capable of denoising images in low dose CT scans. However, its long time processing prevents clinical work flows in routine examinations. The one of the solutions is a few iterations with high-performance computing.

BackgroundIterative computed tomography (CT) image reconstruction shows high potential for the preservation of image quality in diagnostic CT while reducing patients’ exposure; it has become available for low-dose CT (LD-CT) in high-end hybrid imaging systems (e.g. single-photon emission computed tomography [SPECT]-CT).PurposeTo examine the effect of an iterative CT reconstruction algorithm on image quality, image noise, detectability, and the reader’s confidence for LD-CT data by a subjective assessment.Material and MethodsThe LD-CT data were validated for 40 patients examined by an abdominal hybrid SPECT-CT (U = 120 kV, I = 40 mA, pitch = 1.375). LD-CT was reconstructed using either filtered back projection (FBP) or an iterative image reconstruction algorithm (Adaptive Statistical Iterative Reconstruction [ASIR]®) with different parameters (ASIR levels 50% and 100%). The data were validated by two independent blinded readers using a scoring system for image quality, image noise, detectability, and reader confidence, for a predefined set of 16 anatomic substructures.ResultsThe image quality was significantly improved by iterative reconstruction of the LD-CT data compared with FBP (P ≤ 0.0001). While detectability increased in only 2/16 structures (P ≤ 0.03), the reader’s confidence increased significantly due to iterative reconstruction (P ≤ 0.002). Meanwhile, at the ASIR level of 100%, the detectability in bone structure was highly reduced (P = 0.003).ConclusionAn ASIR level of 50% represents a good compromise in abdominal LD-CT image reconstruction. The specific ASIR level improved image quality (reduced image noise) and reader confidence, while preserving detectability of bone structure.

To prospectively compare traditional filtered back projection (FBP) and iterative image reconstruction for the evaluation of heavily calcified arteries with coronary computed tomography (CT) angiography. The study had institutional review board approval and was HIPAA compliant. Written informed consent was obtained from all patients. Fifty-five consecutive patients (35 men, 20 women; mean age, 58 years ± 12 [standard deviation]) with Agatston scores of at least 400 underwent coronary CT angiography and cardiac catheterization. Image data were reconstructed with both FBP and iterative reconstruction techniques with corresponding cardiac algorithms. Image noise and subjective image quality were compared. To objectively assess the effect of FBP and iterative reconstruction on blooming artifacts, volumes of circumscribed calcifications were measured with dedicated volume analysis software. FBP and iterative reconstruction series were independently evaluated for coronary artery stenosis greater than 50%, and their diagnostic accuracy was compared, with cardiac catheterization as the reference standard. Statistical analyses included paired t tests, Kruskal-Wallis analysis of variance, and a modified McNemar test. Image noise measured significantly lower (P = .011-.035) with iterative reconstruction instead of FBP. Image quality was rated significantly higher (P = .031 and .042) with iterative reconstruction series than with FBP. Calcification volumes measured significantly lower (P = .019 and .026) with iterative reconstruction (44.3 mm(3) ± 64.7 and 46.2 mm(3) ± 68.8) than with FBP (54.5 mm(3) ± 69.5 and 56.3 mm(3) ± 72.5). Iterative reconstruction significantly improved some measures of per-segment diagnostic accuracy of coronary CT angiography for the detection of significant stenosis compared with FBP (accuracy: 95.9% vs 91.8%, P = .0001; specificity: 95.8% vs 91.2%, P = .0001; positive predictive value: 76.9% vs 61.1%, P = .0001). Iterative reconstruction reduces image noise and blooming artifacts from calcifications, leading to improved diagnostic accuracy of coronary CT angiography in patients with heavily calcified coronary arteries.

Fifth generation (5G) systems are considered as the future of telecommunications and data processing. 5G systems are envisaged to increase current network capacity by 1000-fold. In addition to increasing spectral efficiency and utilizing wider bandwidths, ultra network densification with wireless small cells is considered a significant capacity-enhancing method for meeting the ever-increasing demand of 5G data traffic requirements. The enormous amounts of data traffic generated in inter-small-cell 5G backhaul networks require efficient routing protocols to speed up the routing decisions, while ensuring high data rates, low latency, and low power consumption requirements. Therefore, a parallel routing protocol is proposed to speed up routing decisions in 5G backhaul networks, using high performance computing (HPC). We explore the efficiency of utilizing HPC to manage and speed up the parallel routing protocol using different communication network sizes. Our numerical results indicate that our HPC implementation achieves a routing speed-up of 37× for large network size (2048).

3D or multi-dimensional medical image reconstruction is attracting more and more attention in modern era. Rapid volumetric image reconstruction is of paramount importance to clinicians for prompt diagnosis and analysis of complex tissue alternations. However, due to a high demand on computation for either the analytical or iterative image reconstruction algorithms, it is almost impossible to meet the requirement of real-time reconstruction. On the other hand, with high computing capability and broad banding characteristic, recently multi-core processors provide a new possibility and hopeful start for some of the most computation-demanding applications, such as medical imaging, digital media and entertainment equipment. This study aims to parallelize an analytical 3D CT image reconstruction algorithm, the so-called FDK method, and to implement it on an existing multi-core platform system, towards a real-time 3D medical image reconstruction. In this paper, the parallelization of the 3D image reconstruction algorithm, the coding optimizations and a comparative result of run time savings are described.

Computer vision and sensing systems have been widely applied to industry, and this includes robotics, an area requiring real-time applications. However, robust and high-accuracy systems are usually time-consuming, but high-performance devises can speed up the systems. Many applications use GPU (Graphic Processing Unit) for faster computation. The greatest advantages of the GPU system include high-memory bandwidth and flexibility and ease of programming. However, this method requires transferring data between host (computer) and device (graphic card). Moreover, it only supports data parallelism or overlap copy data and kernel execution. Multi-core CPU offers another approach for improving system performance. OpenMP, an API that supports multi-core CPU programming strongly, offers flexibility and simple programming using compiler directives items; in particular, the computation can be done without transferring data to the device and full task parallel execution. However, multi-core CPU is limited to several cores. Combining the advantages of GPU and multi-core CPU optimizes system performance. This paper proposes a novel high-performance computing model using combination architecture of GPU and multi-core CPU to obtain a two parallel layer system including data parallel and task parallel models. The result implements a highly accurate 3D reconstruction application using the improved HOC structured light coding algorithm. The experimental results of the proposed model reveal a speed 20 times faster than the original implementation in the single CPU.

The dissemination of multi-core architectures and the later irruption of massively parallel devices, led to a revolution in High-Performance Computing (HPC) platforms in the last decades. As a result, Field-Programmable Gate Arrays (FPGAs) are re-emerging as a versatile and more energy-efficient alternative to other platforms. Traditional FPGA design implies using low-level Hardware Description Languages (HDL) such as VHDL or Verilog, which follow an entirely different programming model than standard software languages, and their use requires specialized knowledge of the underlying hardware. In the last years, manufacturers started to make big efforts to provide High-Level Synthesis (HLS) tools, in order to allow a grater adoption of FPGAs in the HPC community.Our work studies the use of multi-core hardware and different FPGAs to address Numerical Linear Algebra (NLA) kernels such as the general matrix multiplication GEMM and the sparse matrix-vector multiplication SpMV. Specifically, we compare the behavior of fine-tuned kernels in a multi-core CPU processor and HLS implementations on FPGAs. We perform the experimental evaluation of our implementations on a low-end and a cutting-edge FPGA platform, in terms of runtime and energy consumption, and compare the results against the Intel MKL library in CPU.

Non-contrast cerebral computed tomography (CT) is frequently performed as a first-line diagnostic approach in patients with suspected ischemic stroke. The purpose of this study was to evaluate the performance of hybrid and model-based iterative image reconstruction for standard-dose (SD) and low-dose (LD) non-contrast cerebral imaging by multi-detector CT (MDCT). We retrospectively analyzed 131 patients with suspected ischemic stroke (mean age: 74.2 ± 14.3 years, 67 females) who underwent initial MDCT with a SD protocol (300 mAs) as well as follow-up MDCT after a maximum of 10 days with a LD protocol (200 mAs). Ischemic demarcation was detected in 26 patients for initial and in 64 patients for follow-up imaging, with diffusion-weighted magnetic resonance imaging (MRI) confirming ischemia in all of those patients. The non-contrast cerebral MDCT images were reconstructed using hybrid (Philips “iDose4”) and model-based iterative (Philips “IMR3”) reconstruction algorithms. Two readers assessed overall image quality, anatomic detail, differentiation of gray matter (GM)/white matter (WM), and conspicuity of ischemic demarcation, if any. Quantitative assessment included signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) calculations for WM, GM, and demarcated areas. Ischemic demarcation was detected in all MDCT images of affected patients by both readers, irrespective of the reconstruction method used. For LD imaging, anatomic detail and GM/WM differentiation was significantly better when using the model-based iterative compared to the hybrid reconstruction method. Furthermore, CNR of GM/WM as well as the SNR of WM and GM of healthy brain tissue were significantly higher for LD images with model-based iterative reconstruction when compared to SD or LD images reconstructed with the hybrid algorithm. For patients with ischemic demarcation, there was a significant difference between images using hybrid versus model-based iterative reconstruction for CNR of ischemic/contralateral unaffected areas (mean ± standard deviation: SD_IMR: 4.4 ± 3.1, SD_iDose: 3.5 ± 2.3, P < 0.0001; LD_IMR: 4.6 ± 2.9, LD_iDose: 3.2 ± 2.1, P < 0.0001). In conclusion, model-based iterative reconstruction provides higher CNR and SNR without significant loss of image quality for non-enhanced cerebral MDCT.

In recent several years, motivated by the need to reduce radiation doses in CT exams, all of the major CT manufacturers have commercialized different iterative image reconstruction techniques and these innovative techniques were used in clinical routines with increasing popularity. However, due to the intrinsic nonlinearity of these new techniques, the well accepted quantitative image quality assessment metrics such as spatial resolution and contrast to noise ratio are not sufficient to provide the needed quantitative metrics for assessing image quality and for guiding the CT scan protocol optimization. This symposium aims at providing a thorough update to AAPM community on what we have understood in the past for linear CT imaging system, what are the new challenges and opportunities offered by the nonlinear iterative image reconstruction, and what would be the future directions in quantitative image quality assessment that the AAPM community can work together to address the challenges and to adapt the nonlinear image reconstruction methods to routine clinical practice to improve patient care. Three invited speakers will lead the discussions in this symposium.Dr. Ke Li from the University of Wisconsin‐Madison will be presenting the new challenges introduced in model‐based iterative reconstruction (MBIR) method with a focus on how the nonlinear nature of the MBIR reconstruction poses challenges in quantitative image quality metrics such as spatial resolution, noise power spectrum, and the limitations of CNR in protocol optimization which eventually leads to the need of task‐based detectability index as the metric.Dr. Shuai Leng from Mayo Clinic will then present how the CNR metric is insufficient for nonlinear reconstruction algorithms, and how task based model observers can be used to more accurately quantify image quality, and to optimize imaging system and scanning protocols for nonlinear iterative image reconstruction algorithms based on specific imaging task. Practical considerations regarding the application of task based image quality metrics will also be discussed.Dr. Ehsan Samei from Duke University will start the symposium by summarizing a to‐be‐released AAPM Task Group report (TG 233: Performance Evaluation of Computed Tomography Systems). He will cover what has been understood and accepted for linear CT imaging system, what are the new challenges in CT image quality assessment introduced by the nonlinear iterative reconstruction (IR) techniques, and summarize both taskneutral and task‐based metrologies developed in TG 233 to partially address these challenges, and foresee additional challenges in the future. Learning Objectives: Understand the challenges in quantitative image quality assessment introduced by nonlinear iterative reconstruction techniques Understand the challenges and methods in noise performance assessment for nonlinear iterative reconstruction techniques Understand the challenges and methods in in spatial resolution assessment for nonlinear iterative reconstruction techniques Understand the challenges and methods in in task‐based observer performance assessment for nonlinear iterative reconstruction techniquesFunding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; S. Leng, R01 EB071095; U01 EB017185; E. Samei, Research grant, Siemens; Research grant, GE; K. Li, Funding from NIH, DOD and AAPM

This paper presents the optimized design and implementation of sparse tensor-times-dense matrix multiply (SpTTM) for CPU and GPU platforms. This primitive is a critical bottleneck in data analysis and mining applications based on tensor methods, such as the Tucker decomposition. We first design and implement sequential SpTTM to avoid explicit data transformations between a tensor and a matrix, which is the conventional approach. We further optimize SpTTM on multicore CPU and GPU systems by parallelizing, avoiding locks, and exploiting data locality. Our sequential SpTTM is up to 3.5× faster than the SpTTM from Tensor Toolbox and 1.5× over that from Cyclops Tensor Framework. Our parallel algorithms show 4.1× speedup on multicore Intel Core i7 and 18.8× speedup on NVIDIA K40c GPU over our sequential SpTTM respectively.

The rapid adoption of machine learning (ML) has increased the computational and storage demands on high-performance computing (HPC) and cloud datacenters. Unfortunately, deploying ML models on HPC and cloud platforms is challenging because it is often difficult to manage, financially costly, and poses substantial environmental concerns. This dissertation attempts to answer a simple but ambitious question: can we deploy ML models to meet the Quality of Service (QoS) requirements while remaining cost-effective and environmentally sustainable? The dissertation demonstrates that achieving this balance is indeed possible, albeit through employing non-intuitive strategies and user engagements. We show that a combination of fast and slow hardware can outperform the common strategy of exclusively employing the latest and fastest hardware when co-optimizing for both performance and cost. Realizing this potential, however, requires a strategy to create an effective hardware mix, a process rigorously developed within this dissertation. This dissertation further reveals that the carbon footprint of ML models can be managed by opportunistically trading off answer quality transiently and re-configuring/partitioning the GPU hardware resources, depending on the carbon intensity of available energy sources in the datacenter's region. Finally, this dissertation discovers that ML deployments can be made more environmentally sustainable by strategically engaging users and tuning response brevity, especially in the context of large language models (LLMs). The scientific advancements made in this dissertation aim to relieve the community from the cost constraints, hardware inefficiencies, and potential negative environmental impacts of large-scale deployment of ML models on HPC and cloud platforms. We are hopeful that its open-source contributions will promote further innovation and broader engagement toward sustainable ML deployment.--Author's abstract

From its founding in 1952 until today, Lawrence Livermore National Laboratory (LLNL) has made significant strategic investments to develop high performance computing (HPC) and its application to national security and basic science. Now, 60 years later, the Computation Directorate and its myriad resources and capabilities have become a key enabler for LLNL programs and an integral part of the effort to support our nation's nuclear deterrent and, more broadly, national security. In addition, the technological innovation HPC makes possible is seen as vital to the nation's economic vitality. LLNL, along with other national laboratories, is working to make supercomputing capabilities and expertise available to industry to boost the nation's global competitiveness. LLNL is on the brink of an exciting milestone with the 2012 deployment of Sequoia, the National Nuclear Security Administration's (NNSA's) 20-petaFLOP/s resource that will apply uncertainty quantification to weapons science. Sequoia will bring LLNL's total computing power to more than 23 petaFLOP/s-all brought to bear on basic science and national security needs. The computing systems at LLNL provide game-changing capabilities. Sequoia and other next-generation platforms will enable predictive simulation in the coming decade and leverage industry trends, such as massively parallel and multicore processors, to run petascale applications. Efficient petascale computing necessitates refining accuracy in materials property data, improving models for known physical processes, identifying and then modeling for missing physics, quantifying uncertainty, and enhancing the performance of complex models and algorithms in macroscale simulation codes. Nearly 15 years ago, NNSA's Accelerated Strategic Computing Initiative (ASCI), now called the Advanced Simulation and Computing (ASC) Program, was the critical element needed to shift from test-based confidence to science-based confidence. Specifically, ASCI/ASC accelerated the development of simulation capabilities necessary to ensure confidence in the nuclear stockpile-far exceeding what might have been achieved in the absence of a focused initiative. While stockpile stewardship research pushed LLNL scientists to develop new computer codes, better simulation methods, and improved visualization technologies, this work also stimulated the exploration of HPC applications beyond the standard sponsor base. As LLNL advances to a petascale platform and pursues exascale computing (1,000 times faster than Sequoia), ASC will be paramount to achieving predictive simulation and uncertainty quantification. Predictive simulation and quantifying the uncertainty of numerical predictions where little-to-no data exists demands exascale computing and represents an expanding area of scientific research important not only to nuclear weapons, but to nuclear attribution, nuclear reactor design, and understanding global climate issues, among other fields. Aside from these lofty goals and challenges, computing at LLNL is anything but 'business as usual.' International competition in supercomputing is nothing new, but the HPC community is now operating in an expanded, more aggressive climate of global competitiveness. More countries understand how science and technology research and development are inextricably linked to economic prosperity, and they are aggressively pursuing ways to integrate HPC technologies into their native industrial and consumer products. In the interest of the nation's economic security and the science and technology that underpins it, LLNL is expanding its portfolio and forging new collaborations. We must ensure that HPC remains an asymmetric engine of innovation for the Laboratory and for the U.S. and, in doing so, protect our research and development dynamism and the prosperity it makes possible. One untapped area of opportunity LLNL is pursuing is to help U.S. industry understand how supercomputing can benefit their business. Industrial investment in HPC applications has historically been limited by the prohibitive cost of entry, the inaccessibility of software to run the powerful systems, and the years it takes to grow the expertise to develop codes and run them in an optimal way. LLNL is helping industry better compete in the global market place by providing access to some of the world's most powerful computing systems, the tools to run them, and the experts who are adept at using them. Our scientists are collaborating side by side with industrial partners to develop solutions to some of industry's toughest problems. The goal of the Livermore Valley Open Campus High Performance Computing Innovation Center is to allow American industry the opportunity to harness the power of supercomputing by leveraging the scientific and computational expertise at LLNL in order to gain a competitive advantage in the global economy.