Multicenter aortic vessel tree extraction using deep learning

Deep Learning (DL) has become a prominent machine learning technique due to the availability of efficient computational resources in the form of Graphics Processing Units (GPUs), large-scale datasets and a variety of models. The newer generation of GPUs are being designed with special emphasis on optimizing performance for DL applications. Also, the availability of easy-to-use DL frameworks—like PyTorch and TensorFlow— has enhanced productivity of domain experts to work on their custom DL applications from diverse domains. However, existing Deep Neural Network (DNN) training approaches may not fully utilize the newly emerging powerful GPUs like the NVIDIA A100—this is the primary issue that we address in this paper. Our motivating analyses show that the GPU utilization on NVIDIA A100 can be as low as 43% using traditional DNN training approaches for small-to-medium DL models and input data size. This paper proposes AccDP—a data-parallel distributed DNN training approach—to accelerate GPU-based DL applications. AccDP exploits the Message Passing Interface (MPI) communication library coupled with the NVIDIA’s Multi-Process Service (MPS) to increase the amount of work assigned to parallel GPUs resulting in higher utilization of compute resources. We evaluate our proposed design on different small-to-medium DL models and input sizes on the state-of-the-art HPC clusters. By injecting more parallelism into DNN training using our approach, the evaluation shows up to 58% improvement in training performance on a single GPU and up to 62% on 16 GPUs compared to regular DNN training. Furthermore, we conduct an in-depth characterization to determine the impact of several DNN training factors and best practices—including the batch size and the number of data loading workers— to optimally utilize GPU devices. To the best of our knowledge, this is the first work that explores the use of MPS and MPI to maximize the utilization of GPUs in distributed DNN training.

Recently deep-learning (DL) techniques have been widely adopted in side-channel power analysis. A DL-assisted SCA generally consists of two phases: a deep neural network (DNN) training phase and a follow-on attack phase using the trained DNN. However, currently the two phases are not well aligned, as there is no conclusion on what metric used in the training can result in the most effective attack in the second phase. When traditional loss functions such as negative log-likelihood (NLL) are used in training a DNN, the trained model does not yield optimal follow-on attack. Recently some information theoretical SCA leakage metrics are proposed, either as the validation metric to stop the DNN training with traditional loss functions, or as both the validation metric and the training loss function. None of those proposed metrics, however, directly measures the SCA effectiveness. We propose to conduct DNN training directly with a common SCA effectiveness metric, Guessing Entropy (GE). We overcome the prior practical difficulty of using GE in DNN training by utilizing the GEEA estimation algorithm introduced in CHES 2020. We show that using GEEA as either the validation metric or the loss function produces DNN models that lead to much more effective follow-on attacks. Our work consolidates the DL-assisted SCA framework with a consistent metric, which shows great potential to be adopted as the universal SCA-oriented DNN training framework.

Dose reduction techniques for computed tomography angiography (CTA) in children with cardiovascular diseases have the potential of reducing risks of radiation-induced cancer. To evaluate effectiveness of these techniques, both radiation dose and image quality must be compared. While clinically practical methods of estimating effective dose are available, there are no generally accepted metrics for the assessment of image quality in CTA. We introduce a measurable and reproducible image quality index, CTA QI. Using this index, along with calculated effective dose, we test the hypothesis that volume scan CTA delivers comparable image quality at substantially reduced radiation dose when compared to helical CTA. CTA QI is a measure of intraluminal contrast variation in three-dimensions, and it is calculated from standardized measurements of means and standard deviations of Hounsfield units in the thoracic descending aorta. From institutional database, 83 studies of CTA for thoracic cardiovascular diseases were retrospectively identified. CTA QI values were independently measured by two radiologists and compared using correlation. CTA QI and DLP-derived effective dose were compared for the following groups: non-cardiac gated wide-detector and helical CTA, ECG-synchronized retrospective wide-detector and helical CTA, ECG-synchronized wide detector retrospective and target technique CTA. Statistical significance was evaluated with the Student-t test. The correlations of CTA QI values between the radiologists were 0.83 and 0.92 for non-gated studies and ECG-synchronized studies respectively. Comparing non-gated volume scan CTA to helical CTA, there was a radiation dose reduction of 69% (P < 0.0001) without a significant change in CTA QI (1.4 ± 1.0 vs. 1.9 ± 1.4, P = 0.13). Comparing retrospective ECG-synchronized wide-detector CTA to helical CTA, there was a radiation dose reduction of 46% (P < 0.0001) with and improvement in CTA QI (1.0 ± 0.8 vs. 3.7 ± 3.4, P < 0.01). Comparing ECG-synchronized wide-detector target CTA to retrospective CTA, there was a radiation dose reduction of 68% (P < 0.0001, but at the cost of a significant reduction in CTA QI (2.0 ± 1.0 vs. 0.8 ± 0.4, P < 0.0044). CTA QI is a simple, reproducible metric of image quality suited for comparing CTA studies. Using this quality index, we establish that CTA performed with wide-detector scan techniques can yield substantially lower radiation dose without compromising diagnostic imaging quality. A wide-detector target technique can further reduce effective dose compared to wide-detector retrospective ECG-synchronization, but with a reduction in image quality.

To evaluate the efficiency of CT angiography (CTA) with 16-row MSCT compared with MR angiography (MRA) in analyzing the arterial anatomy in patients undergoing liver surgery. In 30 patients, MRA and CTA studies of the abdominal vessels were reviewed. CT parameters: slice thickness 3 mm; collimation 1.5; reconstruction interval 2 mm (Philips MX 8000 IDT); 120 ml contrast media (400 mg/ml) at a rate of 4 ml/sec; acquisition of arterial-phase scans. The anatomy of the hepatic artery was evaluated from axial and reconstructed maximum-intensity-projection (MIP) images ("Slab-Viewer", Philips). MR parameters: contrast-enhanced coronal FLASH-3D sequences; slice thickness 1.4 mm; TR 3.47, TE 1.3; 1.5 T scanner (Siemens Somatom). Image quality was rated with a scoring system. Contrast enhancement of the hepatic artery and the liver parenchyma was measured. The image quality of CTA was rated as excellent in 18 (MRA 5); good in 8 (MRA 22); satisfactory in 4 patients (MRA 3), and non-diagnostic in 0 patient (MRA 1). Compared to MRA, the image quality of CTA was better in 15/30 patients; equal for both in 13 and worse in 2 patients. CTA provided a better depiction of the segmental branches of the hepatic arteries in 15/30 patients and revealed important anatomic variations of the hepatic artery in 8/30 patients. These variations were not be seen in MRA: e. g., MRA missed a left gastric arterial supply to the left liver. The ratio of contrast enhancement in liver parenchyma and hepatic artery was 4.7 in CTA and 4.5 in MRA. CTA with multislice scanners delivers better image quality and depiction of the hepatic arteries than MRA. Thus, MRI of the hepatic arteries can be replaced by routine CT, which is already part of the preoperative evaluation for liver transplantation.

Background suppression single-shot electrocardiogram trigger non-contrast-enhanced magnetic resonance angiography (BASS-TRANCE) is a recently introduced non-contrast-enhanced magnetic resonance angiography (NCE-MRA) technique. It was reported to be a practical test for the evaluation of peripheral artery disease (PAD) with an imaging performance comparable to that of the known NCE-MRA technique quiescent interval single-shot (QISS). However, its performance compared to the most commonly used diagnostic methods for digital subtraction angiography (DSA) and computed tomography angiography (CTA) remains unclear. Using DSA as the test standard, this study evaluated the image quality and clinical diagnostic accuracy of BASS-TRANCE and CTA in PAD patients. BASS-TRANCE, CTA, and DSA were examined successively in 30 patients with PAD. Two senior physicians scored the image quality of CTA and BASS-TRANCE using the 3-point method, and the images of both were evaluated according to the 5-level stenosis evaluation method. Either paired t-test or Wilcoxon signed-rank test was used to compare the difference of image quality between the two groups. Using DSA results as the gold standard, the sensitivity and specificity of CTA and BASS-TRANCE for lower limb artery stenosis >50% were calculated by McNemar's test. Of 570 segments, 12 (2.1%) and 42 (7.4%) inconclusive segments were excluded from BASS-TRANCE and CTA analysis, respectively (P<0.05). The DSA results were available for 392 of the remaining segments. Among the 516 vessels with reliable image quality, there was no significant difference in scores between BASS-TRANCE {2.42 [95% confidence interval (CI): 2.36-2.47]} and CTA [2.39 (95% CI: 2.33-2.45); P>0.05]. Furthermore, BASS-TRANCE demonstrated significant efficacy in detecting vascular stenosis with a sensitivity of 92.8% and specificity of 96.1%, which were comparable to those achieved by CTA (94.9% and 96.8%, respectively). As an NCE-MRA technique, BASS-TRANCE can display the lower extremity vessels in a short time; it is expected to replace CTA as a safe and effective examination method for PAD patients in the future.

The sensitivity of wide parameter-space searches for continuous gravitational waves (CWs) is limited by their high computational cost. Deep learning is being studied as an alternative method to replace various aspects of a CW search. In previous work [], new design principles were presented for deep neural network (DNN) search of CWs and such DNNs were trained to perform a targeted search with matched-filtering sensitivity. In this paper, we adapt these design principles to build a DNN architecture for wide parameter-space searches in 10 days of data from two detectors (H1 and L1). We train a DNN for each of the benchmark cases: six all-sky searches and eight directed searches at different frequencies in the search band of 20–1000 Hz. We compare our results to the DNN sensitivity achieved from Dreissigacker and Prix [] and find that our trained DNNs are more sensitive in all the cases. The absolute improvement in detection probability ranges from 6.5% at 20 Hz to 38% at 1000 Hz in the all-sky cases and from 1.5% at 20 Hz to 59.4% at 500 Hz in the directed cases. An all-sky DNN trained on the entire search band of 20–1000 Hz shows a high sensitivity at all frequencies, providing a proof of concept for training a single DNN to perform the entire search. We also study the generalization of the DNN performance to signals with different signal amplitude, frequency, and the dependence of the DNN sensitivity on sky position. Published by the American Physical Society 2024

Abdominal Aortic Aneurysm (AAA) is still a socially relevant public health challenge, evidenced by an 82.1% increase in associated fatalities from 1990 to 2019 (with 172,427 deaths in 2019 alone). In a clinical setting, computed tomography angiography (CTA) is the imaging modality of choice for monitoring and/or presurgical planning of AAA patients. However, manual segmentation of CTA images is labor intensive and time consuming. Hence, there is a growing need for automated segmentation algorithms, particularly when these influence treatment planning. The deep-learning pipeline proposed in this work is designed to automatically segment AAA CTA images. The framework adapted a fully developed patch-based dilated modified U-Net model, which shows remarkable efficiency in accurately delineating AAA regions within the CTA scans. During the prediction phase, the deep learning architecture demonstrates exceptional speed, requiring 17 ± 0.02 milliseconds per frame to generate the final segmented output. Building upon this work, we included the application of Non-Uniform Rational B-Splines (NURBS) to enhance the segmentation process. This advancement is essential in addressing the critical need for clinical accuracy in medical image segmentation. NURBS enables the creation of continuous curves that seamlessly conform to the intricate contours of anatomical structures, offering a significant improvement in segmentation accuracy. Through the integration of advanced deep learning architectures and the precision of NURBS for segmentation refinement, coupled with the fast processing time and accurate segmentation, the the proposed model represents a promising clinical tool that can be used in the clinical management of AAAs.

<b>Objective:</b> To explore the preoperative whole perforator evaluation and intraoperative eccentric design of anterolateral thigh flap (ALTF) based on superficial fascial perforators by modified computed tomography angiography (CTA), and the clinical effects were observed. <b>Methods:</b> A prospective observational study was adopted. Twelve patients with oral and maxillofacial tumors and 10 patients with open injury of the upper limb with large soft tissue defects were hospitalized in the Department of Hand & Microsurgery and Department of Oral & Maxillofacial Surgery of Affiliated Hospital of Binzhou Medical University from January 2021 to July 2022, with 12 males and 10 females, aged from 33 to 75 years, an average age of 56.6 years. The wounds of the patients with oral and maxillofacial tumors were reconstructed by ALTF after the extensive tumor resection and radical cervical lymph node dissection in the same stage; the wounds of the patients with skin and soft tissue defect on the upper limb were covered by ALTF in stage Ⅱ after debridement in stage Ⅰ. After debridement, the area of wound was 3.5 cm×3.5 cm-25.0 cm×10.0 cm and the area of the required flap area was 4.0 cm×4.0 cm-23.0 cm×13.0 cm. Modified CTA scan was performed on the donor site of ALTF before the operation, with the parameters of modified CTA being set to mainly reduce the tube voltage and tube current, and increase the contrast dose and the dual phase scan. The acquired image data were sent to GE AW 4.7 workstation and adopted the volume reconstruction function for visual reconstruction and evaluation of the whole perforator. The information of perforator and source artery was marked on the body surface before operation according to the above evaluation. During the operation, an eccentric flap centered on the visible superficial fascia whole perforator was designed and cut according to the desired flap area and shape. The donor sites of the flap were repaired by direct sutures or full-thickness skin grafts. The total radiation dose was compared between the modified CTA scan and the traditional CTA scan. The distribution of outlet point of perforator of double thighs, the length and direction of superficial fascia perforators based modified CTA were recorded. The type, number, and origin of the target perforator, distribution of of outlet point of perforator, and the diameter, course, and branch of the source artery observed before the operation were compared with those observed during the operation. The healing of donor site wound and the survival of flaps in recipient site were observed after operation. The texture and appearance of flap, oral and upper limb functions, and the functions of femoral donor sites were followed up. <b>Results:</b> The total radiation dose of modified CTA scan was lower than that of the traditional CTA scan. A total of 48 perforators of double thighs were observed, among which, 31 (64.6%) perforators went outward and downward, 9 (18.8%) perforators went inward and downward, 6 (12.5%) perforators went outward and upward, and 2 (4.2%) perforators went inward and upward, and the average length of superficial fascia perforators was 19.94 mm. The preoperative observed type, number, and source of the perforator, the distribution of the outlet point of the perforator, diameter, course, and branches of the source artery were basically consistent with the intraoperative exploration. The types of 15 septocutaneous (including musculoseptocutaneous) perforators and 10 musculocutaneous perforators observed before the operation was consistent with intraoperative exploration. The distance between the mark of the surface perforator point and the actual exit point of the perforator during operation was (0.38±0.11) mm. All flaps survived without vascular crisis. The donor site wounds of 5 cases of skin grafting and 17 cases of direct suturing wounds healed well. The postoperative follow-up was 2 months to 1 year, with an average of 8.2 months, the flaps were soft and slightly bloated; the function of diet and mouth closing was accessible in patients with oral and maxillofacial tumors, the speech function was mildly impaired in patients with tongue cancer, but they could complete basic oral communication; the wrist and elbow joints and forearm rotation function were not significantly limited in patients with upper limb soft tissue injuries; there was no obvious tightness in the donor sites, and the function of the hip and knee joints was not limited. <b>Conclusions:</b> The whole perforator and even the subcutaneous perforator of the donor site of ALTF can be evaluated by modified CTA, and the flap can be used in oral or maxillofacial reconstruction and repair of skin and soft tissue defects of upper limbs to achieve good results. By clarifying the type, number, and source of the perforator, the distribution of the outlet point of the perforator, diameter, course, and branches of the source artery before the operation, the eccentric design of the ALTF based on the superficial fascia perforator was realized. This study has strong guiding value.

Deep learning (DL) is transforming industry as decision-making processes are being automated by deep neural networks (DNNs) trained on real-world data. Driven partly by rapidly-expanding literature on DNN approximation theory showing they can approximate a rich variety of functions, such tools are increasingly being considered for problems in scientific computing. Yet, unlike traditional algorithms in this field, little is known about DNNs from the principles of numerical analysis, e.g., stability, accuracy, computational efficiency and sample complexity. In this paper we introduce a computational framework for examining DNNs in practice, and use it to study empirical performance with regard to these issues. We study performance of DNNs of different widths & depths on test functions in various dimensions, including smooth and piecewise smooth functions. We also compare DL against best-in-class methods for smooth function approx. based on compressed sensing (CS). Our main conclusion from these experiments is that there is a crucial gap between the approximation theory of DNNs and their practical performance, with trained DNNs performing relatively poorly on functions for which there are strong approximation results (e.g. smooth functions), yet performing well in comparison to best-in-class methods for other functions. To analyze this gap further, we provide some theoretical insights. We establish a practical existence theorem, asserting existence of a DNN architecture and training procedure that offers the same performance as CS. This establishes a key theoretical benchmark, showing the gap can be closed, albeit via a strategy guaranteed to perform as well as, but no better than, current best-in-class schemes. Nevertheless, it demonstrates the promise of practical DNN approx., by highlighting potential for better schemes through careful design of DNN architectures and training strategies.

Automatic Detection of Visceral Arterial Aneurysms on Computed Tomography Angiography Using Artificial Intelligence Based Segmentation of the Vascular System

Survey on Deep Neural Networks in Speech and Vision Systems

Recently deep neural networks (DNNs) have attracted a great deal of interest among researchers for speech recognition. DNN training is computationally expensive due to the model large number of parameters. We can improve DNN performance by using pre-training methods. However, DNN learning is hard if pre-training is inefficient. This paper proposes a new pre-training method that utilizes both gender and phoneme information for speech recognition. We use speaker gender information across phoneme information to construct acoustic models more precisely. The new approach named gender-aware deep Boltzmann machine (GADBM) is used for DNN pre-training. GADBM utilizes additional information, which improves recognition accuracy. For this purpose, we have changed the overall structure of deep Boltzmann machine (DBM) to consider additional information. Experimental results on TIMIT dataset show that the proposed method outperforms deep belief network and DBM for phone recognition task. In addition, parameter tuning in the proposed method improves the model performance.

In recent years, magnetized inductively coupled plasma (MICP) has been proposed as an improved version of inductively coupled plasma to meet the increasing production process requirements. However, due to the more complex structure of the plasma system, numerical simulations face challenges such as modeling difficulty, model convergence issues, and long computation times. In this paper, a deep neural network (DNN) with a multi-hidden layer structure is developed based on deep learning technology to serve as an assisted modeling tool of traditional fluid simulations. This approach aims to study the discharge characteristics and plasma chemistry of argon-oxygen MICP more efficiently. The simulation data from the fluid model is used to train the neural network. The well-trained DNN can efficiently and accurately predict the target plasma characteristics under new discharge parameters, such as electron density, ionization rate, and particle reaction rate. The effectiveness of the DNN is verified by comparing its predictions with fluid simulation results. Compared to the traditional fluid simulation, which takes thousands of seconds, the trained DNN can produce highly consistent prediction results only within 1 s. The computing efficiency is significantly improved. The prediction results of the DNN demonstrate that by adjusting discharge parameters such as magnetic field coil current, power, and O2 ratio, the plasma discharge characteristics and plasma chemistry can be modulated, including charged particle density, ionization rate, chemical reaction rate, and plasma radial uniformity. Additionally, the DNN shows good generalization ability across multiple input and output scales, providing a paradigm for its application in other plasma research fields. This study indicates that the DNN is a feasible method for studying MICP discharge characteristics and offers a novel and effective computational tool for future plasma discharge research based on deep learning methods.

&lt;p&gt;Deep neural networks have achieved tremendous success in many domains (e.g., computer vision~\cite{Alexnet12,vggnet15,fastrcnn15}, speech recognition~\cite{hinton2012deep,dahl2012context}, natural language processing~\cite{dahl2012context,collobert2011natural}, games~\cite{silver2017mastering,silver2016mastering}), however, there are still many challenges in deep learning comunity such as how to speed up training large deep neural networks, how to compress large nerual networks for mobile/embed device without performance loss, how to automatically design the optimal network structures for a certain task, and how to further design the optimal networks with improved performance and certain model size with reduced computation cost.&lt;/p&gt; &lt;p&gt;To speed up training large neural networks, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples.&lt;/p&gt; &lt;p&gt;To compress large neural network structures, we propose a simple yet powerful method for compressing the size of deep Convolutional Neural Networks (CNNs) based on parameter binarization. The striking difference from most previous work on parameter binarization/quantization lies at different treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k&gt;1$), where we only binarize $k\times k$ convolutions into binary patterns. By doing this, we show that previous deep CNNs such as GoogLeNet and Inception-type Nets can be compressed dramatically with marginal drop in performance. Second, in light of the different functionalities of $1\times 1$ (data projection/transformation) and $k\times k$ convolutions (pattern extraction), we propose a new block structure codenamed the pattern residual block that adds transformed feature maps generated by $1\times 1$ convolutions to the pattern feature maps generated by $k\times k$ convolutions, based on which we design a small network with $\sim 1$ million parameters. Combining with our parameter binarization, we achieve better performance on ImageNet than using similar sized networks including recently released Google MobileNets.&lt;/p&gt; &lt;p&gt;To automatically design neural networks, we study how to design a genetic programming approach for optimizing the structure of a CNN for a given task under limited computational resources yet without imposing strong restrictions on the search space. To reduce the computational costs, we propose two general strategies that are observed to be helpful: (i) aggressively selecting strongest individuals for survival and reproduction, and killing weaker individuals at a very early age; (ii) increasing mutation frequency to encourage diversity and faster evolution. The combined strategy with additional optimization techniques allows us to explore a large search space but with affordable computational costs.&lt;/p&gt; &lt;p&gt;To further design the optimal networks with improved performance and certain model size under reduced computation cost, we propose an ecologically inspired genetic approach for neural network structure search , that includes two types of succession: primary and secondary succession as well as accelerated extinction. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Accelerated extinction is applied in both stages to reduce computational cost. In addition, we also introduce the gene duplication to further utilize the novel block of layers that appeared in the discovered network structure.&lt;/p&gt;

Forest fires pose a serious threat to the environment with the potential of causing ecological harm, financial losses, and human casualties. While research suggests that climate change will increase the frequency and severity of these fires, recent developments in deep learning and convolutional neural networks (CNN) have greatly enhanced fire detection techniques and capability. These models can be leveraged by unmanned aerial vehicles (UAVs) to automatically monitor burning areas. However, drones can carry only limited computational and power resources; therefore, on-board computing capabilities are constrained by hardware limitations. This work focuses on the design of segmentation models to identify and localize active burning areas from aerial RGB images processed on limited computing resources. To achieve this goal, the research compares the performance of different variants of the DeepLabv3 neural network model for fire segmentation when trained and tested with the FLAME dataset using a k-fold cross validation approach. Experimental results are compared with U-Net, a benchmark model used with the FLAME dataset, by implementing this model in the same codebase as the DeepLabv3 model. This work demonstrates that a refined version of DeepLabv3, with a MobileNetv2 backbone using pretrained layers and a simplified atrous spatial pyramid pooling (ASPP) module, yields a similar performance to U-Net, with a precision of 87.8% and a recall of 83.2%, while only requiring 20% of the number of parameters involved with the U-Net topology. This significantly reduces memory and power consumption, enabling longer UAV flight duration and reducing the processing overhead associated with sensor input, making it more suitable for deployment on unmanned aerial vehicles. The model’s compact architecture, implemented using TensorFlow and Keras for model design and training, along with OpenCV for image preprocessing, makes it portable and easy to integrate with edge devices such as NVIDIA Jetson boards.