ContourDiff: Unpaired Medical Image Translation with Structural Consistency

Many learning problems in computer vision can be posed as structured prediction problems, where the input and output instances are structured objects such as trees, graphs or strings rather than, single labels {+1, -1} or scalars. Kernel methods such as Structured Support Vector Machines, Twin Gaussian Processes (TGP), Structured Gaussian Processes, and vector-valued Reproducing Kernel Hilbert Spaces (RKHS), offer powerful ways to perform learning and inference over these domains. Positive definite kernel functions allow us to quantitatively capture similarity between a pair of instances over these arbitrary domains. A poor choice of the kernel function, which decides the RKHS feature space, often results in poor performance. Automatic kernel selection methods have been developed, but have focused only on kernels on the input domain (i.e.'one-way'). In this work, we propose a novel and efficient algorithm for learning kernel functions simultaneously, on both input and output domains. We introduce the idea of learning polynomial kernel transformations, and call this method Simultaneous Twin Kernel Learning (STKL). STKL can learn arbitrary, but continuous kernel functions, including 'one-way' kernel learning as a special case. We formulate this problem for learning covariances kernels of Twin Gaussian Processes. Our experimental evaluation using learned kernels on synthetic and several real-world datasets demonstrate consistent improvement in performance of TGP's.

BackgroundExisting methods for segmenting liver cancer from single-modal medical images fail to effectively leverage potential correlations across modalities. These correlations between the anatomical structures of liver cancer and the liver are also crucial for accurate liver cancer segmentation. These challenges not only limit the performance and scalability of liver cancer segmentation models but also pose significant challenges for researchers seeking to develop multimodal, low-annotation-dependent solutions. Therefore, it is necessary to propose a universal liver cancer segmentation network for abdominal computed tomography (CT) and magnetic resonance (MR) medical images.MethodBased on the above, we propose a two-stage universal liver cancer segmentation network for 3D dual-modality abdominal nuclear medical images using a mixed-label, multi-type training strategy. In stage 1, two CT and MR liver segmentation models are trained to generate liver mask images for CT and MR multimodal abdominal images without liver mask label images, thereby solving the laborious technical problem of liver labeling. In stage 2, a mixed-label strategy is proposed, where a mixed-label pool is constructed from CT and MR liver mask images generated by the aforementioned liver segmentation models, along with liver label images and their corresponding liver cancer label images. Subsequently, a universal liver cancer segmentation model is trained using a mixed-label, multi-type training strategy that fully considers potential correlations among different medical imaging modalities, liver cancer, and the liver.ResultsThe proposed liver cancer segmentation model, based on Medformer and the proposed mixed-label, multi-type abdominal-image training strategy, performs best, validating the effectiveness of the proposed strategy.DiscussionThe proposed two-stage universal liver cancer segmentation network, based on a mixed-label and multi-type training strategy, can effectively segment liver cancer in different 3D dual-modality abdominal CT and MR images, which may become an indispensable quantitative analysis tool for liver cancer in clinical practice.

Deep learning (DL) has shown unprecedented performance for many image analysis and image enhancement tasks. Yet, solving large-scale inverse problems like tomographic reconstruction remains challenging for DL. These problems involve non-local and space-variant integral transforms between the input and output domains, for which no efficient neural network models are readily available. A prior attempt to solve tomographic reconstruction problems with supervised learning relied on a brute-force fully connected network and only allowed reconstruction with a 1284 system matrix size. This cannot practically scale to realistic data sizes such as 5124 and 5126 for three-dimensional datasets. Here we present a novel framework to solve such problems with DL by casting the original problem as a continuum of intermediate representations between the input and output domains. The original problem is broken down into a sequence of simpler transformations that can be well mapped onto an efficient hierarchical network architecture, with exponentially fewer parameters than a fully connected network would need. We applied the approach to computed tomography (CT) image reconstruction for a 5124 system matrix size. This work introduces a new kind of data-driven DL solver for full-size CT reconstruction without relying on the structure of direct (analytical) or iterative (numerical) inversion techniques. This work presents a feasibility demonstration of full-scale learnt reconstruction, whereas more developments will be needed to demonstrate superiority relative to traditional reconstruction approaches. The proposed approach is also extendable to other imaging problems such as emission and magnetic resonance reconstruction. More broadly, hierarchical DL opens the door to a new class of solvers for general inverse problems, which could potentially lead to improved signal-to-noise ratio, spatial resolution and computational efficiency in various areas.

Introduction: Protein conformational switches are often constructed by fusing an input domain, which recognizes a target ligand, to an output domain that establishes a biological response. Prior designs have employed binding-induced folding of the input domain to drive a conformational change in the output domain. Adding a second input domain can in principle harvest additional binding energy for performing useful work. It is not obvious, however, how to fuse two binding domains to a single output domain such that folding of both binding domains combine to effect conformational change in the output domain.Methods: Here, we converted the ribonuclease barnase (Bn) to a switchable enzyme by duplicating a C-terminal portion of its sequence and appending it to its N-terminus, thereby establishing a native fold (OFF state) and a circularly permuted fold (ON state) that competed for the shared core in a mutually exclusive fashion. Two copies of FK506 binding protein (FKBP), both made unstable by the V24A mutation and one that had been circularly permuted, were inserted into the engineered barnase at the junctions between the shared and duplicated sequences.Results: Rapamycin-induced folding of FK506 binding protein stretched and unfolded the native fold of barnase via the mutually exclusive folding effect, and rapamycin-induced folding of permuted FK506 binding protein stabilized the permuted fold of barnase by the loop-closure entropy principle. These folding events complemented each other to turn on RNase function. The cytotoxic switching mechanism was validated in yeast and human cells, and in vitro with purified protein.Discussion: Thermodynamic modeling and experimental results revealed that the dual action of loop-closure entropy and mutually exclusive folding is analogous to an engine transmission in which loop-closure entropy acts as the low gear, providing efficient switching at low ligand concentrations, and mutually exclusive folding acts as the high gear to allow the switch to reach its maximum response at high ligand concentrations.

Case series. To describe 4 cases of unstable subaxial cervical spine injury not demonstrated with initial radiograph, computed tomography (CT), and magnetic resonance (MR) imaging. When evaluating the cervical spine for ligamentous or osseous injuries in the blunt trauma patient population, negative predictive value measurements of 100% for CT and MR imaging have been published. Unstable subaxial cervical spine injury has rarely been reported in the spine literature in conjunction with initial radiograph, CT, and MR imaging demonstrating no osseous or ligamentous injury. Historically, reports of subacute cervical spine instability following trauma exist and were presented before the availability of MR and multidetector CT imaging. We report 4 examples of unstable subaxial cervical spine injury each with initial imaging interpreted as negative. All 4 cases presented at a level-one tertiary care facility. Follow-up radiographs demonstrated unstable cervical spine injuries requiring surgical stabilization. Notwithstanding high sensitivities, specificities, and negative predictive values for cervical spine imaging and "clearance" mechanisms, 4 cases that illustrate the potential for undetected unstable cervical spine injuries are presented. Tremendous advancements in medical imaging have been made. However, radiograph, CT, and MR imaging may still fail to accurately translate the anatomic and dynamic complexity of the cervical spine into digital images that accurately guide clinical practice. A full understanding of and keen appreciation for the fact that no imaging technique, classification method, or clearance protocol can produce 100% sensitivity at all times is essential. These case reports cumulatively demonstrate a 0.04% to 0.2% incidence of undetected cervical injury requiring surgical stabilization. Removing cervical collar immobilization as quickly as possible based on negative interpretation of imaging data may prove harmful in a measurable percentage of patients and must be undertaken with caution only after clinical correlation and strict follow-up is established.

Computed Tomography (CT) images play a crucial role in medical diagnosis and treatment planning. However, acquiring CT images can be difficult in certain scenarios, such as patients inability to undergo radiation exposure or unavailability of CT scanner. An alternative solution can be generating CT images from other imaging modalities. In this work, we propose a medical image translation pipeline for generating high-quality CT images from Positron Emission Tomography (PET) images using a Pix2Pix Generative Adversarial Network (GAN), which are effective in image translation tasks. However, traditional GAN loss functions often fail to capture the structural similarity between generated and target image. To alleviate this issue, we introduce a Multi-Scale Structural Similarity Index Measure (MS-SSIM) loss in addition to the GAN loss to ensure that the generated images preserve the anatomical structures and patterns present in the real CT images. Experiments on the ‘QIN-Breast’ dataset demonstrate that our proposed architecture achieves a Peak Signal-to-Noise Ratio (PSNR) of 17.70 dB and a Structural Similarity Index Measure (SSIM) of 42.51% in the region of interest.

A primary issue in black-box testing is how to generate adequate test cases from input domain of the system under test on the basis of user's requirement specification. However, for some types of systems including embedded systems, developing test cases from output domain is more suitable than developing from input domain, especially, when the output domain is smaller. This approach ensures better reliability of the system under test. In this paper, the authors present a new approach to automate the generation of test cases from output domain of a pilot project "Temperature Monitoring and Controlling of Nuclear Reactor System" (TMCNRS) which is an embedded system developed using modified Cleanroom Software Engineering methodology. An Automated Test Case Generator (ATCG) that uses Genetic algorithms (GAs) extensively and generates test cases from output domain is proposed. The ATCG generates test cases which are useful to conduct pseudo - exhaustive testing to detect single, double and several multimode faults in the system. The generator considers most of the combinations of outputs, and finds the corresponding inputs while optimizing the number of test cases generated. In order to investigate the effectiveness of this approach, test cases were generated by ATCG and the tests were conducted on the target embedded system at a minimum cost and time. Experimental results show that this approach is very promising.

Radiotherapy treatment for prostate cancer relies on computed tomography (CT) and/or magnetic resonance imaging (MRI) for segmentation of target volumes and organs at risk (OARs). Manual segmentation of these volumes is regarded as the gold standard for ground truth in machine learning applications, but to acquire such data is tedious and time-consuming. A publicly available clinical dataset is presented, comprising MRI- and synthetic CT (sCT) images, target and OARs segmentations, and radiotherapy dose distributions for 432 prostate cancer patients treated with MRI-guided radiotherapy. An extended dataset with 35 patients is also included, with the addition of deep learning (DL)-generated segmentations, DL segmentation uncertainty maps, and DL segmentations manually adjusted by four radiation oncologists. The publication of these resources aims to aid research in automated radiotherapy treatment planning, segmentation, inter-observer analyses, and DL model uncertainty investigation. The dataset is hosted on the AIDA Data Hub and offers a free-to-use resource for the scientific community, valuable for the advancement of medical imaging and prostate cancer radiotherapy research.

Objectives:Rotator cuff tears (RCT) are common injuries that increase in frequency with age. Deep learning is a subfield of artificial intelligence that uses artificial neural networks to perform a variety of tasks, including image analysis. Several previous studies have investigated the utility of deep learning for identifying fatty infiltration, muscle atrophy, and tear size of the supraspinatus muscle on shoulder magnetic resonance imaging (MRI). However, limited data exists on the use of deep learning for image analysis to determine the presence of supraspinatus pathology versus normal anatomy on shoulder MRI. RadImageNet, a recently published database consisting of 1.35 million annotated computed tomography (CT), MRI, and ultrasound (US) images from greater than 130,000 patients, contains thousands of shoulder MRI images. No research to date has investigated whether deep learning algorithms can accurately detect supraspinatus pathology on MRIs in the RadImageNet database.The objective of the present study was to determine if deep learning algorithms can accurately differentiate shoulder MRI images as those with supraspinatus pathology versus normal anatomy in the RadImageNet database.Methods:The RadImageNet database was queried for all MRI images containing supraspinatus pathology. Both supraspinatus tears (partial and full thickness) and tendinosis images are categorized under the same “supraspinatus pathology” label in the database. In the comparison cohort with normal anatomy, coronal and sagittal images capturing the supraspinatus muscle were included. Although the supraspinatus was still visualized, all sagittal images medial to the glenoid were excluded to better represent the supraspinatus pathology images. The training/validation/test datasets were created using a random 80/10/10 split of the entire cohort. Data were pre-processed using standard data augmentation, including random cropping, rotation, and translation of images. ConvNeXt, a pre-trained deep learning model for the purposes of image analysis, was chosen as the base architecture for transfer learning to optimize both accuracy and computing speed. The ConvNeXt model was trained using training (N=4,937 images) and validation (N=619 images) sets for 15 epochs. Algorithm performance was assessed on an independent test cohort (N=619 images) not used during model training. Performance metrics included classification accuracy, sensitivity, specificity, and positive and negative predictive values. Model training was performed in PyTorch on an 8-gigabyte NIVIDIA Quadro M4000 Cloud Graphics Processing Unit.Results:A total of 5,241 shoulder MRIs with supraspinatus pathology and 932 without any pathology were obtained for comparison. The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of the classifier on the independent test set was 98.9% (95% confidence interval: 97.7% – 99.5%), 98.7%, 100.0%, 100.0%, and 93.1%, respectively. A confusion matrix for model predictions on the test set is displayed in Figure 1. Interpretation plots displaying six examples of incorrect model predictions are shown in Figure 2. Class 1 and 2 designate images without pathology and those with supraspinatus pathology, respectively. All six images were cases where the model predicted “normal” when the ground truth label of the image was supraspinatus pathology. Figure 3 demonstrates an example of class activation mapping, which visually indicates the areas on an image that influenced the model prediction the most. Bright yellow coloration indicates highly influential regions whereas purple coloration corresponds to regions that were less influential. In Figure 3, which is an example image of supraspinatus pathology, one of the most influential regions appears to localize to the supraspinatus tendon.Conclusions:Deep learning is a viable method for differentiating supraspinatus pathology from normal anatomy on shoulder MRIs. The ConvNeXt model demonstrated excellent performance in predicting images with supraspinatus pathology on shoulder MRIs in the RadImageNet database. Future research using deep learning techniques to classify different rotator cuff tear types and sizes is warranted.

CS-GAN: Cross-Structure Generative Adversarial Networks for Chinese calligraphy translation

Small hepatocellular carcinomas in cirrhosis: differences in contrast enhancement effects between helical CT and MR imaging during multiphasic dynamic imaging

In medical image processing, image fusion is the process of combining complementary information from different or multimodality images to obtain an informative fused image in order to improve clinical diagnostic accuracy. In this paper, we propose a two‐stage fusion framework for computed tomography (CT) and magnetic resonance (MR) images. First, the intensity and geometric structure features in both CT and MR images are extracted by the saliency detection method and structure tensor, respectively, and an initial fused image is obtained. Then, the initial fused image is optimized by a variational model which contains a fidelity term and a regularization term. The fidelity term is to retain the intensity of the initial fused image, and the regularization term is to constrain the gradient information of the fused image to approximate the MR image. The primal‐dual algorithm is proposed to solve the variational problem. The proposed method is applied on five pairs of clinical medical CT and MR‐T1\\MR‐T2 images, and the comparison metrics SF, MI,QAB/F,QW, and VIFF are calculated for assessment. Compared with seven state‐of‐the‐art methods, the proposed method shows a comprehensive advantage in preserving the salient intensity features, as well as texture structure information, not only in visual effects but also in objective assessments.

BI&T Tell me about the Medical Imaging and Technology Alliance. David Fisher The Medical Imaging & Technology Alliance (MITA), a division of the National Electrical Manufacturers Association (NEMA), is the leading organization and collective voice of medical imaging equipment manufacturers, innovators, and product developers. It represents companies whose sales comprise more than 90 percent of the global market for medical imaging technology. The goals of MITA are to: • Increase the awareness and understanding of the value of medical imaging • Achieve efficient and reasonable regulation of medical imaging technologies • Interact with appropriate government agencies on reimbursement and technology assessment policies • Expand the global acceptance of the digital communications standard (DICOM) that allows digital imaging technologies to interact seamlessly • Improve regulatory harmonization of the global market for medical imaging products • Develop and represent industry positions in technical, trade and other issues • Provide market data unique to this industry

To determine retrospectively whether unenhanced computed tomographic (CT) images of the brain have added value in distinguishing tumefactive demyelinating lesions (TDLs) from primary glioma or central nervous system (CNS) lymphoma, compared with conventional contrast material-enhanced magnetic resonance (MR) images only. This study was approved by the institutional review board, and informed consent was waived. Unenhanced CT and MR images in 15 patients with TDLs (seven women, eight men; mean age, 42 years; range, 27-57 years) and 48 patients with primary brain tumor (27 women, 21 men; mean age, 48 years; range, 19-70 years; 10 lymphomas, 38 gliomas) were retrospectively reviewed. The CT attenuation of regions that were enhanced or unenhanced at MR imaging was visually categorized into three grades, and CT attenuation values were determined quantitatively. The diagnostic accuracy of MR imaging for differentiating TDLs from tumors was compared with that of MR imaging plus CT. The following MR imaging features were found more frequently in patients with TDL than in those with brain tumor: incomplete rim enhancement, mixed T2-weighted iso- and hyperintensity of enhanced regions, absence of a mass effect, and absence of cortical involvement (all P values < .05). CT hypoattenuation of MR enhanced regions was observed in 14 (93%) of 15 patients with TDL but in only two (4%) of 48 patients with tumor. The CT attenuation of MR enhanced regions was significantly lower for patients with TDL than for those with tumor (P < .001). The diagnostic accuracy of CT plus MR imaging was significantly higher than that of MR imaging alone (97% vs 73.0%, respectively; P < .001), and the diagnostic accuracy of CT was significantly higher than that of unenhanced T1-weighted MR imaging (95% vs 63%, P < .001). Unenhanced CT plus MR imaging was more accurate for distinguishing TDLs from glioma or CNS lymphoma than contrast-enhanced MR imaging alone.

Synthesis of a large set of high-quality medical images with variability in anatomical representation and image appearance has the potential to provide solutions for tackling the scarcity of properly annotated data in medical image analysis research. In this paper, we propose a novel framework consisting of image segmentation and synthesis based on mask-conditional GANs for generating high-fidelity and diverse Cardiac Magnetic Resonance (CMR) images. The framework consists of two modules: i) a segmentation module trained using a physics-based simulated database of CMR images to provide multi-tissue labels on real CMR images, and ii) a synthesis module trained using pairs of real CMR images and corresponding multi-tissue labels, to translate input segmentation masks to realistic-looking cardiac images. The anatomy of synthesized images is based on labels, whereas the appearance is learned from the training images. We investigate the effects of the number of tissue labels, quantity of training data, and multi-vendor data on the quality of the synthesized images. Furthermore, we evaluate the effectiveness and usability of the synthetic data for a downstream task of training a deep-learning model for cardiac cavity segmentation in the scenarios of data replacement and augmentation. The results of the replacement study indicate that segmentation models trained with only synthetic data can achieve comparable performance to the baseline model trained with real data, indicating that the synthetic data captures the essential characteristics of its real counterpart. Furthermore, we demonstrate that augmenting real with synthetic data during training can significantly improve both the Dice score (maximum increase of 4%) and Hausdorff Distance (maximum reduction of 40%) for cavity segmentation, suggesting a good potential to aid in tackling medical data scarcity.