OpenVocabCT: Towards Universal Text-driven CT Image Segmentation

Detection of metastasis of mediastinal lymph nodes in lung cancer patients with an artificial intelligence model.

Automated segmentation of liver and lesion in contrast-enhanced abdominal computed tomography (CT) scans is a crucial step for computer-assisted diagnosis and surgical planning of liver diseases. Although deep convolutional neural networks (DCNNs) have contributed many breakthroughs in liver and lesion segmentation task from CT images, the problem remains challenging due to low segmentation accuracy and high complexity. To address these issues, we proposed a light-weight convolutional neural networks for liver and lesion segmentation from CT images, i.e., LW-Mnet. Firstly, we improve the basic module of MobileNetv3 as the backbone of LW-Mnet. The second is that we employ LWASPP to capture multi-scale feature information for enhance representation power of network. The proposed network is evaluated on the public LiTS dataset, and the results show that LW-Mnet achieved the better performance than other networks with fewer parameters and faster inference speed.

Computer Aided Diagnosis of liver tumors from abdominal Computer Tomography (CT) images requires segmentation of tumor. Automatic segmentation of tumor from CT images is a challenging task due in part to the variation of size, shape, and position of the tumor and on the other part to the presence of other objects with the same intensity in the CT images. Therefore, it is necessary to segment the liver first so that tumor can then be segmented accurately. This paper presents a new method for automatic segmentation of liver and tumor from CT images. The proposed method uses combination of intensity analysis, region-growing and preprocessing steps for automatic segmentation of liver and a second region growing process for tumor segmentation.

Lung cancer is one of the most prevalent and lethal malignant tumors globally, characterized by high morbidity and mortality rates. Early diagnosis is crucial for enhancing treatment efficacy, reducing mortality, and improving patients’ quality of life. Traditional manual interpretation of computed tomography (CT) images—the primary screening tool for lung cancer—is often limited by subjectivity, heavy workload, and potential missed diagnoses. Artificial Intelligence (AI) methodologies have emerged as indispensable assets for radiologists, empowering them to conduct in-depth analyses of lung CT scans. This technology facilitates a more streamlined and unbiased evaluation of anatomical structures and pathological features depicted in chest CT images. Driven by advances in deep learning models such as convolutional neural networks (CNNs), the accuracy and robustness of AI methods for lung cancer diagnosis have been significantly improved, providing reliable references for clinical practice. This paper presents a brief review of the applications of AI methods in lung cancer diagnosis based on CT imaging in recent years. It systematically organizes and elaborates on the applications of AI methods represented by deep learning in pulmonary nodule detection, lung tumor segmentation, benign-malignant classification of lung cancer, and lung cancer-related gene mutation detection. Additionally, it analyzes the performance of different models on relevant datasets. Furthermore, the limitations of AI in CT imaging-based lung cancer diagnosis and future research directions are summarized. Through this survey, we aim to provide a concise and comprehensive overview of the current status of lung cancer diagnosis using CT-AI integrated systems, and ultimately promote the development and clinical application of these technologies in the early and accurate detection of lung cancer.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Since deep learning models have been successfully used in many fields, they have been used to identify sick and healthy people in X-ray or Computed Tomography (CT) chest radiology images. In this study, Covid-19 and pneumonia classification is performed on both X-ray and CT images using the robust Convolutional Neural Network (CNN). BGR, HSV, and CIE LAB color space transformations are applied to X-ray and CT images to show that the model performs a successful classification independent of dataset characteristics. The binary classification accuracy rates of Covid-19 and pneumonia for X-ray images and CT images are 98.7% and 98.4%, 97.6% and 99.4%, respectively. Precision, Recall, Specificity, F1 score, and Mean Squared Error metrics are calculated for each X-ray and CT dataset. In addition, 5-fold cross-validation proved accuracy of the model. Although X-ray and CT chest radiology images are transformed into different color spaces, the proposed model performed a successful classification. Thus, even if the image characteristics of the radiology device brands change, the computer-based system will be able to make successful disease diagnoses at low cost where expert personnel are insufficient.

Use Of Deep Learning To Predict Tumor Response To Neoadjuvant Therapy In Pancreatic Adenocarcinoma: Pure And Hybrid Modelling

BackgroundWith the rapid advances in artificial intelligence—particularly convolutional neural networks—researchers now exploit CT, PET/CT and other imaging modalities to predict epidermal growth factor receptor (EGFR) mutation status in non-small-cell lung cancer (NSCLC) non-invasively, rapidly and repeatably. End-to-end deep-learning models simultaneously perform feature extraction and classification, capturing not only traditional radiomic signatures such as tumour density and texture but also peri-tumoural micro-environmental cues, thereby offering a higher theoretical performance ceiling than hand-crafted radiomics coupled with classical machine learning. Nevertheless, the need for large, well-annotated datasets, the domain shifts introduced by heterogeneous scanning protocols and preprocessing pipelines, and the “black-box” nature of neural networks all hinder clinical adoption. To address fragmented evidence and scarce external validation, we conducted a systematic review to appraise the true performance of deep-learning and radiomics models for EGFR prediction and to identify barriers to clinical translation, thereby establishing a baseline for forthcoming multicentre prospective studies.MethodsFollowing PRISMA 2020, we searched PubMed, Web of Science and IEEE Xplore for studies published between 2018 and 2024. Fifty-nine original articles met the inclusion criteria. QUADAS-2 was applied to the eight studies that developed models using real-world clinical data, and details of external validation strategies and performance metrics were extracted systematically.ResultsThe pooled internal area under the curve (AUC) was 0.78 for radiomics–machine-learning models and 0.84 for deep-learning models. Only 17 studies (29%) reported independent external validation, where the mean AUC fell to 0.77, indicating a marked domain-shift effect. QUADAS-2 showed that 31% of studies had high risk of bias in at least one domain, most frequently in Index Test and Patient Selection.ConclusionAlthough deep-learning models achieved the best internal performance, their reliance on single-centre data, the paucity of external validation and limited code availability preclude their use as stand-alone clinical decision tools. Future work should involve multicentre prospective designs, federated learning, decision-curve analysis and open sharing of models and data to verify generalisability and facilitate clinical integration.

Accurate kidney and tumor segmentation of computed tomography (CT) scans is vital for diagnosis and treatment, but manual methods are time-consuming and inconsistent, highlighting the value of AI automation. This study develops a fully automated AI model using vision transformers (ViTs) and convolutional neural networks (CNNs) to detect and segment kidneys and kidney tumors in Contrast-Enhanced (CECT) scans, with a focus on improving sensitivity for small, indistinct tumors. The segmentation framework employs a ViT-based model for the kidney organ, followed by a 3D UNet model with enhanced connections and attention mechanisms for tumor detection and segmentation. Two CECT datasets were used: a public dataset (KiTS23: 489 scans) and a private institutional dataset (Private: 592 scans). The AI model was trained on 389 public scans, with validation performed on the remaining 100 scans and external validation performed on all 592 private scans. Tumors were categorized by TNM staging as small (≤4 cm) (KiTS23: 54%, Private: 41%), medium (>4 cm to ≤7 cm) (KiTS23: 24%, Private: 35%), and large (>7 cm) (KiTS23: 22%, Private: 24%) for detailed evaluation. Kidney and kidney tumor segmentations were evaluated against manual annotations as the reference standard. The model achieved a Dice score of 0.97 ± 0.02 for kidney organ segmentation. For tumor detection and segmentation on the KiTS23 dataset, the sensitivities and average false-positive rates per patient were as follows: 0.90 and 0.23 for small tumors, 1.0 and 0.08 for medium tumors, and 0.96 and 0.04 for large tumors. The corresponding Dice scores were 0.84 ± 0.11, 0.89 ± 0.07, and 0.91 ± 0.06, respectively. External validation on the private data confirmed the model's effectiveness, achieving the following sensitivities and average false-positive rates per patient: 0.89 and 0.15 for small tumors, 0.99 and 0.03 for medium tumors, and 1.0 and 0.01 for large tumors. The corresponding Dice scores were 0.84 ± 0.08, 0.89 ± 0.08, and 0.92 ± 0.06. The proposed model demonstrates consistent and robust performance in segmenting kidneys and kidney tumors of various sizes, with effective generalization to unseen data. This underscores the model's significant potential for clinical integration, offering enhanced diagnostic precision and reliability in radiological assessments.

Cancer treatment has greatly benefited from advancements in radiopharmaceutical therapy, which requires precise dosimetry to enhance therapeutic efficacy and minimize risks to healthy tissues. This review investigated the role of artificial intelligence (AI) in theranostic radiopharmaceutical dosimetry, focusing on image quality enhancement, dose estimation, and organ segmentation. An in-depth review of the literature was conducted using targeted keywords searches in Google Scholar, PubMed, and Scopus. Selected studies were evaluated for their methodologies and outcomes. Traditional dosimetry techniques such as organ-level and voxel-based methods are discussed. Deep learning (DL) models based on U-Net, generative adversarial networks, and hybrid transformer networks for image synthesis and generation, image quality improvement, organ segmentation, and radiation dose estimation are reviewed and discussed. While DL shows great potential for enhancing dosimetry accuracy and efficiency, challenges such as the need for accurate dose estimation from theranostic pairs, lack of imaging data, and modeling of radionuclide decay chains must be addressed using DL models. In addition, the optimization and standardization of DL and AI models is crucial for ensuring clinical reliability and should be given high priority to support their effective integration into clinical practice.

Aims: This study aimed to explore the effect of training set diversity on the performance of deep learning models for predicting early gastric cancer (EGC) under endoscopy. Methods: Images of EGC and non-cancerous lesions under narrow-band imaging (ME-NBI) and magnifying blue laser imaging (ME-BLI) were retrospectively collected. Training set 1 was composed of 150 non-cancerous and 309 EGC ME-NBI images, training set 2 was composed of 1505 non-cancerous and 309 EGC ME-BLI images, and training set 3 was the combination of training set 1 and 2. Test set 1 was composed of 376 non-cancerous and 1052 EGC ME-NBI images, test set 2 consisted of 529 non-cancerous and 71 EGC ME-BLI images, and test set 3 was the combination of test set 1 and test set 2. Three deep learning models, convolutional neural network (CNN) 1, CNN 2 and CNN 3 (CNN 1, CNN 2 and CNN 3 were independently trained using training set 1, training set 2 and training set 3, respectively), were constructed, and their performances on each test set were respectively evaluated. One hundred and thirty-eight ME-NBI videos and 17 ME-BLI videos were further collected to evaluate and compare the performance of each model in real time. Results: On the whole, the performance of CNN 3 was the best. The accuracy (Acc), sensitivity (Sn), specificity (Sp) and area under the curve (AUC) of test set 1 in CNN 3 were 87.89% (1255/1428), 90.96% (342/376), 86.79% (913/1052) and 94.60%, respectively. The Acc, Sn, Sp and AUC of test set 2 in CNN 3 were 95% (570/600), 97.92% (518/529), 73.24% (52/71) and 90.93% respectively. The Acc, Sn, Sp and AUC of test set 3 in CNN 3 were 89.99% (1825/2028), 95.03% (860/905), 85.93% (965/1123) and 94.89%, respectively. The performance of CNN 3 was also the best in videos test set. The Acc, Sn and Sp of videos test set in CNN 3 were 91.03% (142/156), 90.58% (125/138) and 94.44% (17/18), respectively. Conclusions: The deep learning model with the most diverse training data has the best diagnostic effect.

Contrastive learning, a successful form of representational learning, has shown promising results in pretraining deep learning (DL) models for downstream tasks. When working with limited annotation data, as in medical image segmentation tasks, learning domain-specific local representations can further improve the performance of DL models. In this work, we extend the contrastive learning framework to utilize domain-specific contrast information from unlabeled Magnetic Resonance (MR) images to improve the performance of downstream MR image segmentation tasks in the presence of limited labeled data. The contrast in MR images is controlled by underlying tissue properties (e.g., T1 or T2) and image acquisition parameters. We hypothesize that learning to discriminate local representations based on underlying tissue properties should improve subsequent segmentation tasks on MR images. We propose a novel constrained contrastive learning (CCL) strategy that uses tissue-specific information via a constraint map to define positive and negative local neighborhoods for contrastive learning, embedding this information in the representational space during pretraining. For a given MR contrast image, the proposed strategy uses local signal characteristics (constraint map) across a set of related multi-contrast MR images as a surrogate for underlying tissue information. We demonstrate the utility of the approach for downstream: (1) multi-organ segmentation tasks in T2-weighted images where a DL model learns T2 information with constraint maps from a set of 2D multi-echo T2-weighted images (n=101) and (2) tumor segmentation tasks in multi-parametric images from the public brain tumor segmentation (BraTS) (n=80) dataset where DL models learn T1 and T2 information from multi-parametric BraTS images. Performance is evaluated on downstream multi-label segmentation tasks with limited data in (1) T2-weighted images of the abdomen from an in-house Radial-T2 (Train/Test=30/20), (2) public Cartesian-T2 (Train/Test=6/12) dataset, and (3) multi-parametric MR images from the public brain tumor segmentation dataset (BraTS) (Train/Test=40/50). The performance of the proposed CCL strategy is compared to state-of-the-art self-supervised contrastive learning techniques. In each task, a model is also trained using all available labeled data for supervised baseline performance. The proposed CCL strategy consistently yielded improved Dice scores, Precision, and Recall metrics, and reduced HD95 values across all segmentation tasks. We also observed performance comparable to the baseline with reduced annotation effort. The t-SNE visualization of features for T2-weighted images demonstrates its ability to embed T2 information in the representational space. On the BraTS dataset, we also observed that using an appropriate multi-contrast space to learn T1+T2, T1, or T2 information during pretraining further improved the performance of tumor segmentation tasks. Learning to embed tissue-specific information that controls MR image contrast with the proposed constrained contrastive learning improved the performance of DL models on subsequent segmentation tasks compared to conventional self-supervised contrastive learning techniques. The use of such domain-specific local representations could help understand, improve performance, and mitigate the scarcity of labeled data in MR image segmentation tasks.

Automatic segmentation of hepatic lesions in computed tomography (CT) images is a challenging task to perform due to heterogeneous, diffusive shape of tumors and complex background. To address the problem more and more researchers rely on assistance of deep convolutional neural networks (CNN) with 2D or 3D type architecture that have proven to be effective in a wide range of computer vision tasks, including medical image processing. In this technical report, we carry out research focused on more careful approach to the process of learning rather than on complex architecture of the CNN. We have chosen MICCAI 2017 LiTS dataset for training process and the public 3DIRCADb dataset for validation of our method. The proposed algorithm reached DICE score 78.8% on the 3DIRCADb dataset. The described method was then applied to the 2019 Kidney Tumor Segmentation (KiTS-2019) challenge, where our single submission achieved 96.38% for kidney and 67.38% for tumor Dice scores.

Lungs are the most crucial organs in a human body. Since the cancer detection began, lung cancer has been the most common terminal disease amongst all type of cancers. The contribution of deep learning, especially the convolution neural networks has widely reduced the mortality rates resulting from lung cancer. The classification of Computed Tomography (CT) images has enhanced the early diagnosis of lung cancer that has enabled victims to undergo treatment at an early stage. The resolution of the CT images have been variedly used for the accuracy of the model. Besides, the detection of lumps or anomalies in the images has greatly supported early diagnosis. Classification plays a vital role in the deep learning models to sort out the input images as positive and negative based on the attribute of the model built. However, the generalisation of classifiers has reduced the accuracy of the corresponding models built. To increase the accuracy and efficiency of the deep learning model, an optimised classification technique is used to predict lung cancer from the CT images. The purpose of optimisation here will enable the model to adapt stipulated feature extraction process according to the input images fed into the network. The model will be trained for predicting purpose given any resolution of the images. KEYWORDS: Lung cancer, CT images, Classification techniques, Optimised Classification, Prediction

Automatic identification of consistently defined body regions in medical images is vital in many applications. In this paper, we describe a method to automatically demarcate the superior and inferior boundaries for neck, thorax, abdomen, and pelvis body regions in computed tomography (CT) images. For any three-dimensional (3D) CT image I, following precise anatomic definitions, we denote the superior and inferior axial boundary slices of the neck, thorax, abdomen, and pelvis body regions by NS(I), NI(I), TS(I), TI(I), AS(I), AI(I), PS(I), and PI(I), respectively. Of these, by definition, AI(I)=PS(I), and so the problem reduces to demarcating seven body region boundaries. Our method consists of a two-step approach. In the first step, a convolutional neural network (CNN) is trained to classify each axial slice in I into one of nine categories: the seven body region boundaries, plus legs (defined as all axial slices inferior to PI(I)), and the none-of-the-above category. This CNN uses a multichannel approach to exploit the interslice contrast, providing the neural network with additional visual context at the body region boundaries. In the second step, to improve the predictions for body region boundaries that are very subtle and that exhibit low contrast, a recurrent neural network (RNN) is trained on features extracted by CNN, limited to a flexible window about the predictions from the CNN. The method is evaluated on low-dose CT images from 442 patient scans, divided into training and testing sets with a ratio of 70:30. Using only the CNN, overall absolute localization error for NS(I), NI(I), TS(I), TI(I), AS(I), AI(I), and PI(I) expressed in terms of number of slices (nS) is (mean±SD): 0.61±0.58, 1.05±1.13, 0.31±0.46, 1.85±1.96, 0.57±2.44, 3.42±3.16, and 0.50±0.50, respectively. Using the RNN to refine the CNN's predictions for select classes improved the accuracy of TI(I) and AI(I) to: 1.35±1.71 and 2.83±2.75, respectively. This model outperforms the results achieved in our previous work by 2.4, 1.7, 3.1, 1.1, and 2 slices, respectively for TS(I), TI(I), AS(I), AI(I)=PS(I), and PI(I) classes with statistical significance. The model trained on low-dose CT images was also tested on diagnostic CT images for NS(I), NI(I), and TS(I) classes; the resulting errors were: 1.48±1.33, 2.56±2.05, and 0.58±0.71, respectively. Standardized body region definitions are a prerequisite for effective implementation of quantitative radiology, but the literature is severely lacking in the precise identification of body regions. The method presented in this paper significantly outperforms earlier works by a large margin, and the deviations of our results from ground truth are comparable to variations observed in manual labeling by experts. The solution presented in this work is critical to the adoption and employment of the idea of standardized body regions, and clears the path for development of applications requiring accurate demarcations of body regions. The work is indispensable for automatic anatomy recognition, delineation, and contouring for radiation therapy planning, as it not only automates an essential part of the process, but also removes the dependency on experts for accurately demarcating body regions in a study.