Enhancing Weakly Supervised Semantic Segmentation With Multi-Label Contrastive Learning and LLM Features Guidance.

Eliminating tissue-fold artifacts in histopathological whole-slide images for improved image-based prediction of cancer grade

Whole slide image (WSI) segmentation is a crucial task aiding tumour and cancerous cell diagnosis. Generalized Normal Distribution Optimization (GNDO) algorithm is adopted for whole slide image segmentation based on thresholding in this paper. GNDO algorithm utilizes the generalized normal distribution's properties to determine the ideal thresholds for image segmentation. Through various metrics, the efficacy of GNDO in comparison to traditional Otsu thresholding methods is demonstrated. As demonstrated by the results, it can offer reliable and flexible solutions for different histopathology images.

e21044 Background: Almost 70% of cases of lung cancer are diagnosed at advanced stage, albeit with sophisticated imaging modalities available. Of these, 35% are tiny nodules that are often missed at initial radiological screening, owing to limitations of resolution of human vision.Dramatic shift in therapeutic paradigm in the management of lung cancer in the last decade has ushered an era of stratified medicine. In India, one-third population inhabits rural lands;lack of awareness for molecular medicine poses challenges to treating oncologists. AI-based cancer detection and prediction of potential oncogenic driver at diagnosis, will help alleviate patient anxiety caused due to diagnostic delays, aiding in prompt initiation of optimal therapy.Here, a comprehensive deep-CNN based diagnosis of lung adenocarcinoma (LUAD) was implemented using both PET/CT and histopathology images of 100 patients. Methods: Data collection: 143 whole-slide images were downloaded from Dartmouth Lung Cancer Histology Dataset from Biomedical Informatics Research and Data Science website and PET/CT dataset-A Large-Scale CT and PET/CT Dataset for Lung Cancer Diagnosis (Lung-PET-CT-Dx) from The Cancer Imaging Archive. This comprises 251,135 images from 355 participants across 436 studies. Creation of Deep-CNN model: Residual Neural Network - 18 was used for histopathlogic classification followed by Detectron2’s region-based deep-CNN for detection of presence of tumours in lung PET/CT images. Properties such as size, morphology, attenuation, malignancy, and spiculation were also measured. The 18-layer deep network, ResNet-18, detected the histologic patterns using whole-slide histopathology images. Results: Results for 100 patients whose PET/CT images, histopathology images, and mutational status of three genes (EGFR, ALK, ROS) were readily available. The histologic patterns of tumour cells appear to be related to the mutational status of genes. The ResNet-18 CNN could predict various histologic patterns (Acinar, Lepidic, Papillary, Micropapillary, and Solid patterns). Conclusions: CNN models can help in early diagnosis and prompt treatment of lung cancer, if fairly accurate. Larger datasets are needed to improve the same. The final validation of the model is underway and will be reported in near future.[Table: see text]

Anomaly-guided weakly supervised lesion segmentation on retinal OCT images

Replication timing is a costly but powerful tool for characterizing cellular mechanisms that underlie chromatin organization, cancer epigenetics, and genomic instability. Genome-wide replication timing profiles reflect the temporal order of DNA synthesis during S phase and are closely linked to chromatin accessibility, transcriptional activity, and proliferative state. Prior work has demonstrated a robust inverse relationship between replication timing and DNA methylation at the domain scale, enabling methylation-based proxies to approximate replication timing in large cohorts where direct experimental measurement is impractical. Given the tight coupling between replication timing, chromatin structure, and cellular phenotype, we hypothesized that histologic morphology encodes information consistent with replication timing states. To test this hypothesis, we implemented Vision Transformer (ViT) architectures to predict replication timing proxies from whole-slide histopathology images. Patch-level embeddings were extracted using a pretrained ViT and aggregated through attention-based multiple instance learning to predict sample-level replication timing. In parallel, CellViT models were employed to perform cell-level prediction, enabling direct comparison between patch-based and cellular representations. Across both modeling strategies, statistically significant correlations were observed between image-derived features and methylation-based replication timing proxies. Patch-level models achieved correlations of approximately 46%, while cell-level models consistently reached correlations exceeding 50% on the validation cohort. Prediction error remained stable across folds, with mean absolute error and mean squared error values ranging between 0.4 and 0.6. These results demonstrate that replication timing–associated epigenomic states are reflected in tissue morphology and can be inferred using deep learning models applied to routine histopathology. This work establishes a feasible, noninvasive framework for replicosomic inference from whole-slide images and supports future efforts toward spatially resolved replication timing analysis and integrative modeling of replicative stress in cancer.

Colorectal cancer (CRC) stands as one of the deadliest cancer types worldwide. Identifying specific gene mutations crucial for targeted therapy is pivotal for CRC treatment. However, the current genetic testing method involves liquid biopsy, which is often cumbersome and cost-intensive. Histopathological whole-slide images (WSIs) are routinely generated for diagnostic purposes and can offer valuable insights into cellular heterogeneity. In this study, we aim to develop an integrated deep learning pipeline for predicting TP53 mutations in CRC, one of the most frequently mutated genes in CRC. We commenced by collecting 424 diagnostic WSIs from the TCGA-COAD cohorts. Each WSI was split as small patches at a resolution of 0.5 micrometers per pixel for following analyses. Quality control was implemented using Otsu thresholding and Gaussian blur filtering, both at the slide-level masking and patch-level filtering stages. Additionally, we applied the Macenko algorithm for color normalization to eliminate systematic errors. Following image preprocessing, we constructed a tissue-type classification model based on VGG19, which was trained and tested using the NCT-CRC-HE-100K and CRC-VAL-HE-7K datasets, both containing pixels with tissue types labeled by human experts. The trained classifier was then used to identify adenocarcinoma epithelium patches from TCGA-COAD. Subsequently, we applied multiple pre-trained Convolutional Neural Network (CNN) models, including ResNet-101, DenseNet-201, and Inception-ResNet-v2, to predict TP53 mutations for the identified adenocarcinoma epithelium patches from TCGA-COAD. A novel integration approach was employed to aggregate patch-level predictions into slide-level outcomes. We evaluated the trained models using various metrics, including accuracy, precision, recall, F1 score, AUC (Area Under the Curve), and Cohen's Kappa coefficient. The tissue-type classification model exhibited robust performance, achieving an overall accuracy of 94.7%, a Cohen's Kappa coefficient of 94%, a precision of 95.3%, a recall of 99.76%, and an F1 score of 97.48% for identifying adenocarcinoma epithelium patches. Regarding the TP53 mutation prediction models, ResNet-101, DenseNet-201, and Inception-ResNet-v2 exhibit the predictive ability to both patch level (AUC=65%) and slide level (AUC=70%). In summary, the proposed deep learning pipeline offers an efficient way to acquire gene mutation information from WSIs. Moreover, it demonstrates a capability to discover potential molecular aberration from the clinical images. Citation Format: Geng-Yun Tien, Liang-Chuan Lai, Tzu-Pin Lu, Mong-Hsun Tsai, Hsiang-Han Chen, Eric Y. Chuang. Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4938.

<div>AbstractPurpose:<p>Histopathology evaluation is the gold standard for diagnosing clear cell (ccRCC), papillary, and chromophobe renal cell carcinoma (RCC). However, interrater variability has been reported, and the whole-slide histopathology images likely contain underutilized biological signals predictive of genomic profiles.</p>Experimental Design:<p>To address this knowledge gap, we obtained whole-slide histopathology images and demographic, genomic, and clinical data from The Cancer Genome Atlas, the Clinical Proteomic Tumor Analysis Consortium, and Brigham and Women's Hospital (Boston, MA) to develop computational methods for integrating data analyses. Leveraging these large and diverse datasets, we developed fully automated convolutional neural networks to diagnose renal cancers and connect quantitative pathology patterns with patients' genomic profiles and prognoses.</p>Results:<p>Our deep convolutional neural networks successfully detected malignancy (AUC in the independent validation cohort: 0.964–0.985), diagnosed RCC histologic subtypes (independent validation AUCs of the best models: 0.953–0.993), and predicted stage I ccRCC patients' survival outcomes (log-rank test <i>P</i> = 0.02). Our machine learning approaches further identified histopathology image features indicative of copy-number alterations (AUC > 0.7 in multiple genes in patients with ccRCC) and tumor mutation burden.</p>Conclusions:<p>Our results suggest that convolutional neural networks can extract histologic signals predictive of patients' diagnoses, prognoses, and genomic variations of clinical importance. Our approaches can systematically identify previously unknown relations among diverse data modalities.</p></div>

Recently proposed methods for weakly-supervised semantic segmentation have achieved impressive performance in predicting pixel classes despite being trained with only image labels which lack positional information. Because image annotations are cheaper and quicker to generate, weak supervision is more practical than full supervision for training segmentation algorithms. These methods have been predominantly developed to solve the background separation and partial segmentation problems presented by natural scene images and it is unclear whether they can be simply transferred to other domains with different characteristics, such as histopathology and satellite images, and still perform well. This paper evaluates state-of-the-art weakly-supervised semantic segmentation methods on natural scene, histopathology, and satellite image datasets and analyzes how to determine which method is most suitable for a given dataset. Our experiments indicate that histopathology and satellite images present a different set of problems for weakly-supervised semantic segmentation than natural scene images, such as ambiguous boundaries and class co-occurrence. Methods perform well for datasets they were developed on, but tend to perform poorly on other datasets. We present some practical techniques for these methods on unseen datasets and argue that more work is needed for a generalizable approach to weakly-supervised semantic segmentation. Our full code implementation is available on GitHub: https://github.com/lyndonchan/wsss-analysis.

Characterization of collagen deposition in immunostained images is relevant to various pathological conditions, particularly in human immunodeficiency virus (HIV) infection. Accurate segmentation of these collagens and extracting representative features of underlying diseases are important steps to achieve quantitative diagnosis. While a first order statistic derived from the segmented collagens can be useful in representing pathological evolutions at different timepoints, it fails to capture morphological changes and spatial arrangements. In this work, we demonstrate a complete pipeline for extracting key histopathology features representing underlying disease progression from histopathology whole-slide images (WSIs) via integration of deep learning and graph theory. A convolutional neural network is trained and utilized for histopathological WSI segmentation. Parallel processing is applied to convert 100K ~ 150K segmented collagen fibrils into a single collective attributed relational graph, and graph theory is applied to extract topological and relational information from the collagenous framework. Results are in good agreement with the expected pathogenicity induced by collagen deposition, highlighting potentials in clinical applications for analyzing various meshwork-structures in whole-slide histology images.

Artificial Intelligence-Powered Digital Pathology to Improve Diagnosis and Personalized Prognostic Assessment in Patient with Myeloid Neoplasms

Uterine cervical and endometrial cancers have different subtypes with different clinical outcomes. Therefore, cancer subtyping is essential for proper treatment decisions. Furthermore, an endometrial and endocervical origin for an adenocarcinoma should also be distinguished. Although the discrimination can be helped with various immunohistochemical markers, there is no definitive marker. Therefore, we tested the feasibility of deep learning (DL)-based classification for the subtypes of cervical and endometrial cancers and the site of origin of adenocarcinomas from whole slide images (WSIs) of tissue slides. WSIs were split into 360 × 360-pixel image patches at 20× magnification for classification. Then, the average of patch classification results was used for the final classification. The area under the receiver operating characteristic curves (AUROCs) for the cervical and endometrial cancer classifiers were 0.977 and 0.944, respectively. The classifier for the origin of an adenocarcinoma yielded an AUROC of 0.939. These results clearly demonstrated the feasibility of DL-based classifiers for the discrimination of cancers from the cervix and uterus. We expect that the performance of the classifiers will be much enhanced with an accumulation of WSI data. Then, the information from the classifiers can be integrated with other data for more precise discrimination of cervical and endometrial cancers.

Existing multi-label learning algorithms mostly assume that all class labels in an obtained data set are completely accurate. This assumption is rarely valid. As the scale of the label space increases, owing to limited domain knowledge and differences among methods of data acquisition, some examples will be unlabeled or mislabeled. The introduction of such noise results in the poor performance of trained multi-label classifiers. This paper proposes a new method for multi-label learning with label noise, called low-rank-constraint-based multi-label learning with label noise (ML-LRC), by incorporating low-rank matrix recovery and features selection into the learning process. First, we assume that two strongly correlated labels share more samples than two weakly correlated labels. Based on this assumption, the ML-LRC applies low-rank constraints to the label matrix to mine the local correlations of class labels and remove noise from the observed label matrix. Second, we believe that each class label only correlates to some features. So we control the sparsity of the coefficient matrix to filter out label-specific features, which can reduce the complexity of the model and greatly improve the performance of the classifier. The accelerated proximal gradient method and alternating direction method of multipliers are adopted to solve the problem. Experiments conducted on four benchmark multi-label datasets demonstrate the competitive performance of our proposed method compared with several state-of-the-art methods.

BackgroundHistopathology image analysis is a gold standard for cancer recognition and diagnosis. Automatic analysis of histopathology images can help pathologists diagnose tumor and cancer subtypes, alleviating the workload of pathologists. There are two basic types of tasks in digital histopathology image analysis: image classification and image segmentation. Typical problems with histopathology images that hamper automatic analysis include complex clinical representations, limited quantities of training images in a dataset, and the extremely large size of singular images (usually up to gigapixels). The property of extremely large size for a single image also makes a histopathology image dataset be considered large-scale, even if the number of images in the dataset is limited.ResultsIn this paper, we propose leveraging deep convolutional neural network (CNN) activation features to perform classification, segmentation and visualization in large-scale tissue histopathology images. Our framework transfers features extracted from CNNs trained by a large natural image database, ImageNet, to histopathology images. We also explore the characteristics of CNN features by visualizing the response of individual neuron components in the last hidden layer. Some of these characteristics reveal biological insights that have been verified by pathologists. According to our experiments, the framework proposed has shown state-of-the-art performance on a brain tumor dataset from the MICCAI 2014 Brain Tumor Digital Pathology Challenge and a colon cancer histopathology image dataset.ConclusionsThe framework proposed is a simple, efficient and effective system for histopathology image automatic analysis. We successfully transfer ImageNet knowledge as deep convolutional activation features to the classification and segmentation of histopathology images with little training data. CNN features are significantly more powerful than expert-designed features.

A correlation graph attention network for classifying chromosomal instabilities from histopathology whole-slide images

BackgroundTargeted therapy and immunotherapy put forward higher demands for accurate lung cancer classification, as well as benign versus malignant disease discrimination. Digital whole slide images (WSIs) witnessed the transition from traditional histopathology to computational approaches, arousing a hype of deep learning methods for histopathological analysis. We aimed at exploring the potential of deep learning models in the identification of lung cancer subtypes and cancer mimics from WSIs.MethodsWe initially obtained 741 WSIs from the First Affiliated Hospital of Sun Yat-sen University (SYSUFH) for the deep learning model development, optimization, and verification. Additional 318 WSIs from SYSUFH, 212 from Shenzhen People’s Hospital, and 422 from The Cancer Genome Atlas were further collected for multi-centre verification. EfficientNet-B5- and ResNet-50-based deep learning methods were developed and compared using the metrics of recall, precision, F1-score, and areas under the curve (AUCs). A threshold-based tumour-first aggregation approach was proposed and implemented for the label inferencing of WSIs with complex tissue components. Four pathologists of different levels from SYSUFH reviewed all the testing slides blindly, and the diagnosing results were used for quantitative comparisons with the best performing deep learning model.ResultsWe developed the first deep learning-based six-type classifier for histopathological WSI classification of lung adenocarcinoma, lung squamous cell carcinoma, small cell lung carcinoma, pulmonary tuberculosis, organizing pneumonia, and normal lung. The EfficientNet-B5-based model outperformed ResNet-50 and was selected as the backbone in the classifier. Tested on 1067 slides from four cohorts of different medical centres, AUCs of 0.970, 0.918, 0.963, and 0.978 were achieved, respectively. The classifier achieved high consistence to the ground truth and attending pathologists with high intraclass correlation coefficients over 0.873.ConclusionsMulti-cohort testing demonstrated our six-type classifier achieved consistent and comparable performance to experienced pathologists and gained advantages over other existing computational methods. The visualization of prediction heatmap improved the model interpretability intuitively. The classifier with the threshold-based tumour-first label inferencing method exhibited excellent accuracy and feasibility in classifying lung cancers and confused nonneoplastic tissues, indicating that deep learning can resolve complex multi-class tissue classification that conforms to real-world histopathological scenarios.