Semi-weakly Supervised Learning for Prostate Cancer Image Classification with Teacher-Student Deep Convolutional Networks

Development in Computer vision applications, most prominently in Artificial Intelligence (AI) over the past decade, has encouraged researchers to use computational algorithms. Machine Learning (ML) and Deep Learning (DL) algorithms are getting attention in the biomedical area to early diagnosis and prognosis of lethiferous diseases with higher accuracy using statistical analysis. Cervical cancer is one of the predominant gynecological infectious forms of cancer in females ecumenically, generally transpiring in less-developed nations. DL algorithms are more optimized and convenient due to the automatic feature extraction in Cervical lesion detection at an early phase. Here we analyze the convolutional neural network (CNN) models' performance using variation in channels depth comprises in the convolution layers to classify multi-class liquid-based cytology (LBC) Whole slide images (WSI). We propose three CNN models having two convolution layers with the number of channels (4,8), (8,16), and (32,64) and two pooling layers. CNN model with the highest number of channels in the convolution layers gives the leading performance with an accuracy of 96.89%, precision of 93.38%, the sensitivity of 93.75%, and F-score of 94.15%.

Terminal duct lobular unit (TDLU) involution is the regression of milk-producing structures in the breast. Women with less TDLU involution are more likely to develop breast cancer. A major bottleneck in studying TDLU involution in large cohort studies is the need for labor-intensive manual assessment of TDLUs. We developed a computational pathology solution to automatically capture TDLU involution measures. Whole slide images (WSIs) of benign breast biopsies were obtained from the Nurses’ Health Study. A set of 92 WSIs was annotated for acini, TDLUs and adipose tissue to train deep convolutional neural network (CNN) models for detection of acini, and segmentation of TDLUs and adipose tissue. These networks were integrated into a single computational method to capture TDLU involution measures including number of TDLUs per tissue area, median TDLU span and median number of acini per TDLU. We validated our method on 40 additional WSIs by comparing with manually acquired measures. Our CNN models detected acini with an F1 score of 0.73±0.07, and segmented TDLUs and adipose tissue with Dice scores of 0.84±0.13 and 0.87±0.04, respectively. The inter-observer ICC scores for manual assessments on 40 WSIs of number of TDLUs per tissue area, median TDLU span, and median acini count per TDLU were 0.71, 0.81 and 0.73, respectively. Intra-observer reliability was evaluated on 10/40 WSIs with ICC scores of >0.8. Inter-observer ICC scores between automated results and the mean of the two observers were: 0.80 for number of TDLUs per tissue area, 0.57 for median TDLU span, and 0.80 for median acini count per TDLU. TDLU involution measures evaluated by manual and automated assessment were inversely associated with age and menopausal status. We developed a computational pathology method to measure TDLU involution. This technology eliminates the labor-intensiveness and subjectivity of manual TDLU assessment, and can be applied to future breast cancer risk studies.

BackgroundIn digital pathology, image properties such as color, brightness, contrast and blurriness may vary based on the scanner and sample preparation. Convolutional Neural Networks (CNNs) are sensitive to these variations and may underperform on images from a different domain than the one used for training. Robustness to these image property variations is required to enable the use of deep learning in clinical practice and large scale clinical research.AimsCNN Stability Training (CST) is proposed and evaluated as a method to increase CNN robustness to scanner and Immunohistochemistry (IHC)-based image variability.MethodsCST was applied to segment epithelium in immunohistological cervical Whole Slide Images (WSIs). CST randomly distorts input tiles and factors the difference between the CNN prediction for the original and distorted inputs within the loss function. CNNs were trained using 114 p16-stained WSIs from the same scanner, and evaluated on 6 WSI test sets, each with 23 to 24 WSIs of the same tissue but different scanner/IHC combinations. Relative robustness (rAUC) was measured as the difference between the AUC on the training domain test set (i.e., baseline test set) and the remaining test sets.ResultsAcross all test sets, The AUC of CST models outperformed “No CST” models (AUC: 0.940–0.989 vs. 0.905–0.986, p < 1e − 8), and obtained an improved robustness (rAUC: [−0.038, −0.003] vs. [−0.081, −0.002]). At a WSI level, CST models showed an increase in performance in 124 of the 142 WSIs. CST models also outperformed models trained with random on-the-fly data augmentation (DA) in all test sets ([0.002, 0.021], p < 1e-6).ConclusionCST offers a path to improve CNN performance without the need for more data and allows customizing distortions to specific use cases. A python implementation of CST is publicly available at https://github.com/TIGACenter/CST_v1.

Non‐Small Cell Lung Cancer (NSCLC) has the highest cancer‐related mortality rate worldwide. While biopsy‐based diagnosis is critical for prognosis and treatment, the intricate anatomical features in Whole Slide Images (WSIs) make manual classification challenging for pathologists. Current deep learning models have been developed to aid in the automatic classification of NSCLC, but many rely on extensive manual annotations and lack efficient multi‐scale feature extraction, limiting their ability to capture diverse patterns in WSIs. There is a need to explore multipath, multi‐scale Convolutional Neural Networks (CNN) that can effectively capture these diverse patterns in WSIs. This study proposes a novel deep learning model, a Multi‐scale, Dual‐Path CNN (MDP‐CNN), designed to automatically classify NSCLC subtypes by capturing heterogeneous patterns and features in WSIs. The model was trained on two independent datasets, LC25000 and The Cancer Genome Atlas (TCGA), demonstrating notable improvements in performance metrics, achieving accuracy scores of 0.981 and 0.958, Area Under Curve (AUC) scores of 0.978 and 0.995, and kappa scores of 0.957 and 0.903 for the LC25000 and TCGA datasets, respectively. Extensive analyses, including ablation studies, interpretation plots, and cross‐dataset analysis, were conducted to demonstrate the efficacy of the proposed model. Multi‐scale processing improved the model's precision in classifying lung cancer subtypes by capturing variations in histopathological features across different resolutions. The proposed model outperformed state‐of‐the‐art models by approximately 8% in accuracy and 3% in AUC, demonstrating the effectiveness of MDP CNNs in improving WSI‐based diagnostics and supporting automated NSCLC classification and clinical decisions.

Recently, many fields have widely used cluster analysis: psychology, biology, statistics, pattern recognition, information retrieval, machine learning, and data mining. Diagnosis of histopathological images of prostate cancer is one of the routine tasks for pathologists and it is challenging for pathologists to analyze the formation of glands and tumors based on the Gleason grading system. In this study, unsupervised classification has been performed for differentiating malignant (cancerous) from benign (non-cancerous) tumors. Therefore, the unsupervised-based computer-aided diagnosis (CAD) technique would be of great benefit in easing the workloads of pathologists. This technique is used to find meaningful clustering objects (i.e., individuals, entities, patterns, or cases) and identify useful patterns. Radiomic-based features were extracted for cluster analysis using the gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), and gray-level size zone matrix (GLSZM) techniques. Multi-clustering techniques used for the unsupervised classification are K-means clustering, K-medoids clustering, Agglomerative Hierarchical (AH) clustering, Gaussian mixture model (GMM) clustering, and Spectral clustering. The quality of the clustering algorithms was determined using Purity, Silhouettes, Adjusted Rand, Fowlkes Mallows, and Calinski Harabasz (CH) scores. However, the best-performing algorithm (i.e., K-means) has been applied to predict and annotate the cancerous regions in the whole slide image (WSI) to compare with the pathologist annotation.

Background: Prostate cancer (PCa) frequently presents as multifocal lesions with histologic and molecular heterogeneity. A comprehensive understanding of tumor immune responses within the context of the complex PCa microenvironment remains elusive. Through the use of Digital Pathology and innovative Pathomics image analysis tools, tumor infiltrating lymphocytes (TILs) can be mapped, visualized, and quantified in whole slide images (WSIs) of H&amp;E stained PCa slides. We have developed a novel workflow to map and quantify TILs by Gleason histologic grade (Gleason-TIL maps) in whole radical prostatectomy specimens. Approach: WSIs from 211 H&amp;E tissue sections from eight whole radical prostatectomy specimens containing Gleason histologic patterns 3 and 4 were used in the development of the PCa Pathomics workflow. Pathomics tools generated PCa Tumor-TIL and Gleason-TIL maps for each corresponding H&amp;E WSI for all eight prostatectomy specimens. To support tissue sampling for correlative biomarker studies and genomic testing in PCa clinical trials, each Tumor-TIL and Gleason-TIL map was partitioned to quantify total PCa content, Gleason grade (3 vs 4/5) content, and intratumoral and peritumoral TILs per 0.5 x 0.5 cm area tile of tissue in the WSI. Results: Both Tumor-TIL and Gleason-TIL maps reveal complex and heterogeneous tumor immune interactions across multifocal lesions within each radical prostatectomy specimen. Our novel workflow added the ability to quantify PCa, Gleason grade, and TILs to specific 0.5 x 0.5 cm tissue tiles to quantify TILs relative to Gleason grade that varies within and across each case of multifocal PCa. Quantification of the TIL tile maps showed that in 6 of the 8 prostatectomies the “hot spot” tile with the most intratumoral TILs had more TILs in the Gleason 4/5 patches than in the adjacent Gleason 3 patches. Conclusionsand Future Studies: This novel workflow was developed to generate Tumor-TIL and Gleason-TIL maps for every PCa H&amp;E WSI to serve as powerful visual tools for real world laboratory tissue sampling for downstream genomics, immunohistochemistry, and other correlative studies for PCa clinical trials. Adding a 0.5 x 0.5 cm tile partitioning function increases the efficiency of identifying and quantitatively mapping TILs in the complex and heterogeneous PCa microenvironment. The preliminary association of TILs with higher grade Gleason 4/5 PCa appears provocative as potential actionable information as an immune biomarker in PCa. Citation Format: Yuwei Zhang, Rajarsi Gupta, Joel Saltz, Tahsin Kurc, Jakub Kaczmarzyk, Erich Bremer, Veronica Wallaengen, Amanda Galvez, Abigail Bartley, Leonard Salcedo, Alan Pollack, Sanoj Punnen, Oleksandr Kryvenko, Radka Stoyanova, Sandra M. Gaston. Mapping tumor infiltrating lymphocytes in whole prostatectomy specimens to visualize and quantify the immune microenvironment of prostate cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5315.

Ki-67 index is an important diagnostic factor in gastrointestinal neuroendocrine tumor (GI-NET). The current gold standard for grading GI-NETs involves the visual screening of histopathologically stained tissues, for hot-spots containing high amounts of proliferating tumor cells (stained with Ki-67 antibody). Subsequently, the Ki-67 index, i.e. the percentage of proliferating tumor cells within the hot-spot is manually obtained. To automate this subjective and time consuming process, we have developed an integrated pipeline, termed SKIE (synaptophysin-Ki-67 index estimator), combining double-immunohistochemical (IHC) staining for synaptophysin (stains tumor) and Ki-67, with whole slide image (WSI) analysis. The Ki-67 index for 50 human GI-NET WSIs were estimated by SKIE and compared with three pathologists' assessment, and the gold standard (exhaustive counting by a fourth pathologist) based on the double-stained image. All four pathologists unanimously graded 38 WSIs, among which, SKIE achieved 94.74% accuracy. One discrepant case was attributed to staining inconsistencies and the other to SKIE selecting a better hot-spot. The remaining 12 WSIs had discrepant grades among pathologists, and hence, the gold standard was chosen for comparison, wherein, 10 WSI grades matched with that of the gold standard, and SKIE assigned a lower and higher grade to two cases. Overall, SKIE agreed with the gold standard with a substantial linear weighted Cohen's kappa κ = 0.622 with CI [0.286, 0.958]. We further expanded our method to deep-SKIE, wherein, a deep convolutional neural network (DCNN) was trained and validated using 13,736 hotspot-sized tiles from 40 WSIs, each categorized into one of four classes (background, non-tumor, tumor grade 1, tumor grade 2) by SKIE and tested on 9 WSIs. Deep-SKIE achieved an accuracy of 91.63% with near-perfect agreement (κ = 0.88 with CI [0.87, 0.89]) with the gold standard.

BackgroundMicroinfarcts, characteristic lesions of vascular dementia (VaD), are heterogenous and vary in appearance, which pose a considerable challenge for VaD grading as there is great interrater variability in microinfarct assessment. We propose a novel application of machine learning (ML) in the automated screening of microinfarcts, addressing a gap in the post‐mortem analysis of VaD in whole slide images (WSIs) from human brain.MethodOur study adapts a patch‐based pipeline with convolutional neural networks (CNNs) to automate microinfarct screening in WSIs. We compare performance and computational cost of different fields‐of‐view (FOV) for the patch‐based method. Additionally, we propose a postprocessing step and leverage multiple FOVs to mitigate false positives. This study is validated against 66 annotated WSIs (N = 40 infarcted, N = 26 un‐infarcted) from Frontal, Parietal, and Occipital white matter regions across 22 cases. Annotations are from a single trained expert, who delineated microinfarct regions and graded white matter rarefaction.ResultWe report screening performance, i.e, the ability to distinguish infarct‐positive from infarct‐negative WSIs, and detection performance, i.e, the ability to localize the microinfarct regions within a WSI. Despite the inherent challenges of the inexact boundaries of microinfarcts, our models demonstrate notable efficacy in screening WSIs for infarcts, with ResNet‐18 achieving 100% accuracy at WSI level. However, the sensitivity in detecting infarct regions is below 50%, based on Intersection over Union thresholds above 20% overlap (a performance metric used to evaluate the accuracy of object detection models). Leveraging multiple FOV improves both detection and screening performances, reducing false positives.ConclusionThis work presents a proof‐of‐concept pipeline driven by machine learning to automate microinfarct screening &amp; detection in WSIs. This workflow provides new avenues in the automated analysis of neuropathological images of microinfarcts, bringing potential advancement in ML‐based dementia and neuropathology research.

Colorectal cancer (CRC) stands as one of the deadliest cancer types worldwide. Identifying specific gene mutations crucial for targeted therapy is pivotal for CRC treatment. However, the current genetic testing method involves liquid biopsy, which is often cumbersome and cost-intensive. Histopathological whole-slide images (WSIs) are routinely generated for diagnostic purposes and can offer valuable insights into cellular heterogeneity. In this study, we aim to develop an integrated deep learning pipeline for predicting TP53 mutations in CRC, one of the most frequently mutated genes in CRC. We commenced by collecting 424 diagnostic WSIs from the TCGA-COAD cohorts. Each WSI was split as small patches at a resolution of 0.5 micrometers per pixel for following analyses. Quality control was implemented using Otsu thresholding and Gaussian blur filtering, both at the slide-level masking and patch-level filtering stages. Additionally, we applied the Macenko algorithm for color normalization to eliminate systematic errors. Following image preprocessing, we constructed a tissue-type classification model based on VGG19, which was trained and tested using the NCT-CRC-HE-100K and CRC-VAL-HE-7K datasets, both containing pixels with tissue types labeled by human experts. The trained classifier was then used to identify adenocarcinoma epithelium patches from TCGA-COAD. Subsequently, we applied multiple pre-trained Convolutional Neural Network (CNN) models, including ResNet-101, DenseNet-201, and Inception-ResNet-v2, to predict TP53 mutations for the identified adenocarcinoma epithelium patches from TCGA-COAD. A novel integration approach was employed to aggregate patch-level predictions into slide-level outcomes. We evaluated the trained models using various metrics, including accuracy, precision, recall, F1 score, AUC (Area Under the Curve), and Cohen's Kappa coefficient. The tissue-type classification model exhibited robust performance, achieving an overall accuracy of 94.7%, a Cohen's Kappa coefficient of 94%, a precision of 95.3%, a recall of 99.76%, and an F1 score of 97.48% for identifying adenocarcinoma epithelium patches. Regarding the TP53 mutation prediction models, ResNet-101, DenseNet-201, and Inception-ResNet-v2 exhibit the predictive ability to both patch level (AUC=65%) and slide level (AUC=70%). In summary, the proposed deep learning pipeline offers an efficient way to acquire gene mutation information from WSIs. Moreover, it demonstrates a capability to discover potential molecular aberration from the clinical images. Citation Format: Geng-Yun Tien, Liang-Chuan Lai, Tzu-Pin Lu, Mong-Hsun Tsai, Hsiang-Han Chen, Eric Y. Chuang. Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4938.

Histopathological identification of tumor tissue is one of the routine pathological diagnoses for pathologists. Recently, computational pathology has been successfully interpreted by a variety of deep learning-based applications. Nevertheless, the high-efficient and spatial-correlated processing of individual patches have always attracted attention in whole-slide image (WSI) analysis. In this paper, we propose a high-throughput system to detect tumor regions in colorectal cancer histology slides precisely. We train a deep convolutional neural network (CNN) model and design a Monte Carlo (MC) adaptive sampling method to estimate the most representative patches in a WSI. Two conditional random field (CRF) models are designed, namely the correction CRF and the prediction CRF are integrated for spatial dependencies of patches. We use three datasets of colorectal cancer from The Cancer Genome Atlas (TCGA) to evaluate the performance of the system. The overall diagnostic time can be reduced from 56.7 percent to 71.7 percent on the slides of a varying tumor distribution, with an increase in classification accuracy.

Automated segmentation of cell membranes to evaluate HER2 status in whole slide images using a modified deep learning network

Ovarian cancer is the deadliest among gynecologic cancers. It is partly due to the mild symptoms in early stages and relatively high diagnostic age (about 63-year-old median age). These factors contribute to the short chemotherapy response time and survival time with about 18- and 36-months median progression-free survival and overall survival duration, respectively. The advancements of convolutional neural network (CNN) machine learning algorithms provide opportunities for the development of cost- and time-effective models which is urgently needed for ovarian cancer patients, especially for those with elevated risk. A CNN model was trained with hematoxylin and eosin (H&amp;E) stained tumor whole slide images (WSIs) from ovarian cancer patients with advanced stages. 773 WSIs of 335 patients from TCGA-OV dataset were separated randomly for training and testing cohort. The model was then applied on testing cohort with advanced stage and age patients for H&amp;E survival score. The c-index of the predicted risk on overall survival (OS) and progression-free survival (PFS) were 0.66 and 0.63, respectively. The log-rank p-values of patients between low and high risk were 0.00137 and 0.00138, respectively. The model was then further evaluated with an external dataset. The c-index of the predicted risk were 0.64 and 0.72 for OS and PFS, respectively. And the log-rank test between patients with low and high risk achieved p-values 0.017 and 0.0047 for OS and PFS, respectively. In conclusion, a novel prognostic CNN model was developed for ovarian cancer patients with advanced age and stage for better disease management. Citation Format: Chun Wai Ng, Kwong-kwok Wong, Berrett Lawson, Sammy Ferri-Borgogno, Samuel Mok. Convolutional neural network algorithms and hematoxylin-and-eosin-stained images predict clinical outcomes in high-grade serous ovarian cancer patients with advanced age and stage [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7662.

Pathology report generation from whole slide images with knowledge retrieval and multi-level regional feature selection.

Whole Slide Image (WSI) classification often relies on weakly supervised Multiple Instance Learning (MIL) methods to handle gigapixel-resolution images. In various MIL methods, attention-based approaches have shown great potential in modern medicine for cancer diagnosis and treatment. These approaches can model the interrelationships among instances to achieve enhanced bag representation using instance scores and thus promote bag-level classification performance. However, the existing attention-based MIL methods face two challenges: (1) The attention-based instance scores cannot accurately represent the contribution of instances to bag-level classification, making it difficult to identify the discriminative regions in WSIs. (2) Whole-slide pathological image analysis frequently suffers from model overfitting and insufficient representation of positive samples for training. To address the problem of poor discriminative regions in WSIs, we design a module to acquire the accurate contribution weights of instances by introducing the Class Activation Map suitable for WSI (WSICAM). For the second challenge, we adopt a Cross-Slide Augmentation (CSA) module to construct new samples with mixed labels on the basis of discriminative instances for model training. Our framework is composed of two WSICAM modules and one CSA module. The experimental results and visualizations demonstrate that our method achieves state-of-the-art in WSI classification on widely used datasets and exhibits robust capabilities in tumor lesion localization.

Reliability of whole slide images as a diagnostic modality for renal allograft biopsies