Point annotations for nucleus segmentation in histopathology images via envelope enhancement network

A nuclei segmentation in the histopathological images is a challenging problem, specifically in the images stained with the hematoxylin and eosin dye. To alleviate this, a new whale optimization algorithm based fuzzy clustering method has been presented in this work for nuclei segmentation in the histopathological images. In the proposed method, the whale optimization algorithm is applied to find the best fuzzy clusters based on the objective function defined as compactness. The evaluation of the introduced method is conducted on the publicly available histopathological image dataset, namely H&E stained ER+ breast cancer images. The experimental results evince that the proposed approach acquires comparatively better accuracy of segmentation than the standard clustering based image segmentation methods in terms of both the qualitative and quantitative measurements. Therefore, it can be stated that the introduced method is an efficacious nuclei segmentation method for histopathological images.

Nuclei Segmentation is a very essential and intermediate step for automatic cancer detection from H and E stained histopathology images. In the recent advent, the rise of Convolutional Neural Network (CNN), has enabled researchers to detect nuclei automatically from histopathology images with higher accuracy. However, the performance of automatic nuclei segmentation by CNN is fraught with overfitting, due to very less number of annotated segmented images available. Indeed, we find that the problem of nuclei segmentation is an unsupervised problem, because still now there is no automatic tool available which can make annotated images (nuclei segmented images) accurately, to the best of our knowledge. In this research article, we present a Logarithmic-Base2 of Gaussian (Log-Base2-G) Kernel which has the ability to track only the nuclei portions automatically from Colorectal Cancer H and E stained histopathology images. First, Log-Base2-G Kernel is applied to the input images. Thereafter, we apply an adaptive Canny Edge detector, in order to segment only the nuclei edges from H and E stained histopathology images. Experimental results revealed that our proposed method achieved higher accuracy and F1 score, without the help of any annotated data which is a significant improvement. We have used two different datasets (Con-SeP dataset, and Glass-contest dataset, both contains Colorectal Cancer histopathology images) to check the effectiveness and validity of our proposed method. These results have shown that our proposed method outperformed other image processing or unsupervised methods both qualitatively and quantitatively.

Nuclei segmentation is a fundamental upstream task of digital pathology image analysis. Existing nuclei segmentation methods usually require pixel-level labeled images from experienced pathologists. In this paper, we proposed an innovative data augmentation workflow for histopathology images: a) generates a set of initial central points randomly with existing human-annotated histopathology image datasets; b) generates nuclei segmentation masks based on the generated centroid points of step a); c) generates Haematoxylin and Eosin (H&E)-stained histopathology images corresponding to the generated nuclei masks. In addition, we proposed a deep attention feature fusion generative adversarial network (DAFF -GAN) to improve the image quality and the photorealism of the generated image. We conducted extensive experiments on several existing nuclei segmentation methods, comparing using raw data with the augmented data by our strategy. Extensive experiments proved the effectiveness of our proposed strategy.

Due to the complexity of histopathology tissues, an accurate classification and segmentation of cancer diagnosis is a challenging task in computer vision. The nuclei segmentation of microscopic images is a key prerequisite for cancerous pathological image analysis. However, an accurate nuclei segmentation is a long running major challenge due to the enormous color variability of staining, nuclei shapes, sizes, and clustering of overlapping cells. To address these challenges and early diagnosis as well as reduce the bias decisions of expert lab technician of cancer in clinical practice, the authors study the classification of computer-aided frameworks and automatic nuclei segmentation frameworks based on histopathology images by convolutional deep learning. The authors have used a publicly available PatchCamelyon and 2018 Data Science Bowl histology image dataset for this study. The results are compared and expected to be useful clinically for technician experts in the analysis of cancer diagnosis and the survival chances of patients.

Segmenting cell nuclei in histopathological images is an extremely important process for computational pathology, affecting not only the accuracy of a disease diagnosis but also the analysis of biomarkers and the assessment of cells performed on a large scale. Although many deep learning models can take out global and local features, it is still difficult to find a good balance between semantic context and fine boundary precision, especially when nuclei are overlapping or have changed shapes. In this paper, we put forward a novel deep learning model named Dual-Stream HyperFusionNet (DS-HFN), which is capable of explicitly representing the global contextual and boundary-sensitive features for the robust nuclei segmentation task by first decoupling and then fusing them. The dual-stream encoder in DS-HFN can simultaneously acquire the semantic and edge-focused features, which can be later combined with the help of the attention-driven HyperFeature Embedding Module (HFEM). Additionally, the dual-decoder concept, together with the Gradient-Aligned Loss Function, facilitates structural precision by making the segmentation gradients that are predicted consistent with the ground-truth contours. On various benchmark datasets like TNBC and MoNuSeg, DS-HFN not only achieves better results than other 30 state-of-the-art models in all evaluation metrics but also is less computationally expensive. These findings indicate that DS-HFN provides a capability for accurate nuclei segmentation, which is essential for clinical diagnosis and biomarker analysis, across a wide range of tissues in digital pathology.

Nucleus segmentation is an imperative step in the qualitative study of imaging datasets, considered as an intricate task in histopathology image analysis. Segmenting a nucleus is an important part of diagnosing, staging, and grading cancer, but overlapping regions make it hard to separate and tell apart independent nuclei. Deep Learning is swiftly paving its way in the arena of nucleus segmentation, attracting quite a few researchers with its numerous published research articles indicating its efficacy in the field. This paper presents a systematic survey on nucleus segmentation using deep learning in the last five years (2017-2021), highlighting various segmentation models (U-Net, SCPP-Net, Sharp U-Net, and LiverNet) and exploring their similarities, strengths, datasets utilized, and unfolding research areas.

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

An efficient deep equilibrium model for medical image segmentation

In automatic cancer detection, nuclei segmentation is a very essential step which enables the classification task simpler and computationally more efficient. However, automatic nuclei detection is fraught with the problems of inter-class variability of nuclei size and shapes. In this research article, a novel unsupervised edge detection technique, is proposed for segmenting the nuclei regions in liver cancer Hematoxylin and Eosin (H&E) stained histopathology images. In this novel edge detection technique, the notion of computing local standard deviation is incorporated, instead of computing gradients. Since, local standard deviation value is correlated with the edge information of image, this novel method can extract the nuclei edges efficiently, even at multiscale. The edge-detected image is further converted into a binary image by employing Ostu (IEEE Trans Syst Man Cybern 9(1):62–66, 1979)’s thresholding operation. Subsequently, an adaptive morphological filter is also employed in order to refine the final segmented image. The proposed nuclei segmentation method is also tested on a well-recognized multi-organ dataset, in order to check its effectiveness over wide variety of dataset. The visual results of both datasets indicate that the proposed segmentation method overcomes the limitations of existing unsupervised methods, moreover, its performance is comparable with the same of recent deep neural models like DIST, HoverNet, etc. Furthermore, three quality metrics are computed in order to measure the performance of several nuclei segmentation methods quantitatively. The mean value of quality metrics reveals that proposed segmentation method indeed outperformed other existing nuclei segmentation methods.

Nuclei segmentation is a fundamental step to measure cell motility and morphology in High-throughput microscopy images. One prerequisite of automatic nuclei segmentation is to manually label a huge amount of cell images, which is a labor-intensive and time-consuming process. This paper develops a semi-supervised learning approach to reduce the dependence on the amount of labeled images, and it consists of three main steps. First, cell regions are automatically extracted from the background through a novel convolutional neural network (CNN). Comparing with the state-of-art segmentation network structure, our designed network structure has fewer parameters and combines layers of the feature hierarchy that refines the spatial precision of the output probability map. Second, discriminator networks are explored to produce confidence maps for unlabeled microscopy images by applying a fully convolution network (FCN) as the discriminator network. It also can be used to improve the performance of the segmentation network by coupling the adversarial loss with standard cross-entropy loss. Finally, combining both labeled images and unlabeled images with confidence maps to train the segmentation network can identify individual nuclei on the extracted cell regions. Experimental results on only 84 images showed that SEG score could achieve 77.9% and F1 score could achieve 76.0%, which outperformed existing using 670 images segmentation methods. Such promising result suggested that the proposed semi-supervised nuclei segmentation approach could achieve high segmentation accuracy with a small set of labeled images, which is a desirable property to analyze microscopy images in the real clinical practice.

Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987–2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient’s medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.

Nuclei segmentation and classification provide an essential basis for tumor immune microenvironment analysis. The previous nuclei segmentation and classification models require splitting large images into smaller patches for training, leading to two significant issues. First, nuclei at the borders of adjacent patches often misalign during inference. Second, this patch-based approach significantly increases the model's training and inference time. Recently, Mamba has garnered attention for its ability to model large-scale images with linear time complexity and low memory consumption. It offers a promising solution for training nuclei segmentation and classification models on full-sized images. However, the Mamba orientation-based scanning method lacks account for category-specific features, resulting in suboptimal performance in scenarios with imbalanced class distributions. To address these challenges, this paper introduces a novel scanning strategy based on category probability sorting, which independently ranks and scans features for each category according to confidence from high to low. This approach enhances the feature representation of uncertain samples and mitigates the issues caused by imbalanced distributions. Extensive experiments conducted on four public datasets demonstrate that our method outperforms state-of-the-art approaches, delivering superior performance in nuclei segmentation and classification tasks.

Recent advances in deep learning, like 3D fully convolutional networks (FCNs), have improved the state-of-the-art in dense semantic segmentation of medical images. However, most network architectures require severely downsampling or cropping the images to meet the memory limitations of today’s GPU cards while still considering enough context in the images for accurate segmentation. In this work, we propose a novel approach that utilizes auto-context to perform semantic segmentation at higher resolutions in a multi-scale pyramid of stacked 3D FCNs. We train and validate our models on a dataset of manually annotated abdominal organs and vessels from 377 clinical CT images used in gastric surgery, and achieve promising results with close to 90% Dice score on average. For additional evaluation, we perform separate testing on datasets from different sources and achieve competitive results, illustrating the robustness of the model and approach.

A Deep Learning Approach for Histology-Based Nucleus Segmentation and Tumor Microenvironment Characterization

Computer aided diagnosis based on computational pathology combines the concepts of pathology with computer science to develop automated mechanisms for interpretation of histological images. Nuclei segmentation is a type of computation pathology that enables the identification and separation of Nuclei cells in a histopathology image. These segmentation routines play a critical role in quantification of cellular structures and provide insights to identify abnormalities related to pathological conditions such as cancer. In order to determine the progress and severity of a pathological condition it is necessary to use tools that measure the properties of tissue structure in different stages of a disease. Biomedical image analysis for Nuclei segmentation using deep learning methodologies is an ongoing area of research wherein to improve histopathology examination it is desired to develop a model that can generalize and accurately segment different types of nuclei. However, creating a suitable model for this process is difficult due to variability in patient data, overlapping nuclei cell boundaries, image quality, background noise, and provision of well-annotated data. In this study, we describe our findings on the performance of a model developed using the standard U-Net architecture and compare it against a model that is developed based on pre-trained weights and models using ImageNet as the backbone. The aim is to develop a semantic segmentation model that can capture high-level information in terms of features from a data set that consists of Whole-slide images (WSI). The outcome of the research is to be able to select a semantic segmentation model that is efficient and fast to classify Nuclei cells for the given biomedical tissue images. The model uses a stochastic gradient descent optimizer and the evaluation criteria for this network i.e., the ground truth against prediction is based on Mean Intersection over Union.