Nuclei segmentation in histopathology images using deep neural networks

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

BackgroundHistopathology image analysis is a gold standard for cancer recognition and diagnosis. Automatic analysis of histopathology images can help pathologists diagnose tumor and cancer subtypes, alleviating the workload of pathologists. There are two basic types of tasks in digital histopathology image analysis: image classification and image segmentation. Typical problems with histopathology images that hamper automatic analysis include complex clinical representations, limited quantities of training images in a dataset, and the extremely large size of singular images (usually up to gigapixels). The property of extremely large size for a single image also makes a histopathology image dataset be considered large-scale, even if the number of images in the dataset is limited.ResultsIn this paper, we propose leveraging deep convolutional neural network (CNN) activation features to perform classification, segmentation and visualization in large-scale tissue histopathology images. Our framework transfers features extracted from CNNs trained by a large natural image database, ImageNet, to histopathology images. We also explore the characteristics of CNN features by visualizing the response of individual neuron components in the last hidden layer. Some of these characteristics reveal biological insights that have been verified by pathologists. According to our experiments, the framework proposed has shown state-of-the-art performance on a brain tumor dataset from the MICCAI 2014 Brain Tumor Digital Pathology Challenge and a colon cancer histopathology image dataset.ConclusionsThe framework proposed is a simple, efficient and effective system for histopathology image automatic analysis. We successfully transfer ImageNet knowledge as deep convolutional activation features to the classification and segmentation of histopathology images with little training data. CNN features are significantly more powerful than expert-designed features.

Objective. Automated cell nuclei segmentation is vital for the histopathological diagnosis of cancer. However, nuclei segmentation from ‘hematoxylin and eosin’ (HE) stained ‘whole slide images’ (WSIs) remains a challenge due to noise-induced intensity variations and uneven staining. The goal of this paper is to propose a novel deep learning model for accurately segmenting the nuclei in HE-stained WSIs. Approach. We introduce FEEDNet, a novel encoder-decoder network that uses LSTM units and ‘feature enhancement blocks’ (FE-blocks). Our proposed FE-block avoids the loss of location information incurred by pooling layers by concatenating the downsampled version of the original image to preserve pixel intensities. FEEDNet uses an LSTM unit to capture multi-channel representations compactly. Secondly, for datasets that provide class information, we train a multiclass segmentation model, which generates masks corresponding to each class at the output. Using this information, we generate more accurate binary masks than that generated by conventional binary segmentation models. Main results. We have thoroughly evaluated FEEDNet on CoNSeP, Kumar, and CPM-17 datasets. FEEDNet achieves the best value of PQ (panoptic quality) on CoNSeP and CPM-17 datasets and the second best value of PQ on the Kumar dataset. The 32-bit floating-point version of FEEDNet has a model size of 64.90 MB. With INT8 quantization, the model size reduces to only 16.51 MB, with a negligible loss in predictive performance on Kumar and CPM-17 datasets and a minor loss on the CoNSeP dataset. Significance. Our proposed idea of generalized class-aware binary segmentation is shown to be accurate on a variety of datasets. FEEDNet has a smaller model size than the previous nuclei segmentation networks, which makes it suitable for execution on memory-constrained edge devices. The state-of-the-art predictive performance of FEEDNet makes it the most preferred network. The source code can be obtained from https://github.com/CandleLabAI/FEEDNet.

A nuclei segmentation in the histopathological images is a challenging problem, specifically in the images stained with the hematoxylin and eosin dye. To alleviate this, a new whale optimization algorithm based fuzzy clustering method has been presented in this work for nuclei segmentation in the histopathological images. In the proposed method, the whale optimization algorithm is applied to find the best fuzzy clusters based on the objective function defined as compactness. The evaluation of the introduced method is conducted on the publicly available histopathological image dataset, namely H&E stained ER+ breast cancer images. The experimental results evince that the proposed approach acquires comparatively better accuracy of segmentation than the standard clustering based image segmentation methods in terms of both the qualitative and quantitative measurements. Therefore, it can be stated that the introduced method is an efficacious nuclei segmentation method for histopathological images.

Diagnosis and monitoring of kidney diseases and transplants is supported by microscopic analysis of needle-core biopsy samples. The current methods of analysis are affected by inconsistencies, bias, and inaccuracies. We propose and evaluate image processing methods for automatic segmentation of cell nuclei in digital images of renal biopsy samples. The methods evaluated include automatic thresholding, adaptive thresholding, and morphological granulometry. The results are compared to annotations made by an expert pathologist of more than 1500 cells in 18 images from different patients. The three methods provided true-positive ratios in the range 0.80 to 0.93.

Nuclei segmentation is a fundamental upstream task of digital pathology image analysis. Existing nuclei segmentation methods usually require pixel-level labeled images from experienced pathologists. In this paper, we proposed an innovative data augmentation workflow for histopathology images: a) generates a set of initial central points randomly with existing human-annotated histopathology image datasets; b) generates nuclei segmentation masks based on the generated centroid points of step a); c) generates Haematoxylin and Eosin (H&E)-stained histopathology images corresponding to the generated nuclei masks. In addition, we proposed a deep attention feature fusion generative adversarial network (DAFF -GAN) to improve the image quality and the photorealism of the generated image. We conducted extensive experiments on several existing nuclei segmentation methods, comparing using raw data with the augmented data by our strategy. Extensive experiments proved the effectiveness of our proposed strategy.

Nucleus segmentation in histopathology images holds great clinical value for disease analysis. With the advancements in deep learning and continuous iteration of devices, existing nucleus segmentation algorithms can achieve satisfactory performance. However, these fully supervised nucleus segmentation methods require pixel-level annotation information, which requires a significant amount of time and effort. To ease this burden of annotations, we propose an envelope enhancement segmentation Network (EESNet) for weak supervison nucleus segmentation. Specifically, we first employ a Voronoi diagram and adaptive k-means clustering algorithm to derive two kinds of informative pixel-level pseudo labels,which are used for pixel-wise segmentation. Furthermore, we introduce an envelope enhancement network to provide rich envelope structure information to the segmentation network, which helps mitigate the problem of inaccurate boundary segmentation caused by point annotations. Finally, our network is verified on a mainstream nucleus segmentation datasets (consep) and achieves satisfactory performance.

With the continuous development of science, more and more research results have proved that machine learning is capable of diagnosing and studying the major depressive disorder (MDD) in the brain. We propose a deep learning network with multibranch and local residual feedback, for four different types of functional magnetic resonance imaging (fMRI) data produced by depressed patients and control people under the condition of listening to positive- and negative-emotions music. We use the large convolution kernel of the same size as the correlation matrix to match the features and obtain the results of feature matching of 264 regions of interest (ROIs). Firstly, four-dimensional fMRI data are used to generate the two-dimensional correlation matrix of one person’s brain based on ROIs and then processed by the threshold value which is selected according to the characteristics of complex network and small-world network. After that, the deep learning model in this paper is compared with support vector machine (SVM), logistic regression (LR), k-nearest neighbor (kNN), a common deep neural network (DNN), and a deep convolutional neural network (CNN) for classification. Finally, we further calculate the matched ROIs from the intermediate results of our deep learning model which can help related fields further explore the pathogeny of depression patients.

Automated cell nuclei segmentation is the most crucial step toward the implementation of a computer-aided diagnosis system for cancer cells. Studies on the automated analysis of cytology pleural effusion images are few because of the lack of reliable cell nuclei segmentation methods. Therefore, this paper presents a comparative study of twelve nuclei segmentation methods for cytology pleural effusion images. Each method involves three main steps: preprocessing, segmentation, and postprocessing. The preprocessing and segmentation stages help enhancing the image quality and extracting the nuclei regions from the rest of the image, respectively. The postprocessing stage helps in refining the segmented nuclei and removing false findings. The segmentation methods are quantitatively evaluated for 35 cytology images of pleural effusion by computing five performance metrics. The evaluation results show that the segmentation performances of the Otsu, k-means, mean shift, Chan–Vese, and graph cut methods are 94, 94, 95, 94, and 93%, respectively, with high abnormal nuclei detection rates. The average computational times per image are 1.08, 36.62, 50.18, 330, and 44.03 seconds, respectively. The findings of this study will be useful for current and potential future studies on cytology images of pleural effusion.

Chapter Thirteen - Toward large-scale histopathological image analysis via deep learning

In today’s technology world, data security has a great importance. Because each data type has its own characteristics, there are various methods of providing this security. The main subject of this study is the security of image data which are more complex structures than text data. Using traditional encryption methods alone to ensure security in image data types can create security weaknesses. For this reason, nowadays, some traditional methods are combined with each other or different methods to encrypt image data. In this study, 131 articles were examined and image encryption methods were classified according to the traditional methods they contain. Studies on both colored and gray level images have been handled together. Finally, the images used in the articles were compared with each other in many ways and the results were presented graphically.

Nuclei Segmentation is a very essential and intermediate step for automatic cancer detection from H and E stained histopathology images. In the recent advent, the rise of Convolutional Neural Network (CNN), has enabled researchers to detect nuclei automatically from histopathology images with higher accuracy. However, the performance of automatic nuclei segmentation by CNN is fraught with overfitting, due to very less number of annotated segmented images available. Indeed, we find that the problem of nuclei segmentation is an unsupervised problem, because still now there is no automatic tool available which can make annotated images (nuclei segmented images) accurately, to the best of our knowledge. In this research article, we present a Logarithmic-Base2 of Gaussian (Log-Base2-G) Kernel which has the ability to track only the nuclei portions automatically from Colorectal Cancer H and E stained histopathology images. First, Log-Base2-G Kernel is applied to the input images. Thereafter, we apply an adaptive Canny Edge detector, in order to segment only the nuclei edges from H and E stained histopathology images. Experimental results revealed that our proposed method achieved higher accuracy and F1 score, without the help of any annotated data which is a significant improvement. We have used two different datasets (Con-SeP dataset, and Glass-contest dataset, both contains Colorectal Cancer histopathology images) to check the effectiveness and validity of our proposed method. These results have shown that our proposed method outperformed other image processing or unsupervised methods both qualitatively and quantitatively.

Prostate cancer is the most common non-skin related cancer affecting 1 in 7 men in the United States. Treatment of patients with prostate cancer still remains a difficult decision-making process that requires physicians to balance clinical benefits, life expectancy, comorbidities, and treatment-related side effects. Gleason score (a sum of the primary and secondary Gleason patterns) solely based on morphological prostate glandular architecture has shown as one of the best predictors of prostate cancer outcome. Significant progress has been made on molecular subtyping prostate cancer delineated through the increasing use of gene sequencing. Prostate cancer patients with Gleason score of 7 show heterogeneity in recurrence and survival outcomes. Therefore, we propose to assess the correlation between histopathology images and genomic data with disease recurrence in prostate tumors with a Gleason 7 score to identify prognostic markers. In the study, we identify image biomarkers within tissue WSIs by modeling the spatial relationship from automatically created patches as a sequence within WSI by adopting a recurrence network model, namely long short-term memory (LSTM). Our preliminary results demonstrate that integrating image biomarkers from CNN with LSTM and genomic pathway scores, is more strongly correlated with patients recurrence of disease compared to standard clinical markers and engineered image texture features. The study further demonstrates that prostate cancer patients with Gleason score of 4+3 have a higher risk of disease progression and recurrence compared to prostate cancer patients with Gleason score of 3+4.

Pathology diagnosis of colorectal cancer is time-consuming and requires a high level of expertise. However, it is an essential step towards establishing the adequate treatment. The need to analyse a large number of these histopathological images calls for automatic tools capable of aiding pathologists in this arduous task. Deep learning techniques, together with the wealth of data available nowadays, provide a promising candidate for such job. Adopting state-of-the-art artificial intelligence algorithms, we developed a model to accurately detect colorectal cancer in digitalised histopathological whole-slide images. Our end-to-end approach uses the principles of multiple-instance learning combined with deep convolutional neural networks in order to fully leverage the information contained within each image and make robust predictions at the patient's level. The model also allows to highlight the areas in the slide most likely to harbour tumour tissue. Given the finite computational resources available, working at maximum resolution can be detrimental. Therefore, we explored the impact of lowering the working image resolution. The algorithms were trained and validated on a subset of more than 1300 patients of the Molecular Epidemiology of Colorectal Cancer study with histopathology images available. These images gave rise to [Formula: see text] tiles of [Formula: see text] pixels each. Once we identified the best-performing model we put it to the test on images from The Cancer Genome Atlas. We obtained the best outcomes working at 4 μm/pix, achieving the following metrics on the test dataset: F1-Score of 0.96, a Matthews correlation coefficient of 0.92 and an area under the receiver operating characteristic curve of 0.99. These results are exceptional and prove that computational costs can be reduced while keeping the performance up to standard.

Due to the complexity of histopathology tissues, an accurate classification and segmentation of cancer diagnosis is a challenging task in computer vision. The nuclei segmentation of microscopic images is a key prerequisite for cancerous pathological image analysis. However, an accurate nuclei segmentation is a long running major challenge due to the enormous color variability of staining, nuclei shapes, sizes, and clustering of overlapping cells. To address these challenges and early diagnosis as well as reduce the bias decisions of expert lab technician of cancer in clinical practice, the authors study the classification of computer-aided frameworks and automatic nuclei segmentation frameworks based on histopathology images by convolutional deep learning. The authors have used a publicly available PatchCamelyon and 2018 Data Science Bowl histology image dataset for this study. The results are compared and expected to be useful clinically for technician experts in the analysis of cancer diagnosis and the survival chances of patients.