Multi-instance learning for skin biopsy image features recognition

Multi-instance learning (MIL) has been widely applied to diverse applications involving complicated data objects, such as images and genes. However, most existing MIL algorithms can only handle small- or moderate-sized data. In order to deal with large-scale MIL problems, we propose MIL based on the vector of locally aggregated descriptors representation (miVLAD) and MIL based on the Fisher vector representation (miFV), two efficient and scalable MIL algorithms. They map the original MIL bags into new vector representations using their corresponding mapping functions. The new feature representations keep essential bag-level information, and at the same time lead to excellent MIL performances even when linear classifiers are used. Thanks to the low computational cost in the mapping step and the scalability of linear classifiers, miVLAD and miFV can handle large-scale MIL data efficiently and effectively. Experiments show that miVLAD and miFV not only achieve comparable accuracy rates with the state-of-the-art MIL algorithms, but also have hundreds of times faster speed. Moreover, we can regard the new miVLAD and miFV representations as multiview data, which improves the accuracy rates in most cases. In addition, our algorithms perform well even when they are used without parameter tuning (i.e., adopting the default parameters), which is convenient for practical MIL applications.

A Novel Automatic Image Annotation Method Based on Multi-Instance Learning

Pineapple is one of the potential commodities in Indonesia. This is due to high market demand, potential suitable land in Indonesia and public awareness of fruit supply. The main factor of crop failure in pineapple plants is the delay in handling pineapple plant diseases. To identify in image processing requires class grouping. Self-Organizing Map (SOM) which divides the input pattern into several groups so that the network output is in the form of the group that is most similar to the given input. However, the SOM algorithm requires data input that characterizes an object to facilitate the identification process. So, in this study the SOM algorithm was improved through color feature extraction with parameters Red, Green, Blue, Hue, Saturation and Value, as well as texture feature extraction with parameters contrast, correlation, energy, and homogeneity in the Gray Level Co-occurrence Matrix (GLCM). Based on the results of tests carried out by the SOM algorithm with color and texture feature extraction parameters, it is able to assist in increasing the accuracy value. The results of testing the SOM model with color and texture feature extraction obtained a precision value of 93.33%, recall of 92.31% and accuracy of 92.78%.

Attention mechanisms are often used in deep neural networks for distantly supervised relation extraction (DS-RE) to distinguish valid from noisy instances. However, traditional 1- D vector attention models are insufficient for the learning of different contexts in the selection of valid instances to predict the relationship for an entity pair. To alleviate this issue, we propose a novel multi-level structured (2-D matrix) self-attention mechanism for DS-RE in a multi-instance learning (MIL) framework using bidirectional recurrent neural networks. In the proposed method, a structured word-level self-attention mechanism learns a 2-D matrix where each row vector represents a weight distribution for different aspects of an instance regarding two entities. Targeting the MIL issue, the structured sentence-level attention learns a 2-D matrix where each row vector represents a weight distribution on selection of different valid instances. Experiments conducted on two publicly available DS-RE datasets show that the proposed framework with a multi-level structured self-attention mechanism significantly outperform state-of-the-art baselines in terms of PR curves, P@N and F1 measures.

Aiming at the problem of object-based image retrieval, a novel semi-supervised multi-instance learning (MIL) algorithm based on RS (rough set) attribute reduction and Transductive support vector machine (TSVM) has been presented---RSTSVM- MIL algorithm. This algorithm regards the whole image as a bag, and the low-level visual feature of the segmented regions as instances, in order to transform every bag to a single sample, RSTSVM-MIL first generates a collection of visual-word to construct projection space by K-means clustering method in the instance feature space, then a nonlinear mapping is defined using these visual-word and maps every bag to a point in the projection space, extracting all bags' projection feature, which converts MIL problem to a standard supervised learning problem. Finally, in order to improve the efficiency of classifier training and to use a large amount of unlabeled image together with the labeled image to build better classifiers, we use RS (rough set) method to reduce the redundant information in projection feature firstly, then semi-supervised learning algorithm TSVM be used to train classifier. Experimental results on the SIVAL image set show that solving MIL problems using semi-supervised learning is feasible, and the RSTSVM-MIL is competitive with other state-of-the-art MIL algorithms in object- based image retrieval problem.

This article described the feature extraction methods of crop disease based on computer image processing technology in detail. Based on color, texture and shape feature extraction method in three aspects features and their respective problems were introduced start from the perspective of lesion leaves. Application research of image feature extraction in the filed of crop disease was reviewed in recent years. The results were analyzed that about feature extraction methods, and then the application of image feature extraction techniques in the future detection of crop diseases in the field of intelligent was prospected.

In multi-instance learning, each example is represented by a bag of instances while associated with a binary label. Under standard multi-instance learning settings, one example is labeled as a positive bag if at least one of its instances is positive. Otherwise, it is labeled as a negative bag. Although based on the above assumption, standard multi-instance learning has achieved much success in solving diverse learning tasks, there are still many real-world problems where this assumption may not necessarily hold. Therefore, researchers aimed to expand the underlying assumption of standard multi-instance learning where two frameworks of generalized multi-instance learning have been proposed. In this paper, the problem definition, learning algorithms and also experimental data sets related to either generalized multi-instance learning framework are briefly reviewed.

Fine-grained classification of whole slide images (WSIs) is essential in precision oncology, enabling precise cancer diagnosis and personalized treatment strategies. The core of this task involves distinguishing subtle morphological variations within the same broad category of gigapixel-resolution images, which presents a significant challenge. While the multi-instance learning (MIL) paradigm alleviates the computational burden of WSIs, existing MIL methods often overlook hierarchical label correlations, treating fine-grained classification as a flat multi-class classification task. To overcome these limitations, we introduce a novel hierarchical multi-instance learning (HMIL) framework. By facilitating on the hierarchical alignment of inherent relationships between different hierarchy of labels at instance and bag level, our approach provides a more structured and informative learning process. Specifically, HMIL incorporates a class-wise attention mechanism that aligns hierarchical information at both the instance and bag levels. Furthermore, we introduce supervised contrastive learning to enhance the discriminative capability for fine-grained classification and a curriculum-based dynamic weighting module to adaptively balance the hierarchical feature during training. Extensive experiments on our large-scale cytology cervical cancer (CCC) dataset and two public histology datasets, BRACS and PANDA, demonstrate the state-of-the-art class-wise and overall performance of our HMIL framework. Our source code is available at https://github.com/ChengJin-git/HMIL.

Presently, most cancer diagnosis is based on human visual examination of images in a qualitative manner. Human visual grading for microscopic biopsy images is very time-consuming, subjective, and inconsistent due to inter-and intra-observer variations. A more quantitative and reproducible approach for analyzing biopsy images is highly desired. In biopsy images, the characteristics of nuclei are the key to estimate the degree of malignancy. The microscopic biopsy images always suffer from the problem of impurities, undesirable elements, and uneven exposure. Thus, there is a need of an automatic cancer diagnosis system based on microscopic biopsy images using image-processing tools. Therefore, the cancer and its type will be detected in a very early stage for complete treatment and cure. This system helps pathologists to improve the accuracy and efficiency in detection of malignancy and to minimize the inter observer variation. In addition, the method may help physicians to analyze the image cell by using classification and clustering algorithms by staining characteristics of the cells. The various image-processing steps involved for cancer detection from biopsy images include acquisition, enhancement, segmentation, feature extraction, image representation, classification, and decision-making. With the help of image, processing tools the sizes of cells, nuclei, and cytoplasm as well as the mean distance between two nearest neighboring nuclei are estimated by the system.

Feature extraction is an important processing step in texture classification. For feature extraction in contourlet domain, statistical features for blocks of subband are computed. In this paper, we present an efficient feature vector extraction method for texture classification. For more discriminative feature a canonical correlation analysis method is propose for feature vector fused to the different sample of texture in the same cluster. The KNN (K-Nearest Neighbor) classifier is utilizing to perform texture classification.

Recent advances in programmed death-1 (PD-1) and programmed death-ligand 1 (PD-L1) immune checkpoint inhibitor (ICI) revolutionized the clinical practice in lung cancer treatment. The PD-L1 immunohistochemistry (IHC) test is a widely used biomarker for ICI responder in lung cancer. However, the accuracy of PD-L1 IHC test for selecting ICI responder is unsatisfactory, especially due to its low specificity. Therefore, methods which could effectively predict the efficacy of ICI are crucial for patient selection. In this article, we apply a deep neural network (DNN) to predict responders for anti-PD1 blockade on the basis of histopathological images of hematoxylin and eosin (H&E) stained tissue. The main difficulty to train DNN for responder prediction is the inability to accurately label at the image patch level. We employed a semi-supervised multi-instance learning (MIL) framework with adaptive positive patch selection in each region-of-interest (ROI). Using a dataset of 250 whole slide images (WSIs) of non-small cell lung cancer (111 responders and 139 nonresponders), we train a DNN-based MIL classifier on a case-level partition of the dataset (150 WSIs) and obtain an area under curve (AUC) of 0.773 for the test dataset (50 WSIs), which outperforms the PD-L1 IHC test (AUC=0.636). These results suggest that a DNN model can be used for assisting clinicians to make decision for treatment plan. We also confirm that the locations of adaptively selected positive patches can give valuable insights into the histological features associated with drug response.

A novel method of color and texture feature extraction for image retrieval is proposed. First, an input image is divided into 4×4 nonoverlapping blocks. Then, each block is quantized into two colors by color moment preserving and merging similar colors according to their color difference values under a preset threshold value. A few dominant colors are obtained, and a color histogram of the reconstructed image is computed for the color feature. For the texture feature, by preserving some color moments in the 4×4 neighborhood of each pixel, a bitmap is generated, which can be matched with 64 predefined bitmaps. Then a bitmap spectrum is obtained by counting bitmaps frequencies when all pixels of the input image are considered. An efficient method of similarity measurement is also presented. Experimental results show the proposed method performs well in terms of precision and recall.

While deep learning is undoubtedly the predominant learning technique across speech processing, it is still not widely used in health-based applications. The corpora available for health-style recognition problems are often small, both concerning the total amount of data available and the number of individuals present. The Bipolar Disorder corpus, used in the 2018 Audio/Visual Emotion Challenge, contains only 218 audio samples from 46 individuals. Herein, we present a multi-instance learning framework aimed at constructing more reliable deep learning-based models in such conditions. First, we segment the speech files into multiple chunks. However, the problem is that each of the individual chunks is weakly labelled, as they are annotated with the label of the corresponding speech file, but may not be indicative of that label. We then train the deep learning-based (ensemble) multi-instance learning model, aiming at solving such a weakly labelled problem. The presented results demonstrate that this approach can improve the accuracy of feedforward, recurrent, and convolutional neural nets on the 3-class mania classification tasks undertaken on the Bipolar Disorder corpus.

Expert radiologists can quickly extract a basic “gist” understanding of a medical image following less than a second exposure, leading to above-chance diagnostic classification of images. It is currently unclear whether this pattern of results is applicable to pathology, another medical imaging domain demanding visual diagnostic interpretation. To further characterize the rapid interpretation medical images, this study examined eye movements and diagnostic decision-making when pathologists were briefly exposed to digital whole slide images of melanocytic skin biopsies. Twelve resident (N = 5), fellow (N = 5), and attending pathologists (N = 2) with experience interpreting dermatopathology briefly viewed 48 cases presented for 500 ms each, and we tracked their eye movements towards abnormalities, their ability to classify images as containing or not containing invasive melanoma, and their ability to localize critical image regions. Results demonstrated rapid shifts of the eyes towards critical abnormalities, high diagnostic sensitivity and specificity, and surprisingly high accuracy localizing critical diagnostic regions. Furthermore, when pathologists fixated critical regions with their eyes, they were subsequently more likely to successfully localize that region on an outline of the image. Results are discussed relative to models of medical image interpretation and innovative methods for monitoring and assessing expertise development during medical education and training.

In this paper, a novel multi-instance learning (MIL) algorithm based on pyramid match kernel (PMK) and classifier ensemble is proposed for recognizing pornographic scene from image database. First, an improved JSEG image segmentation technique is deployed for dividing every image into several regions, and regards the whole image as a "bag", the low-level visual features (i.e. color and texture) of each segmented region as "instance". As a result, the pornographic images filtering problem can be transferred into a typical MIL problem. Second, similarity between the multi-instance bags is measured by PMK method, which allows MIL problem to be solved directly by the support vector machine (SVM). Finally, many base classifiers based on PMK with different levels are constructed, and the performance weighting rule is used to dynamically determine the weights of them, so the strategy of classifier ensemble is used to improve the filtering accuracy. In a real condition image set that the ratio of normal image to pornographic image is 9:1, experimental results show that the proposed algorithm, named PMKCE-MIL, is robust, and its performance is superior to other algorithms.