GM-Citation-KNN: Graph matching based multiple instance learning algorithm

The detection of human action in videos of busy natural scenes with dynamic background is of interest for applications such as video surveillance. Taking a conventional fully supervised approach, the spatio-temporal locations of the action of interest have to be manually annotated frame by frame in the training videos, which is tedious and unreliable. In this paper, for the first time, a weakly supervised action detection method is proposed which only requires binary labels of the videos indicating the presence of the action of interest. Given a training set of binary labelled videos, the weakly supervised learning (WSL) problem is recast as a multiple instance learning (MIL) problem. A novel MIL algorithm is developed which differs from the existing MIL algorithms in that it locates the action of interest spatially and temporally by globally optimising both interand intra-class distance. We demonstrate through experiments that our WSL approach can achieve comparable detection performance to a fully supervised learning approach, and that the proposed MIL algorithm significantly outperforms the existing ones.

For the past 2 decades, detection of buried explosive hazard has been studied extensively and several machine learning algorithms have been developed and adapted to this application. First, a pre-screener is used to identify areas of interest or alarms. Each alarm consists of a 3D data cube that corresponds to spatial down-track, crosstrack, and time respectively. Ground truth information is then used to label each alarm and generate labeled data to train a classifier to discriminate between targets and clutter objects. One of the main challenges in this approach is localizing the true depth of the alarm. On one hand, the buried object signature is not expected to cover all the depth values and extracting one global feature from all depth bins may not discriminate between object and clutter signatures effectively. On the other hand, depth ground truth is not available as this depends on the target type and size, soil properties, and other environmental conditions. Moreover, visually inspecting each alarm to select the optimal depth location is tedious, ambiguous, and not practical for very large training data. Two different approaches have been considered to train learning algorithms. The first one uses simple rules and machine learning algorithms to automate the selection of the optimal depth(s) for each alarm. The second approach avoids the labeling at the depth level and uses multiple instance learning (MIL) algorithms. In this context, each alarm is represented by a bag of multiple instances. Each instance corresponds to a feature extracted at a different depth. Since labels are needed at the bag level and not at the instance level, MIL does not require true depth information. In this paper, we propose a large-scale evaluation of the two approaches. For the first approach, we consider three methods to identify optimal depth locations and analyze their effect on the KNN and SVM classifiers. For the second approach, we consider four MIL algorithms that do not require depth information. For our analysis, we use large data collections accumulated across multiple dates and multiple test sites by a vehicle mounted downward looking ground penetrating radar (GPR) sensor. The data include a diverse set of buried explosive objects of varying shapes, metal content, and underground burial depths. Performance of all algorithms is analyzed using receiver operating characteristics (ROC).

Multiple instance learning: A survey of problem characteristics and applications

Twin support vector machines and subspace learning methods for microcalcification clusters detection

Since real-world data sets usually contain large instances, it is meaningful to develop efficient and effective multiple instance learning (MIL) algorithm. As a learning paradigm, MIL is different from traditional supervised learning that handles the classification of bags comprising unlabeled instances. In this paper, a novel efficient method based on extreme learning machine (ELM) is proposed to address MIL problem. First, the most qualified instance is selected in each bag through a single hidden layer feedforward network (SLFN) whose input and output weights are both initialed randomly, and the single selected instance is used to represent every bag. Second, the modified ELM model is trained by using the selected instances to update the output weights. Experiments on several benchmark data sets and multiple instance regression data sets show that the ELM-MIL achieves good performance; moreover, it runs several times or even hundreds of times faster than other similar MIL algorithms.

Multiple instance learning (MIL) is a form of weakly supervised learning for problems in which training instances are arranged into bags, and a label is provided for whole bags but not for individual instances. Most proposed MIL algorithms focus on bag classification, but more recently, the classification of individual instances has attracted the attention of the pattern recognition community. While these two tasks are similar, there are important differences in the consequences of instance misclassification. In this paper, the scoring function learned by MIL classifiers for the bag classification task is exploited for instance classification by adjusting the decision threshold. A new criterion for the threshold adjustment is proposed and validated using 7 reference MIL algorithms on 3 real-world data sets from different application domains. Experiments show considerable improvements in accuracy over these algorithms for instance classification. In some applications, the unweighted average recall increases by as much as 18%, while the Fi-score increases by 12%.

Multiple instance learning (MIL) is a new paradigm in machine learning that deals with classification of bags of instances, as opposed to the traditional view that aims at learning from single instances. In a typical MIL setting, a negative bag is composed of only negative instances. On the other hand, a bag is considered positive if it contains at least one positive instance. This learning approach provides a natural way of modeling several pattern recognition and computer vision problems, e.g. protein, document and image classification and object recognition, which inherently require learning under ambiguity. In many cases, MIL approaches performs better than the standard single instance learning (SIL) methods. One of the general approaches to MIL is to transform a given MIL problem into a corresponding SIL problem. This transformation is mainly done with selecting a set of representative instances from the training bags, and these group of studies basically differ from each other on how they perform this instance selection step. In this study, we revisit such a MIL approach called MILIS with a different instance selection mechanism. The experimental results show that the proposed approach performs better on MIL benchmark data sets as compared to the original algorithm.

Recently, bag of words (BoW) model has led to many significant results in visual object classification. However, due to the limited descriptive and discriminative ability of visual words, the resulting performance of visual object classification is still incomparable to its analogy in text domain, i.e. document categorization. Furthermore, for weakly labeled image data, where we only know whether an object is present or not, traditional learning based methods may suffer from background clutters and large appearance variations. To address these issues, we propose a novel visual phrase based Multiple Instance Learning (MIL) method. In this method, the visual phrase is first generated from over-segmented image regions of homogeneous appearance and visual words within each region, which may provide enhanced descriptive ability by enforcing the spatial coherency. Then a MIL algorithm is applied to efficiently learn from the weakly labeled image data. The experiments on benchmark datasets show that our proposed method always significantly outperforms several state-of-the-art algorithms, such as Spatial Pyramid Matching (SPM) [5] and Spatial-LTM [8].

Real-time video event detection in crowded scenes using MPEG derived features: A multiple instance learning approach

Multiple instance learning (MIL) is semisupervised learning that predicts the label of a bag with a wide diversity of instances. It has many applications and thus attracts increasingly more attention. In this paper, we propose a new MIL algorithm using the structural information of a bag to predict its label. In the proposed method, a bag is transformed into a graph, and spectral clustering is employed to divide the graph into several subgraphs. Then, the graph Fourier transform is utilized to extract the features of the subgraphs. Finally, an end-to-end neural network is used to predict the label of a bag with the extracted features. An empirical study with 25 datasets was conducted to validate the effectiveness of the proposed method. The experimental results show that the proposed method performs better than the 6 baseline methods on most datasets.

Microcalcification cluster detection using multiscale products based Hessian matrix via the Tsallis thresholding scheme

Mammography is the only technique currently used for detecting microcalcification (MC) clusters, an early indicator of breast cancer. However, mammographic images superimpose a three-dimensional compressed breast image onto two-dimensional projection views, resulting in overlapped anatomical breast structures that may obscure the detection and visualization of MCs. One possible solution to this problem is the use of cone beam computed tomography (CBCT) with a flat-panel (FP) digital detector. Although feasibility studies of CBCT techniques for breast imaging have yielded promising results, they have not shown how radiation dose and x-ray tube voltage affect the accuracy with which MCs are detected by CBCT experimentally. We therefore conducted a phantom study using a FP-based CBCT system with various mean glandular doses and kVp values. An experimental CBCT scanner was constructed with a data acquisition rate of 7.5 frames/s. 10.5 and 14.5 cm diameter breast phantoms made of gelatin were used to simulate uncompressed breasts consisting of 100% glandular tissue. Eight different MC sizes of calcium carbonate grains, ranging from 180-200 microm to 355-425 microm, were used to simulate MCs. MCs of the same size were arranged to form a 5 x 5 MC cluster and embedded in the breast phantoms. These MC clusters were positioned at 2.8 cm away from the center of the breast phantoms. The phantoms were imaged at 60, 80, and 100 kVp. With a single scan (360 degrees), 300 projection images were acquired with 0.5 x, 1x, and 2x mean glandular dose limit for 10.5 cm phantom and with 1x, 2x, and 4x for 14.5 cm phantom. A Feldkamp algorithm with a pure ramp filter was used for image reconstruction. The normalized noise level was calculated for each x-ray tube voltage and dose level. The image quality of the CBCT images was evaluated by counting the number of visible MCs for each MC cluster for various conditions. The average percentage of the visible MCs was computed and plotted as a function of the MGD, the kVp, and the average MC size. The results showed that the MC visibility increased with the MGD significantly but decreased with the breast size. The results also showed that the x-ray tube voltage affects the detection of MCs under different circumstances. With a 50% threshold, the minimum detectable MC sizes for the 10.5 cm phantom were 348(+/-2), 288(+/-7), 257(+/-2) microm at 3, 6, and 12 mGy, respectively. Those for the 14.5 cm phantom were 355 (+/-1), 307 (+/-7), 275 (+/-5) microm at 6, 12, and 24 mGy, respectively. With a 75% threshold, the minimum detectable MC sizes for the 10.5 cm phantom were 367 (+/-1), 316 (+/-7), 265 (+/-3) microm at 3, 6, and 12 mGy, respectively. Those for the 14.5 cm phantom were 377 (+/-3), 334 (+/-5), 300 (+/-2) microm at 6, 12, and 24 mGy, respectively.

Multiple instance learning (MIL) has emerged as the best solution for whole slide image (WSI) classification. It consists of dividing each slide into patches, which are treated as a bag of instances labeled with a global label. MIL includes two main approaches: instance-based and embedding-based. In the former, each patch is classified independently, and then, the patch scores are aggregated to predict the bag label. In the latter, bag classification is performed after aggregating patch embeddings. Even if instance-based methods are naturally more interpretable, embedding-based MILs have usually been preferred in the past due to their robustness to poor feature extractors. Recently, the quality of feature embeddings has drastically increased using self-supervised learning (SSL). Nevertheless, many authors continue to endorse the superiority of embedding-based MIL. We conduct 710 experiments across 4 datasets, comparing 10 MIL strategies, 6 self-supervised methods with 4 backbones, 4 foundation models, and various pathology-adapted techniques. Furthermore, we introduce 4 instance-based MIL methods, never used before in the pathology domain. We show that with a good SSL feature extractor, simple instance-based MILs, with very few parameters, obtain similar or better performance than complex, state-of-the-art (SOTA) embedding-based MIL methods, setting new SOTA results on the BRACS and Camelyon16 datasets. As simple instance-based MIL methods are naturally more interpretable and explainable to clinicians, our results suggest that more effort should be put into well-adapted SSL methods for WSI rather than into complex embedding-based MIL methods.

We address the task of automatic detection of lesions caused by multiple myeloma (MM) in femurs or other long bones from CT data. Such detection is already an important part of the multiple myeloma diagnosis and staging. However, it is so far performed mostly manually, which is very time consuming. We formulate the detection as a multiple instance learning (MIL) problem, where instances are grouped into bags and only bag labels are available. In our case, instances are regions in the image and bags correspond to images. This has the advantage of requiring only subject-level annotation (ground truth), which is much easier to get than voxel-level manual segmentation. We consider a generalization of the standard MIL formulation where we introduce a threshold on the number of required positive instances in positive bags. This corresponds better to the classification procedure used by the radiology experts and is more robust with respect to false positive instances. We extend several existing MIL algorithms to solve the generalized case by estimating the threshold during learning. We compare the proposed methods with the baseline method on a dataset of 220 subjects. We show that the generalized MIL formulation outperforms standard MIL methods for this task. For the task of distinguishing between healthy controls and MM patients with infiltrations, our best method makes almost no mistakes with a mean AUC of 0.982 and F1 = 0.965. We outperform the baseline method significantly in all conducted experiments.

Microcalcification (MC) clusters in mammograms can be important early signs of breast cancer in women. Accurate detection of MC clusters is an important but challenging problem. In this paper, we propose the use of a recently developed machine learning technique -- relevance vector machine (RVM) -- for automatic detection of MCs in digitized mammograms. RVM is based on Bayesian estimation theory, and as a feature it can yield a decision function that depends on only a very small number of so-called <i>relevance vectors</i>. We formulate MC detection as a supervised-learning problem, and use RVM to classify if an MC object is present or not at each location in a mammogram image. MC clusters are then identified by grouping the detected MC objects. The proposed method is tested using a database of 141 clinical mammograms, and compared with a support vector machine (SVM) classifier which we developed previously. The detection performance is evaluated using the free-response receiver operating characteristic (FROC) curves. It is demonstrated that the RVM classifier matches closely with the SVM classifier in detection performance, and does so with a much sparser kernel representation than the SVM classifier. Consequently, the RVM classifier greatly reduces the computational complexity, making it more suitable for real-time processing of MC clusters in mammograms.