Ensemble Learning with LDA Topic Models for Visual Concept Detection

Video semantic concept detection is considered as an important research problem by the multimedia industry in recent years. Classification is the most accepted method used for concept detection, where, the output of the classification system is interpreted as semantic concepts. These concepts can be employed for automatic indexing, searching and retrieval of video objects. However, employed features have high dimensions and thus, concept detection with the existing classifiers experiences high computation complexity. In this paper, a new approach is proposed to reduce the classification complexity and the required time for learning and classification by choosing the most important features. For this purpose genetic algorithms are employed as a feature selector. Simulation results illustrate improvements in the behavior of the classifier.

We present a probabilistic ranking-driven classifier for the detection of video semantic concept, such as airplane, building, etc. Most existing concept detection systems utilize Support Vector Machines (SVM) to perform the detection and ranking of retrieved video shots. However, the margin maximization principle of SVM does not perform ranking optimization but merely classification error minimization. To tackle this problem, we exploit the sparse Bayesian kernel model, namely the relevance vector machine (RVM), as the classifier for semantic concept detection. Based on automatic relevance determination principle, RVM outputs the posterior probabilistic prediction of the semantic concepts. This inference output is optimal for ranking the target video shots, according to the Probabilistic Ranking Principle. The probability output of RVM on individual uni-modal features also facilitates probabilistic fusion of multi-modal evidences to minimize Bayes risk. We demonstrate both theoretically and empirically that RVM outperforms SVM for video semantic concept detection. The testings on TRECVID 07 dataset show that RVM produces statically significant improvements in MAP scores over the SVM-based methods.

The analytic hierarchy process (AHP) is a method to help people make a complex decision by analyzing and synthesizing multiple criteria for the decision objective in a hierarchy. We adopt the AHP as a new strategy for the semantic concept detection (SCD) in video so that multiple factors involved in SCD, including multiple modalities and relating concepts, can be hierarchically analyzed and synthesized. In this paper, we first explain, by an example, why and how the SCD problem can be analyzed by the AHP. Then, following the idea of the AHP, we develop a new rank aggregation (RA) method, called AHP-RA. Experimental results of RA for SCD in video show the effectiveness of this method.

The new global pandemic caused by the 2019 novel coronavirus (COVID-19), novel coronavirus pneumonia, has spread rapidly around the world, causing enormous damage to daily life, public health security, and the global economy. Early detection and treatment of COVID-19 infected patients are critical to prevent the further spread of the epidemic. However, existing detection methods are unable to rapidly detect COVID-19 patients, so infected individuals are not detected in a timely manner, which complicates the prevention and control of COVID-19 to some extent. Therefore, it is crucial to develop a rapid and practical COVID-19 detection method. In this work, we explored the application of deep learning in COVID-19 detection to develop a rapid COVID-19 detection method. Existing studies have shown that novel coronavirus pneumonia has significant radiographic performance. In this study, we analyze and select the features of chest radiographs. We propose a chest X-Ray (CXR) classification method based on the selected features and investigate the application of transfer learning in detecting pneumonia and COVID-19. Furthermore, we combine the proposed CXR classification method based on selected features with transfer learning and ensemble learning and propose an ensemble deep learning model based on transfer learning called COVID-ensemble to diagnose pneumonia and COVID-19 using chest x-ray images. The model aims to provide an accurate diagnosis for binary classification (no finding/pneumonia) and multivariate classification (COVID-19/No findings/ Pneumonia). Our proposed CXR classification method based on selection features can significantly improve the CXR classification accuracy of the CNN model. Using this method, DarkNet19 improved its binary and triple classification accuracies by 3.5% and 5.78%, respectively. In addition, the COVIDensemble achieved 91.5% accuracy in the binary classification task and 91.11% in the multi-category classification task. The experimental results demonstrate that the COVID-ensemble can quickly and accurately detect COVID-19 and pneumonia automatically through X-ray images and that the performance of this model is superior to that of several existing methods. Our proposed COVID-ensemble can not only overcome the limitations of the conventional COVID-19 detection method RT-PCR and provide convenient and fast COVID-19 detection but also automatically detect pneumonia, thereby reducing the pressure on the medical staff. Using deep learning models to automatically diagnose COVID-19 and pneumonia from X-ray images can serve as a fast and efficient screening method for COVID-19 and pneumonia.

In this paper a discriminant analysis (DA) technique called accelerated generalised subclass discriminant analysis (AGSDA) and its GPU implementation are presented. This method identifies a discriminant subspace of the input space in three steps: a) Gram matrix computation, b) eigenvalue decomposition of the between subclass factor matrix, and c) computation of the solution of a linear matrix system with symmetric positive semidefinite (SPSD) matrix of coefficients. Based on the fact that the computationally intensive parts of AGSDA, i.e. Gram matrix computation and identification of the SPSD linear matrix system solution, are highly parallelisable, a GPU implementation of AGSDA is proposed. Experimental results on large-scale datasets of TRECVID for event and concept detection show that our GPU-AGSDA method combined with LSVM outperforms LSVM alone in training time, memory consumption, and detection accuracy.

Single visual concept detection in videos is a hard task, especially for infrequent concepts or for those difficult to model. This question becomes even more difficult in the case of concept pairs. Two main directions may tackle this problem: 1) combine the predictions of their corresponding detectors in a way which is similar to usual information retrieval, or 2) build supervised learners for these pairs of concepts by generating annotations based on the occurrences of the two individual concepts. Each of these approaches have advantages and drawbacks. We evaluated them in the context of the concept pair detection subtask of the TRECVid 2013 semantic indexing (SIN) task and found that information retrieval-like fusions of concept detection scores outperforms the learning approaches. The described methods outperform the best official result of the evaluation campaign cited previously, by 9% in terms of relative improvement on MAP.

Recent research has discovered that leveraging ontology is an effective way to facilitate semantic video concept detection. As an explicit knowledge representation, a formal ontology definition usually consists of a lexicon, properties, and relations. In this paper, we present a comprehensive representation scheme for video semantic ontology in which all the three components are well studied. Specifically, we leverage LSCOM to construct the concept lexicon, describe concept property as the weights of different modalities which are obtained manually or by data-driven approach, and model two types of concept relations (i.e., pairwise concept correlation and hierarchical relation). In contrast with most existing ontologies which are only focused on one or two components for domain-specific videos, the proposed ontology is more comprehensive and general. To validate the effectiveness of this ontology, we further apply it to video concept detection. The experiments on TRECVID 2005 corpus have demonstrated a superior performance compared to existing key approaches to video concept detection.

In this paper, we propose a novel semi-supervised feature analyzing framework for multimedia data understanding and apply it to three different applications: image annotation, video concept detection and 3-D motion data analysis. Our method is built upon two advancements of the state of the art: (1) <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2, 1</sub> -norm regularized feature selection which can jointly select the most relevant features from all the data points. This feature selection approach was shown to be robust and efficient in literature as it considers the correlation between different features jointly when conducting feature selection; (2) manifold learning which analyzes the feature space by exploiting both labeled and unlabeled data. It is a widely used technique to extend many algorithms to semi-supervised scenarios for its capability of leveraging the manifold structure of multimedia data. The proposed method is able to learn a classifier for different applications by selecting the discriminating features closely related to the semantic concepts. The objective function of our method is non-smooth and difficult to solve, so we design an efficient iterative algorithm with fast convergence, thus making it applicable to practical applications. Extensive experiments on image annotation, video concept detection and 3-D motion data analysis are performed on different real-world data sets to demonstrate the effectiveness of our algorithm.

This paper presents a high-level framework for smart robot perception to learn semantic concepts from videos that crawled from known Internet video sites (e.g. youtube, video-google). Smart perception is one of the most challenging problems for industrial robots. The key barrier for smart robot perception such as object detection and/or category classification is lack of annotated training data. Internet is a potential repository to provide a reliable source for semantic concept learning. In this paper, we will propose a novel Internet video-mining approach to bridge the gap between the demand of large-scale semantic concepts and the shortage of of annotated data. An automated video source discovery method will be addressed in concepts detection from the massive Internet videos. Illustrative experimental results with Tera-bytes level videos will be discussed to prove that the addressed method is effective and efficient in smart robot perception.

Multi-level feature representations for video semantic concept detection

Preparing a Two Column Paper with MS Word for Windows

The main aim in ensemble learning is using multiple classifiers rather than one classifier to aggregate classifiers' outputs for more accurate classification. Generating an ensemble classifier generally is composed of three steps: selecting a base classifier, applying a sampling strategy to generate different simple classifiers and aggregating the classifiers' outputs. This paper focuses on the classifiers' outputs aggregation step in ensemble learning and presents a new interval-based aggregation approach using Bagging and Interval Agreement Approach (IAA). Bagging is an ensemble learning approach to generate ensembles of classifiers by manipulation of the training data set and IAA is an aggregation approach in decision making which was introduced to combine decision makers' opinions when they present their opinions by intervals. In this paper, we use Bagging approach to generate uncertainty intervals for simple classifiers in ensemble learning and implement IAA to aggregate the intervals with the aim of capturing uncertainty. In fact, we design some experiments to encourage researchers to use interval modeling in ensemble learning because it preserves more uncertainty and leads to more accurate classification. We compare the results of implementing the proposed method to the majority vote, as the most commonly used aggregation function in ensemble learning, for 10 medical data sets. The results show the better performance of the proposed interval-based aggregation approach in binary classification when it comes to ensemble learning. The Bayesian signed-rank test confirms the competency of our proposed approach in this research.

Recently, a great progress in automatic image captioning has been achieved by using semantic concepts detected from the image. However, we argue that existing concepts-to-caption framework, in which the concept detector is trained using the image-caption pairs to minimize the vocabulary discrepancy, suffers from the deficiency of insufficient concepts. The reasons are two-fold: 1) the extreme imbalance between the number of occurrence positive and negative samples of the concept and 2) the incomplete labeling in training captions caused by the biased annotation and usage of synonyms. In this paper, we propose a method, termed online positive recall and missing concepts mining, to overcome those problems. Our method adaptively re-weights the loss of different samples according to their predictions for online positive recall and uses a two-stage optimization strategy for missing concepts mining. In this way, more semantic concepts can be detected and a high accuracy will be expected. On the caption generation stage, we explore an element-wise selection process to automatically choose the most suitable concepts at each time step. Thus, our method can generate more precise and detailed caption to describe the image. We conduct extensive experiments on the MSCOCO image captioning data set and the MSCOCO online test server, which shows that our method achieves superior image captioning performance compared with other competitive methods.

Semantic concept detection from multimedia features enables high-level access to multimedia content. While constructing robust detectors is feasible for concepts with sufficient training samples, concepts with fewer training samples are hard to train efficiently. Comparable performance may be possible if the dependence of these concepts on the ones that can be robustly modeled is exploited. In this paper we show this phenomenon using the TREC Video 2002 Corpus as a test bed. Using a basic set of 12 semantic concepts modeled with support vector machines, we predict presence of 4 other concepts. We then compare the performance of these predictors with direct SVM models for these 4 concepts and observe improvements of up to 150% in average precision.

Support vector machines (SVMs) has been accepted as a fashionable method in machine learning community. However, it cannot be easily scaled to handle large scale problems for its time and space complexity that is around quadratic with respect to the number of training samples. This paper proposes to combine distributed SVMs by simple Bayesian formalism rules (B-SVMs). B-SVMs randomly decomposes a large-scale task into many smaller and simpler sub-tasks in training phase and uses simple Bayesian formalism rules to make decision for final classification in test phase. B-SVMs was compared with single SVMs that is trained on entire training data set, parallel SVMs combined by majority voting (MV-SVMs), and one kind of fast modular SVMs (FM-SVMs). Experimental results on four problems show that B-SVMs can get higher accuracy than MV-SVMs and FM-SVMs does, the proposed algorithm can significantly reduce training and test time. More importantly, it produces test accuracy that is almost the same as single SVMs does.