Finding Event Videos via Image Search Engine

Transfer learning algorithms (TLAs) are often used to solve the distribution discrepancy issue in fingerprint-based indoor localization. However, existing TLAs cannot react well to real time changes in the environmental dynamics of the target space due to three remarkable shortcomings: a) redundant knowledge in source domain may lead to "negative transfer"; b) the required target domain samples to calculate the distributions are unrealistically feasible for real-time positioning; c) they cannot transfer knowledge efficiently across domains with heterogeneous feature spaces. In this paper, we propose TransLoc, a heterogeneous knowledge transfer framework for fingerprint-based indoor localization, which can perform knowledge transfer efficiently even with only one sample in the target domain. Specifically, we first refine the source domain according to the target domain by removing redundant knowledge in the source domain. Then, we derive a cross-domain mapping, which transfers the specific knowledge of one domain to another domain, to construct a homogeneous feature space. In this new feature space, the transfer weights are computed for training a classifier for target location prediction. To further train the framework efficiently, we combine the mapping and weights learning into a joint objective function and solve it by a three-step iterative optimization algorithm. Extensive simulation and real-world experimental results verify that TransLoc not only significantly outperforms state-of-the-art methods but is also very robust to changing environment.

In an indoor positioning system (IPS), transfer learning (TL) methods are commonly used to predict the location of mobile devices under the assumption that all training instances of the target domain are given in advance. However, this assumption has been criticized for its shortcomings in dealing with the problem of signal distribution variations, especially in a dynamic indoor environment. The reasons are: collecting a sufficient number of training instances is costly, the training instances may arrive online, the feature spaces of the target and source domains may be different, and negative knowledge may be transferred in the case of a redundant source domain. In this work, we proposed an online heterogeneous transfer learning (OHetTLAL) algorithm for IPS-based RSS fingerprinting to improve the positioning performance in the target domain by fusing both source and target domain knowledge. The source domain was refined based on the target domain to avoid negative knowledge transfer. The co-occurrence measure of the feature spaces (Cmip) was used to derive the homogeneous new feature spaces, and the features with higher weight values were selected for training the classifier because they could positively affect the location prediction of the target. Thus, the objective function was minimized over the new feature spaces. Extensive experiments were conducted on two real-world scenarios of datasets, and the predictive power of the different modeling techniques were evaluated for predicting the location of a mobile device. The results have revealed that the proposed algorithm outperforms the state-of-the-art methods for fingerprint-based indoor positioning and is found robust to changing environments. Moreover, the proposed algorithm is not only resilient to fluctuating environments but also mitigates the model's overfitting problem.

Unsupervised partial domain adaptation (PDA) is a unsupervised domain adaptation problem which assumes that the source label space subsumes the target label space. A critical challenge of PDA is the negative transfer problem, which is triggered by learning to match the whole source and target domains. To mitigate negative transfer, we note a fact that, it is impossible for a source sample of outlier classes to find a target sample of the same category due to the absence of outlier classes in the target domain, while it is possible for a source sample of shared classes. Inspired by this fact, we exploit the cycle inconsistency, i.e., category discrepancy between the original features and features after cycle transformations, to distinguish outlier classes apart from shared classes in the source domain. Accordingly, we propose to filter out source samples of outlier classes by weight suppression and align the distributions of shared classes between the source and target domains by adversarial learning. To learn accurate weight assignment for filtering out outlier classes, we design cycle transformations based on domain prototypes and soft nearest neighbor, where center losses are introduced in individual domains to reduce the intra-class variation. Experiment results on three benchmark datasets demonstrate the effectiveness of our proposed method.KeywordsUnsupervised partial domain adaptationNegative transferCycle inconsistency

Transfer learning has shown promising results in leveraging loosely labeled Web images (source domain) to learn a robust classifier for the unlabeled consumer videos (target domain). Existing transfer learning methods typically apply source domain data to learn a fixed model for predicting target domain data once and for all, ignoring rapidly updating Web data and continuously changes of users requirements. We propose an incremental transfer learning framework, in which heterogeneous knowledge are integrated and incrementally added to update the target classifier during learning process. Under the framework, images (image source domain) queried from Web image search engine and videos (video source domain) from existing action datasets are adopted to provide static information and motion information of the target video, respectively. For the image source domain, images are partitioned into several groups according to their semantic information. And for the video source domain, videos are divided in the same way. Unlike traditional methods which measure relevance between the source group and the whole target domain videos, the group weights in this paper are treated as latent variables for each target domain video and learned automatically according to the probability distribution difference between the individual source group and target domain videos. Experimental results on the two challenging video datasets (i.e., CCV and Kodak) demonstrate the effectiveness of our proposed method.KeywordsTarget DomainGroup WeightTarget ClassifierTransfer LearningSource DomainThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In pattern classification, we may have a few labeled data points in the target domain, but a number of labeled samples are available in another related domain (called the source domain). Transfer learning can solve such classification problems via the knowledge transfer from source to target domains. The source and target domains can be represented by heterogeneous features. There may exist uncertainty in domain transformation, and such uncertainty is not good for classification. The effective management of uncertainty is important for improving classification accuracy. So, a new belief-based bidirectional transfer classification (BDTC) method is proposed. In BDTC, the intraclass transformation matrix is estimated at first for mapping the patterns from source to target domains, and this matrix can be learned using the labeled patterns of the same class represented by heterogeneous domains (features). The labeled patterns in the source domain are transferred to the target domain by the corresponding transformation matrix. Then, we learn a classifier using all the labeled patterns in the target domain to classify the objects. In order to take full advantage of the complementary knowledge of different domains, we transfer the query patterns from target to source domains using the K-NN technique and do the classification task in the source domain. Thus, two pieces of classification results can be obtained for each query pattern in the source and target domains, but the classification results may have different reliabilities/weights. A weighted combination rule is developed to combine the two classification results based on the belief functions theory, which is an expert at dealing with uncertain information. We can efficiently reduce the uncertainty of transfer classification via the combination strategy. Experiments on some domain adaptation benchmarks show that our method can effectively improve classification accuracy compared with other related methods.

Unsupervised domain adaptation aims to transfer knowledge from labeled source domain to unlabeled target domain. Recently, multisource domain adaptation (MDA) has begun to attract attention. Its performance should go beyond simply mixing all source domains together for knowledge transfer. In this article, we propose a novel prototype-based method for MDA. Specifically, for solving the problem that the target domain has no label, we use the prototype to transfer the semantic category information from source domains to target domain. First, a feature extraction network is applied to both source and target domains to obtain the extracted features from which the domain-invariant features and domain-specific features will be disentangled. Then, based on these two kinds of features, the named inherent class prototypes and domain prototypes are estimated, respectively. Then a prototype mapping to the extracted feature space is learned in the feature reconstruction process. Thus, the class prototypes for all source and target domains can be constructed in the extracted feature space based on the previous domain prototypes and inherent class prototypes. By forcing the extracted features are close to the corresponding class prototypes for all domains, the feature extraction network is progressively adjusted. In the end, the inherent class prototypes are used as a classifier in the target domain. Our contribution is that through the inherent class prototypes and domain prototypes, the semantic category information from source domains is transformed into the target domain by constructing the corresponding class prototypes. In our method, all source and target domains are aligned twice at the feature level for better domain-invariant features and more closer features to the class prototypes, respectively. Several experiments on public data sets also prove the effectiveness of our method.

Population-based structural health monitoring (PBSHM) involves utilising knowledge from one set of structures in a population and applying it to a different set, such that predictions about the health states of each member in the population can be performed and improved. Central ideas behind PBSHM are those of knowledge transfer and mapping. In the context of PBSHM, knowledge transfer involves using information from a structure, defined as a source domain, where labels are known for a given feature, and mapping these onto the unlabelled feature space of a different, target domain structure. If the mapping is successful, a machine learning classifier trained on the transformed source domain data will generalise to the unlabelled target domain data; i.e. a classifier built on one structure will generalise to another, making Structural Heath Monitoring (SHM) cost-effective and applicable to a wide range of challenging industrial scenarios. This process of mapping features and labels across source and target domains is defined as domain adaptation, a subcategory of transfer learning. However, a key assumption in conventional domain adaptation methods is that there is consistency between the feature and label spaces. This means that the features measured from one structure must be the same dimension as the other (i.e. the same number of spectral lines of a transmissibility), and that labels associated with damage locations, classification and assessment, exist on both structures. These consistency constraints can be restrictive, limiting to which types of population domain adaptation can be applied. This paper, therefore, provides a mathematical underpinning for when domain adaptation is possible in a structural dynamics context, with reference to topology of a graphical representation of structures. By defining when conventional domain adaptation is applicable in a structural dynamics setting, approaches are discussed that could overcome these consistency restrictions. This approach provides a general means for performing transfer learning within a PBSHM context for structural dynamics-based features.KeywordsPopulation-based structural health monitoringTransfer learningDomain adaptation

Domain-transfer learning is a machine learning task to explore a source domain data set to help the learning problem in a target domain. Usually, the source domain has sufficient labeled data, while the target domain does not. In this paper, we propose a novel domain-transfer convolutional model by mapping a target domain data sample to a proxy in the source domain and applying a source domain model to the proxy for the purpose of prediction. In our framework, we firstly represent both source and target domains to feature vectors by two convolutional neural networks and then construct a proxy for each target domain sample in the source domain space. The proxy is supposed to be matched to the corresponding target domain sample convolutional representation vector well. To measure the matching quality, we proposed to maximize their squared-loss mutual information (SMI) between the proxy and target domain samples. We further develop a novel neural SMI estimator based on a parametric density ratio estimation function. Moreover, we also propose to minimize the classification error of both source domain samples and target domain proxies. The classification responses are also smoothened by manifolds of both the source domain and proxy space. By minimizing an objective function of SMI, classification error, and manifold regularization, we learn the convolutional networks of both source and target domains. In this way, the proxy of a target domain sample can be matched to the source domain data and thus benefits from the rich supervision information of the source domain. We design an iterative algorithm to update the parameters alternately and test it over benchmark data sets of abnormal behavior detection in video, Amazon product reviews sentiment analysis, etc.

Annotating events in uncontrolled videos is a challenging task. Most of the previous work focuses on obtaining concepts from numerous labeled videos. But it is extremely time consuming and labor expensive to collect a large amount of required labeled videos for modeling events under various circumstances. In this paper, we try to learn models for video event annotation by leveraging abundant Web images which contains a rich source of information with many events taken under various conditions and roughly annotated as well. Our method is based on a new discriminative structural model called Cross-Domain Structural Model (CDSM) to transfer knowledge from Web images (source domain) to consumer videos (target domain), by jointly modeling the interaction between videos and images. Specifically, under this framework we build a common feature subspace to deal with the feature distribution mismatching between the video domain and the image domain. Further, we propose to use weak semantic attributes to describe events, which can be obtained with no or little labor. Experimental results on challenging video datasets demonstrate the effectiveness of our transfer learning method.

Many sleep studies suffer from the problem of insufficient data to fully utilize deep neural networks as different labs use different recordings set ups, leading to the need of training automated algorithms on rather small databases, whereas large annotated databases are around but cannot be directly included into these studies for data compensation due to channel mismatch. This work presents a deep transfer learning approach to overcome the channel mismatch problem and transfer knowledge from a large dataset to a small cohort to study automatic sleep staging with single-channel input. We employ the state-of-the-art SeqSleepNet and train the network in the source domain, i.e. the large dataset. Afterwards, the pretrained network is finetuned in the target domain, i.e. the small cohort, to complete knowledge transfer. We study two transfer learning scenarios with slight and heavy channel mismatch between the source and target domains. We also investigate whether, and if so, how finetuning entirely or partially the pretrained network would affect the performance of sleep staging on the target domain. Using the Montreal Archive of Sleep Studies (MASS) database consisting of 200 subjects as the source domain and the Sleep-EDF Expanded database consisting of 20 subjects as the target domain in this study, our experimental results show significant performance improvement on sleep staging achieved with the proposed deep transfer learning approach. Furthermore, these results also reveal the essential of finetuning the feature-learning parts of the pretrained network to be able to bypass the channel mismatch problem.

Multisource unsupervised domain adaptation (MUDA) is an important and challenging topic for target classification with the assistance of labeled data in source domains. When we have several labeled source domains, it is difficult to map all source domains and target domain into a common feature space for classifying the targets well. In this article, a new progressive multisource domain adaptation network (PMSDAN) is proposed to further improve the classification performance. PMSDAN mainly consists of two steps for distribution alignment. First, the multiple source domains are integrated as one auxiliary domain to match the distribution with the target domain. By doing this, we can generally reduce the distribution discrepancy between each source and target domains, as well as the discrepancy between different source domains. It can efficiently explore useful knowledge from the integrated source domain. Second, to mine assistance knowledge from each source domain as much as possible, the distribution of the target domain is separately aligned with that of each source domain. A weighted fusion method is employed to combine the multiple classification results for making the final decision. In the optimization of domain adaption, weighted hybrid maximum mean discrepancy (WHMMD) is proposed, and it considers both the interclass and intraclass discrepancies. The effectiveness of the proposed PMSDAN is demonstrated in the experiments comparing with some state-of-the-art methods.

Cross-domain aspect extraction for sentiment analysis: A transductive learning approach

Supervised transfer learning algorithms utilize labeled data from auxiliary domains for learning in another domain where labeled data is scarce or absent. Given sufficient cross-domain corresponding instances, one can learn a robust transformation that maps the features across the domains by using any multi-output regression task. However, this cross-domain corresponding data is not available for real-world transfer tasks across heterogeneous feature spaces such as, cross-domain activity recognition and cross-lingual text/sentiment classification. In this paper, we present a shared label space driven algorithm that transfers labeled knowledge between heterogeneous feature spaces. The proposed algorithm treats the similar label distributions across the domains as pivots to generate cross-domain corresponding data. The shared label distributions and the corresponding data is obtained from the random forest models of the source and target domain. The experimental results on synthetic and real-world benchmark datasets having dissimilar modalities validate the performance of the proposed algorithm against state-of-the-art heterogeneous transfer learning approaches.

In this paper, we propose new approaches for action and event recognition by leveraging a large number of freely available Web videos (e.g., from Flickr video search engine) and Web images (e.g., from Bing and Google image search engines). We address this problem by formulating it as a new multi-domain adaptation problem, in which heterogeneous Web sources are provided. Specifically, we are given different types of visual features (e.g., the DeCAF features from Bing/Google images and the trajectory-based features from Flickr videos) from heterogeneous source domains and all types of visual features from the target domain. Considering the target domain is more relevant to some source domains, we propose a new approach named multi-domain adaptation with heterogeneous sources (MDA-HS) to effectively make use of the heterogeneous sources. In MDA-HS, we simultaneously seek for the optimal weights of multiple source domains, infer the labels of target domain samples, and learn an optimal target classifier. Moreover, as textual descriptions are often available for both Web videos and images, we propose a novel approach called MDA-HS using privileged information (MDA-HS+) to effectively incorporate the valuable textual information into our MDA-HS method, based on the recent learning using privileged information paradigm. MDA-HS+ can be further extended by using a new elastic-net-like regularization. We solve our MDA-HS and MDA-HS+ methods by using the cutting-plane algorithm, in which a multiple kernel learning problem is derived and solved. Extensive experiments on three benchmark data sets demonstrate that our proposed approaches are effective for action and event recognition without requiring any labeled samples from the target domain.

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.