Lifelong Machine Learning

The majority of machine learning research has focused on the single task learning (STL) approach where an hypothesis for a single task is induced from a set of supervised training examples. In contrast, one of the key aspects of human learning is that individuals face a sequence of learning problems over a lifetime. Humans take advantage of this by transferring knowledge from previously learned tasks to facilitate the learning of a new task. Life-long learning, a relatively new area of machine learning research, is concerned with the persistent and cumulative nature of learning (Thrun, 1997). Life-long learning considers situations in which a learner faces a series of different tasks and develops methods of retaining and using prior knowledge to improve the effectiveness (more accurate hypotheses) and efficiency (shorter training times) of learning. Related names for life-long learning in the literature are learning to learn and meta-learning. A challenge often faced by a life-long learning agent is a deficiency of training examples from which to develop accurate hypotheses. Machine learning theory tells us that this problem can be overcome with an appropriate inductive bias (Mitchell, 1997), one source being prior task knowledge (Baxter, 1995). Lacking a theory of knowledge transfer (Caruana, 1997, Thrun, 1997) that distinguishes knowledge from related and unrelated tasks, we have developed one and applied it to life-long learning problems, such as learning a more accurate medical diagnostic model from a small sample of patient data (Silver, 2000). The approach requires (1) a method of selectively transferring previously learned knowledge to a new task based on a measure of task relatedness and (2) a method of retaining learned task knowledge and its recall when learning a new task. In (Silver & Mercer, 1996) we introduced ηMTL, a modified version of the multiple task learning (MTL) method of functional transfer to provide a solution to the first problem of selective transfer. Using a measure of previously learned secondary task to primary task relatedness, an ηMTL network can favourably bias the induction of a hypothesis for a primary task. Section 3 reviews the necessary aspects of ηMTL. This paper focuses on the Task Rehearsal Method (TRM) to solve the second problem of retention and recall of learned task knowledge. TRM uses either the standard MTL or the ηMTL learning algorithms as the method of knowledge transfer and inductive bias. Task rehearsal is so named because previously learned tasks are relearned or rehearsed in parallel O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

Suggestion classification for opinion data is defined as identifying a given utterance by suggestion or non-suggestion class. In this paper, we introduce a method called LLMaxent which is the solution for the cross-domain suggestion classification. LLMaxent is a lifelong machine learning approach using maximum entropy (Maxent). In the course of lifelong learning, the drawn knowledge from the past tasks is retained and supported for the future learning. From that, we build a classifier by using labelled data in existed domains for suggestion classification in a new domain. The experimental results show that the proposed novel model can improve the performance of cross-domain suggestion classification. This is one of the preliminary research in lifelong machine learning using Maxent. Its effect is not only for suggestion classification but also for cross-domain text classification in general.

Most machine learning algorithms assume stationary environments, require a large number of training examples in advance, and begin the learning from scratch. In contrast, humans learn in changing environments with sequential training examples and leverage prior knowledge in new situations. To deal with real-world problems in changing environments, the ability to make human-like quick responses must be developed in machines. Many researchers have presented learning systems that assume the presence of hidden context and concept drift. In particular, several systems have been proposed that use ensembles of classifiers on sequential chunks of training examples. These systems can respond to gradual changes in large-scale data streams but have problems responding to sudden changes and leveraging prior knowledge of recurring contexts. Moreover, these are not pure online learning systems. We propose an online learning system that uses an ensemble of classifiers suited to recent training examples. We use experiments to show that this system can leverage prior knowledge of recurring contexts and is robust against various noise levels and types of drift.

Lifelong learning is the newly introduced approach to machine learning. It follows the true essence of developing learning based intelligent agents. The idea is to build models that can learn with experience in order to become more knowledgeable, following a continuous learning mechanism. It has similarities with other approaches including online learning, transfer learning, reinforcement learning and cumulative learning but has maintained its differences. Lifelong topic modeling refers to the use of lifelong learning approach with topic modeling. The objective is to develop topic models that incorporate past knowledge to generate topics that easily represent the underlying concepts discussed in the dataset. The past knowledge is in the form of a collective sense that helps to give meaningful topics even to domains that have limited or noisy sample data. This review paper puts emphasis on the effectiveness of lifelong learning approach in comparison to the traditional machine learning approaches. It discusses the strengths of lifelong learning and the progress that it has made with topic models as a specific case. In addition the challenges faced by lifelong learning topic models are discussed and the new avenues are highlighted as decisive factors towards the future of lifelong learning approach.

This is an introduction to an advanced machine learning paradigm that continuously learns by accumulating past knowledge that it then uses in future learning and problem solving. In contrast, the current dominant machine learning paradigm learns in isolation: given a training dataset, it runs a machine learning algorithm on the dataset to produce a model that is then used in its intended application. It makes no attempt to retain the learned knowledge and use it in subsequent learning. Unlike this isolated system, humans learn effectively with only a few examples precisely because our learning is very knowledge-driven: the knowledge learned in the past helps us learn new things with little data or effort. Lifelong learning aims to emulate this capability, because without it, an AI system cannot be considered truly intelligent. Research in lifelong learning has developed significantly in the relatively short time since the first edition of this book was published.

Lifelong machine learning methods acquire knowledge over a series of consecutive tasks, continually building upon their experience. Current lifelong learning algorithms rely upon a single learning agent that has centralized access to all data. In this paper, we extend the idea of lifelong learning from a single agent to a network of multiple agents that collectively learn a series of tasks. Each agent faces some (potentially unique) set of tasks; the key idea is that knowledge learned from these tasks may benefit other agents trying to learn different (but related) tasks. Our Collective Lifelong Learning Algorithm (CoLLA) provides an efficient way for a network of agents to share their learned knowledge in a distributed and decentralized manner, while eliminating the need to share locally observed data. We provide theoretical guarantees for robust performance of the algorithm and empirically demonstrate that CoLLA outperforms existing approaches for distributed multi-task learning on a variety of datasets.

Reviewing video from wireless capsule endoscopy is a tedious work that requires concentration for extended hours. Computer algorithms are sought to reduce the review time and large number of training examples, to reduce the generalization error of machine learning algorithms. Therefore efficient and scalable approaches are needed which can modify knowledge structures in an incremental fashion without having to revisit all previously processed data. This paper proposes a novel incremental learning method for SVMs. The intuition is that the samples within the skin are retained alone in training. When additional samples are provided, they will be used together with the skin of the convex hull constructed from previous dataset. This result in much less number of instances and are used in incremental steps of the training process. Experiments were performed to segment frames of capsule endoscopy videos into four sections of gastrointestinal tract.

We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots' components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters mu. The value of mu from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

Cloud detection in satellite images using multi-objective social spider optimization

Humans are natural face recognition experts, far out-performing current automated face recognition algorithms, especially in naturalistic, “in the wild” settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present two methods enabling this approach. By learning and operating within person-specific representations, we are able to significantly outperform the previous state-of-the-art on PubFig83, a challenging benchmark for familiar face recognition in the wild, using a novel method for learning representations in deep visual hierarchies. We suggest that such person-specific representations aid recognition by introducing an intermediate form of regularization to the problem.

In this paper, we address the task of scene image classification in sparse framework. Recent scene image datasets consist of thousands of different size images with size of the order of 106 pixels. Motivated by the fact that every image has a different size, we propose a dynamic kernel1 which works over set of feature maps obtained for an image from last convolutional pooling layer of a pre-trained CNN. The size of feature maps depends on the input image size leading to the requirement of a dynamic kernel to compute similarity score between feature maps of different images. The kernel matrix obtained by using a dynamic kernel is large in size owing to the large number of training examples. To handle this we propose to use the concept of reduced virtual features (RVFs) obtained by diagonalizing the kernel matrix. RVF is a fixed length representation of a scene image irrespective of its true size. Classification is done in sparse framework by applying block sparsity constraint over sparse coefficients using dictionary built from RVFs. The proposed approach tested over standard datasets like Vogel-Schiele, MIT-8, MIT-67 and SUN-397 yields good results.

Few-shot Learning (FSL) methods are being adopted in settings where data is not abundantly available. This is especially seen in medical domains where the annotations are expensive to obtain. Deep Neural Networks have been shown to be vulnerable to adversarial attacks. This is even more severe in the case of FSL due to the lack of a large number of training examples. In this paper, we provide a framework to make few-shot segmentation models adversarially robust in the medical domain where such attacks can severely impact the decisions made by clinicians who use them. We propose a novel robust few-shot segmentation framework, Prototypical Neural Ordinary Differential Equation (PNODE), that provides defense against gradient-based adversarial attacks. We show that our framework is more robust compared to traditional adversarial defense mechanisms such as adversarial training. Adversarial training involves increased training time and shows robustness to limited types of attacks depending on the type of adversarial examples seen during training. Our proposed framework generalises well to common adversarial attacks like FGSM, PGD and SMIA while having the model parameters comparable to the existing few-shot segmentation models. We show the effectiveness of our proposed approach on three publicly available multi-organ segmentation datasets in both in-domain and cross-domain settings by attacking the support and query sets without the need for ad-hoc adversarial training.KeywordsFew-shot segmentationNeural-ODEAdversarial robustness

We introduce a Convolutional Neural Network (CNN) to post-process recordings obtained by means of Background Oriented Schlieren (BOS), a popular technique to visualize compressible/convective flows. To reconstruct BOS image deformation, we devised a lightweight network (LIMA) that has comparatively fewer parameters to train, allowing the deployment of the network on embedded GPU hardware. To train the CNN, we introduce a novel strategy based on the generation of synthetic images with random, irrotational displacement field that mimic those provided by real BOS recording. This allows us to generate a large number of training examples at minimal computational cost. To assess the accuracy of the reconstructed displacement, we consider test cases consisting of synthetic images with sinusoidal displacement as well as images obtained in a real experimental study of flow past and inside a heated hollow hemisphere. By comparing the prediction of the CNN with conventional post-processing techniques such as Direct Image Correlation (DIC) and Particle Image Velocimetry (PIV) cross-correlation, we show that LIMA gives more accurate and robust results for the synthetic example. When applied to the recordings from the real experiment, all methods provide consistent deformation fields. As they offer similar or better accuracy at a fraction of the computational costs, properly designed CNNs offer a valuable alternative to conventional post-processing techniques for BOS experiments.

Lifelong Machine Learning (LML) denotes a scenario involving multiple sequential tasks, each accompanied by its respective dataset, in order to solve specific learning problems. In this context, the focus of LML techniques is on utilizing already acquired knowledge to adapt to new tasks efficiently. Essentially, LML concerns about facing new tasks while exploiting the knowledge previously gathered from earlier tasks not only to help in adapting to new tasks but also to enrich the understanding of past ones. By understanding this concept, one can better grasp one of the major obstacles in LML, known as Knowledge Transfer (KT). This systematic literature review aims to explore state-of-the-art KT techniques within LML and assess the evaluation metrics and commonly utilized datasets in this field, thereby keeping the LML research community updated with the latest developments. From an initial pool of 417 articles from four distinguished databases, 30 were deemed highly pertinent for the information extraction phase. The analysis recognizes four primary KT techniques: Replay, Regularization, Parameter Isolation, and Hybrid. This study delves into the characteristics of these techniques across both neural network (NN) and non-neural network (non-NN) frameworks, highlighting their distinct advantages that have captured researchers’ interest. It was found that the majority of the studies focused on supervised learning within an NN modelling framework, particularly employing Parameter Isolation and Hybrid for KT. The paper concludes by pinpointing research opportunities, including investigating non-NN models for Replay and exploring applications outside of computer vision (CV).

Human beings follow a continuous learning paradigm, i.e., they learn to solve smaller and relatively easy problems, retain the learnt knowledge and apply that knowledge to learn and solve more complex and large-scale problems of the domain. Currently, most machine learning and evolutionary computing systems lack this ability to reuse the previous learnt knowledge. This paper presents a lifelong machine learning model for text classification that extracts the useful knowledge from simple problems of a domain and reuses the learnt knowledge to learn complex problems of the domain. The proposed approach adopts a rule-based learning classifier system, and a rich encoding scheme is used to extract and reuse building units of knowledge. The experimental results show that the continuous learning approach outperformed the baseline classifier system.