Life-long Learning Through Task Rehearsal and Selective Knowledge Transfer

Lifelong Machine Learning (or Lifelong Learning) is an advanced machine learning paradigm that learns continuously, accumulates the knowledge learned in previous tasks, and uses it to help future learning. In the process, the learner becomes more and more knowledgeable and effective at learning. This learning ability is one of the hallmarks of human intelligence. However, the current dominant machine learning paradigm learns in isolation: given a training dataset, it runs a machine learning algorithm on the dataset to produce a model. It makes no attempt to retain the learned knowledge and use it in future learning. Although this isolated learning paradigm has been very successful, it requires a large number of training examples, and is only suitable for well-defined and narrow tasks. In comparison, we humans can learn effectively with a few examples because we have accumulated so much knowledge in the past which enables us to learn with little data or effort. Lifelong learning aims to achieve this capability. As statistical machine learning matures, it is time to make a major effort to break the isolated learning tradition and to study lifelong learning to bring machine learning to new heights. Applications such as intelligent assistants, chatbots, and physical robots that interact with humans and systems in real-life environments are also calling for such lifelong learning capabilities. Without the ability to accumulate the learned knowledge and use it to learn more knowledge incrementally, a system will probably never be truly intelligent. This book serves as an introductory text and survey to lifelong learning.

Fine-grained visual categorization aims at classifying visual data at a subordinate level, e.g., identifying different species of birds. It is a highly challenging topic receiving significant research attention recently. Most existing works focused on the design of more discriminative feature representations to capture the subtle visual differences among categories. Very limited efforts were spent on the design of robust model learning algorithms. In this paper, we treat the training of each category classifier as a single learning task, and formulate a generic multiple task learning (MTL) framework to train multiple classifiers simultaneously. Different from the existing MTL methods, the proposed generic MTL algorithm enforces no structure assumptions and thus is more flexible in handling complex inter-class relationships. In particular, it is able to automatically discover both clusters of similar categories and outliers. We show that the objective of our generic MTL formulation can be solved using an iterative reweighted ℓ2 method. Through an extensive experimental validation, we demonstrate that our method outperforms several state-of-the-art approaches.KeywordsFine-grained visual categorizationinter-class relationshipmultiple task learning

Lifelong machine learning methods acquire knowledge over a series of consecutive tasks, continually building upon their experience. Current lifelong learning algorithms rely upon a single learning agent that has centralized access to all data. In this paper, we extend the idea of lifelong learning from a single agent to a network of multiple agents that collectively learn a series of tasks. Each agent faces some (potentially unique) set of tasks; the key idea is that knowledge learned from these tasks may benefit other agents trying to learn different (but related) tasks. Our Collective Lifelong Learning Algorithm (CoLLA) provides an efficient way for a network of agents to share their learned knowledge in a distributed and decentralized manner, while eliminating the need to share locally observed data. We provide theoretical guarantees for robust performance of the algorithm and empirically demonstrate that CoLLA outperforms existing approaches for distributed multi-task learning on a variety of datasets.

Subspace learning (SL) plays a key role in various learning tasks, especially those with a huge feature space. When processing multiple high-dimensional learning tasks simultaneously, it is of great importance to make use of the subspace extracted from some tasks to help learn others, so that the learning performance of all tasks can be enhanced together. To achieve this goal, it is crucial to answer the following question: How can the commonality among different learning tasks and, of equal importance, the individuality of each single learning task, be characterized and extracted from the given datasets, so as to benefit the subsequent learning, for example, classification? Existing multitask SL methods usually focused on the commonality among the given tasks, while neglecting the individuality of the learning tasks. In order to offer a more general and comprehensive framework for multitask SL, in this article, we propose a novel method dubbed commonality and individuality-based SL (CISL). First, we formally define the notions and objective functions of both commonality and individuality with respect to multiple SL tasks. Then, we design an iterative algorithm to solve the formulated objective functions, with the convergence of the algorithm being guaranteed. To show the generality of the proposed method, we theoretically analyze its connections to existing single-task and multitask SL methods. Finally, we demonstrate the necessity and effectiveness of incorporating both commonality and individuality by interpreting the learned subspaces and comparing the performance of CISL (in terms of the subsequent classification accuracy) with that of classical and state-of-the-art SL approaches on both synthetic and real-world multitask datasets. The empirical evaluation validates the effectiveness of the proposed method in characterizing the commonality and individuality for multitask SL.

For traditional human face based biometrics, usually one task (face recognition) is learned at one time. This single task learning (STL) approach may neglect potential rich information resources hidden in other related tasks, while multitask learning (MTL) can make full use of the latent information. MTL is an inductive transfer method which improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias. In this paper, backpropagation (BP) network based MTL approach is proposed for face recognition. The feasibility of this approach is demonstrated through two different face recognition experiments, which show that MTL based on BP neural networks is more effective than the traditional STL approach, and that MTL is also a practical approach for face recognition.

Open Data: EEG and behavioural data from a multiple wordlist learning task for demonstrating the backward and forward effects of testing (testing effects). Data is from a multiple-list learning task described in Pastotter et al. (2011) and Szpunar et al. (2008) to demonstrate both the backward and forward effects of testing. However, the number of lists in the current study is three rather than five as in the original studies. The implementation of the multiple wordlist learning task in PsychoPy from which the data was generated can be found on Zenodo: Beardsley, Marc. (2017, December 20). PsychoPy implementation of a multiple-list learning task (Version v1.0). Zenodo. http://doi.org/10.5281/zenodo.1120260

Automatic segmentation of three clinical target volumes in radiotherapy using lifelong learning

<h3>Abstract</h3> Humans and other animals demonstrate a remarkable ability to generalize knowledge across distinct contexts and objects during natural behavior. We posit that this ability to generalize arises from a specific representational geometry, that we call abstract and that is referred to as disentangled in machine learning. These abstract representations have been observed in recent neurophysiological studies. However, it is unknown how they emerge. Here, using feedforward neural networks, we demonstrate that the learning of multiple tasks causes abstract representations to emerge, using both supervised and reinforcement learning. We show that these abstract representations enable few-sample learning and reliable generalization on novel tasks. We conclude that abstract representations of sensory and cognitive variables may emerge from the multiple behaviors that animals exhibit in the natural world, and, as a consequence, could be pervasive in high-level brain regions. We also make several specific predictions about which variables will be represented abstractly.

This special issue of seminar.net is the result of a collaboration between teachers at the two master programmes, “Communication, Design and Learning” at the University of Oslo and “ICT Supported Learning” at the Oslo and Akershus University College of Applied Sciences. For 2013-2015, we received a grant from Norgesuniversitetet to develop partnership, flexible teaching and study methods, and to increase learning outcomes among students. The four papers represent the first research results from our project. We are grateful to Norgesuniversitetet for its support.Per HetlandGeneral editorThe first paper, by Per Hetland and Anders I. Mørch, both from the University of Oslo has a title called “Ethnography for Investigating the Internet”. They describe the field by giving an overview of the competing concepts, and argue for their chosen path. They point at important crossing roads and selections that needs to be taken for future research.Jan Erik Dahl, also of The University of Oslo, presents the paper “Supporting learning through epistemic scaffolds embedded in a highlighter tool”. It explores how the use of the tool was used to support students’ readings and discussions of research articles. He argues that the use of annotation technologies in education is increasing, and that annotations can play a wide variety of epistemic roles; e.g., they can facilitate a deeper level of engagement, support critical thinking, develop cognitive and metacognitive skills and introduce practices that can support knowledge building and independent learning. Dahl criticises some of the underlying assumption of present research in the field and suggest that one needs to look for the active co-construction that students do in collaboration with others.Monica Johannesen and Leikny Øgrim, of the Oslo and Akershus University College of Applied Sciences, has collaborated with Ole Smørdal of the University of Oslo on the paper “Facebook as an actor - a case of students negotiating their social presence in an online course”. Their paperreports on a study of a higher education online course based on asynchronous communication. In the selection of technology for online discussions they aimed at creating a sense of togetherness among the teachers and the students. Their choice proved to be a insightful experience of how the differences of agency of a virtual learning environment (VLE) are compared to social media when it comes to social presence.Anders I. Mørch of the University of Oslo has collaborated with Louise Mifsud and Bård Kjetil Engen, of Oslo and Akershus University College of Applied Sciences on the fourth paper, called “Problem-Based Learning in Synchronous Networked Environments: Comparing Adobe Connect and Second Life”. Their approach was to compare experiences from using two networked learning environments: Adobe Connect (AC) and Second Life (SL) for supporting teaching and learning in distance education courses. Data were collected from two separate case studies: one in Norway (AC) and the other in the United States (SL), using different but comparable methods of data analysis. They focus on problem-based learning (PBL), using four application characteristics of PBL (learner activity, collaborative learning, feedback, and valuation of previous knowledge).General sectionIn addition to the four papers in the special issue of Seminar.net, we also contribute with two more papers.David Hallberg, representing both The Swedish Red Cross University College and the Stockholm University, has written a paper with Henrik Hansson and Anders G.Nilsson, also from Stockholm University called: “Immigrant Women's Reasoning and Use of Information and Communications Technology in Lifelong Learning”. The paper explores the reasoning and use of information and communications technology (ICT) in lifelong learning by immigrant women. The researchers collected data from semi-structured and unstructured interviews. The study was carried out primarily in a school environment, which also made it possible to draw conclusions about the connection between learning in and outside school environments.Trine Fossland, of University of Tromsø UiT The Arctic University of Norway, presents the last paper in this issue. Her paper, “Stories of technology-enhancement in higher education – a critical approach”, continues a critical stance of educational technology, suggesting that even if there is a large body of research on technology-enhanced learning, questions related to the educational effectiveness of technology use still needs to be questioned. She argues that “digital innovators’” stories about technology enhancement may constitute a rich source for understanding this complex educational phenomenon both in relation to teachers’ daily practices and the implementation of ICT in higher education at large. Her paper relies on biographical interviews with “digital innovators”. Her findings suggest that technology-enhancement is linked to nine key characteristics: different educational models, authenticity, pedagogical added values, meaningful student activities, changing approaches to feedback, assessment and connection with the outside world, as well as holistic planning, supportive leaders and strong micro-cultures. This paper proposes a more nuanced understanding of the term technology enhanced learning and suggests strategies for educational development and further investigations related to this phenomenon in higher education. Yngve Nordkvelle

Recently, many research groups have dedicated their efforts to the development of Weblog (or Blog) tools to facilitate cooperative learning. However, these tools were mainly for young students in regular courses at schools. For adult learners, especially elderly people, lifelong learning is also important in their everyday life. Therefore, it is valuable to study the possible development of these tools for such a user group in the future. The objective of this research is to study the effects of knowledge sources and learning styles of the elders on their motivations to use Weblog tools for lifelong learning. Field studies and questionnaire surveys were conducted to elicit their requirements. The results showed that both knowledge sources and learning styles had significant effects on the willingness to use Blogs as the platform to exchange their knowledge and experience.

CHAPTER 14 - Representation Generation in an Exploratory Learning System

Multiagent Reinforcement Learning (MARL) has been well adopted due to its exceptional ability to solve multiagent decision-making problems. To further enhance learning efficiency, knowledge transfer algorithms have been developed, among which experience-sharing-based and action-advising-based transfer strategies share the mainstream. However, it is notable that, although there exist many successful applications of both strategies, they are not flawless. For the long-developed action-advising-based methods (namely KT-AA, short for knowledge transfer based on action advising), their data efficiency and scalability are not satisfactory. As for the newly proposed experience-sharing-based knowledge transfer methods (KT-ES), although the shortcomings of KT-AA have been partially overcome, they are incompetent to correct specific bad decisions in the later learning stage. To leverage the superiority of both KT-AA and KT-ES, this study proposes KT-Hybrid, a hybrid knowledge transfer approach. In the early learning phase, KT-ES methods are employed, expecting better data efficiency from KT-ES to enhance the policy to a basic level as soon as possible. Later, we focus on correcting specific errors made by the basic policy, trying to use KT-AA methods to further improve the performance. Simulations demonstrate that the proposed KT-Hybrid outperforms well-received action-advising- and experience-sharing-based methods.

Transforming the Thai society into a knowledge-based one requires enabling all Thais to have equal access to education and the development of human resources through continuous lifelong learning, as outlined by the objective of Thailand's National Education Standards. Thus, this paper aimed to study and identify the relationship between general knowledge and depth knowledge for lifelong learning in higher education. The data collection was conducted from 411 randomly selected students from Rajabhat Mahasarakham University, Thailand. As the results showed, it is clear that there is an opportunity for general education to affect a particular course by about half the total. However, analysis of the data may have employed only some of the data collected.

Tibetan Few‐Shot Learning Model With Deep Contextualised Two‐Level Word Embeddings

We develop a novel framework that aims at automatically adapting previously learned information extraction knowledge from a source Web site to a new unseen target site in the same domain. Two kinds of features related to the text fragments from the Web documents are investigated. The first type of feature is called, a site-invariant feature. These features likely remain unchanged in Web pages from different sites in the same domain. The second type of feature is called a site-dependent feature. These features are different in the Web pages collected from different Web sites, while they are similar in the Web pages originating from the same site. In our framework, we derive the site-invariant features from previously learned extraction knowledge and the items previously collected or extracted from the source Web site. The derived site-invariant features will be exploited to automatically seek a new set of training examples in the new unseen target site. Both the site-dependent features and the site-invariant features of these automatically discovered training examples will be considered in the learning of new information extraction knowledge for the target site. We conducted extensive experiments on a set of real-world Web sites collected from three different domains to demonstrate the performance of our framework. For example, by just providing training examples from one online book catalog Web site, our approach can automatically extract information from ten different book catalog sites achieving an average precision and recall of 71.9% and 84.0% respectively without any further manual intervention.