Self-Distillation with the New Paradigm in Multi-Task Learning

Maize is an essential cereal for humans and animals worldwide, and it is one of the staple food in Kenya. One of the main challenges facing the maize crop in Kenya is the presence of diseases spreading quickly. Early recognition of maize pathogen and disease help at preventing the disease from spreading throughout the field. This paper proposes a regularized Multitask learning (MTL)–Convolutional Neural Networks (CNN) model for simultaneously identifying maize disease and its pathogen from diseased maize images. MTL allows training one model for multiple tasks at a time, which may improve the accuracy of each task by taking advantage of their commonalities. Our baseline is made of two CNN classification models, one of them being overfitting. We then build an MTL based on the two models, which increases the test accuracy of the overfitting model from 60.08% to 74.48%. The results show that the accuracy rises to 77.44% while combining MTL to the Early stopping method. However, the test accuracy goes up to 85.22 percent when MTL is combined with Early Stopping and Transfer Learning. The model is deployed to an android mobile application for maize farmers as end-users which is very important for costs reduction and time saving.

In this paper, we study the multi-task learning problem with a new perspective of considering the link structure of data and task relationship modeling simultaneously. In particular, we first introduce the Matrix Generalized Inverse Gaussian (MGIG) distribution and define a Matrix Gaussian Matrix Generalized Inverse Gaussian (MG-MGIG) prior. Based on this prior, we propose a novel multi-task learning algorithm, the Bayesian Multi-task Relationship Learning (BMTRL) algorithm. To incorporate the link structure into the framework of BMTRL, we propose link constraints between samples. Through combining the BMTRL algorithm with the link constraints, we propose the Bayesian Multi-task Relationship Learning with Link Constraints (BMTRL-LC) algorithm. To make the computation tractable, we simultaneously use a convex optimization method and sampling techniques. In particular, we adopt two stochastic EM algorithms for BMTRL and BMTRL-LC, respectively. The experimental results on Cora dataset demonstrate the promise of the proposed algorithms.

Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level. We first utilize a pre-trained detector to detect objects. Then, we train a 3D convolutional neural network to produce discriminative anomaly-specific information by jointly learning multiple proxy tasks: three self-supervised and one based on knowledge distillation. The self-supervised tasks are: (i) discrimination of forward/backward moving objects (arrow of time), (ii) discrimination of objects in consecutive/intermittent frames (motion irregularity) and (iii) reconstruction of object-specific appearance information. The knowledge distillation task takes into account both classification and detection information, generating large prediction discrepancies between teacher and student models when anomalies occur. To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture. Our lightweight architecture outperforms the state-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSD Ped2. Additionally, we perform an ablation study demonstrating the importance of integrating self-supervised learning and normality-specific distillation in a multi-task learning setting.

<p><strong>This thesis explores innovative methods for enhancing the manipulation and geometric understanding of 360° images in virtual reality (VR) applications, with a focus on depth estimation, surface normal prediction, and multi-task learning. The research presents a comprehensive approach to improving the manipulation of stereo 360° images, particularly in the context of image composition. A novel method is proposed to seamlessly integrate new visual elements into stereo 360° scenes, thereby enhancing user interaction and immersion in VR environments.</strong></p><p>To support such advanced editing applications, a new technique for estimating depth maps from monocular 360° images is introduced. This method leverages both local and global scene information, significantly improving the accuracy of depth predictions. This is particularly crucial for applications that require spatial awareness, such as virtual and augmented reality, where accurate depth perception is key to realistic interaction with the virtual environment.</p><p>Furthermore, the thesis introduces a hybrid approach that combines convolutional neural networks (CNNs) and Vision Transformers (ViTs) to improve surface normal estimation. This method takes advantage of CNNs' ability to capture fine-grained details and ViTs' strength in modeling global context, resulting in more precise surface geometry analysis from 360° imagery. This enhanced surface normal estimation plays a vital role in better understanding the spatial structure of the scene.</p><p>In addition, the research demonstrates the effectiveness of multi-task learning (MTL) for comprehensive scene geometry understanding. By predicting both depth and surface normals simultaneously from monocular 360° images, the proposed MTL framework delivers a more detailed and coherent geometric representation, further contributing to the realism and immersion of VR used scenarios.</p><p>Through these contributions, the thesis significantly advances the field of 360° image-based manipulations and scene understanding, offering new tools and methodologies that pave the way for more immersive, interactive, and spatially aware experiences for VR applications.</p>

Chapter 2 - Deep Learning for Multimodal Data Fusion

A new hierarchical nonparametric Bayesian framework is proposed for the problem of multi-task learning (MTL) with sequential data. The models for multiple tasks, each characterized by sequential data, are learned jointly, and the intertask relationships are obtained simultaneously. This MTL setting is used to analyze and sort large databases composed of sequential data, such as music clips. Within each data set, we represent the sequential data with an infinite hidden Markov model (iHMM), avoiding the problem of model selection (selecting a number of states). Across the data sets, the multiple iHMMs are learned jointly in a MTL setting, employing a nested Dirichlet process (nDP). The nDP-iHMM MTL method allows simultaneous task-level and data-level clustering, with which the individual iHMMs are enhanced and the between-task similarities are learned. Therefore, in addition to improved learning of each of the models via appropriate data sharing, the learned sharing mechanisms are used to infer interdata relationships of interest for data search. Specifically, the MTL-learned task-level sharing mechanisms are used to define the affinity matrix in a graph-diffusion sorting framework. To speed up the MCMC inference for large databases, the nDP-iHMM is truncated to yield a nested Dirichlet-distribution based HMM representation, which accommodates fast variational Bayesian (VB) analysis for large-scale inference, and the effectiveness of the framework is demonstrated using a database composed of 2500 digital music pieces.

Multi-task learning (MTL) is a machine learning paradigm that improves the performance of each task by exploiting useful information contained in multiple related tasks. However, the relatedness of tasks can be exploited by attackers to launch data poisoning attacks, which has been demonstrated a big threat to single-task learning. In this paper, we provide the first study on the vulnerability of MTL. Specifically, we focus on multi-task relationship learning (MTRL) models, a popular subclass of MTL models where task relationships are quantized and are learned directly from training data. We formulate the problem of computing optimal poisoning attacks on MTRL as a bilevel program that is adaptive to arbitrary choice of target tasks and attacking tasks. We propose an efficient algorithm called PATOM for computing optimal attack strategies. PATOM leverages the optimality conditions of the subproblem of MTRL to compute the implicit gradients of the upper level objective function. Experimental results on real-world datasets show that MTRL models are very sensitive to poisoning attacks and the attacker can significantly degrade the performance of target tasks, by either directly poisoning the target tasks or indirectly poisoning the related tasks exploiting the task relatedness. We also found that the tasks being attacked are always strongly correlated, which provides a clue for defending against such attacks.

In this paper we address the problem that most face recognition approaches neglect that faces share strong visual similarities, which can be exploited when learning discriminative models. Hence, we propose to model face recognition as multi-task learning problem. This enables us to exploit both, shared common information and also individual characteristics of faces. In particular, we build on Mahalanobis metric learning, which has recently shown good performance for many computer vision problems. Our main contribution is twofold. First, we extend a recent efficient metric learning algorithm to multi-task learning. The resulting algorithm supports label-incompatible learning which allows us to tap the rather large pool of anonymously labeled face pairs also for face identification. Second, we show how to learn and combine person specific metrics for face identification improving the classification power. We demonstrate the method for different face recognition tasks where we are able to match or slightly outperform state-of-the-art multi-task learning approaches.KeywordsFace RecognitionFacial IdentityIEEE InternFace PairMultitask LearningThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Exploiting additional information to improve traditional inductive learning is an active research in machine learning. When data are naturally separated into groups, SVM+[7] can effectively utilize this structure information to improve generalization. Alternatively, we can view learning based on data from each group as an individual task, but all these tasks are somehow related; so the same problem can also be formulated as a multi-task learning problem. Following the SVM+ approach, we propose a new multi-task learning algorithm called svm+MTL, which can be thought as an adaptation of SVM+ for solving MTL problem. The connections between SVM+ and svm+MTL are discussed and their performance is compared using synthetic data sets.

Acute hospital care as performed in the intensive care unit (ICU) is characterized by its frequent, but short-term interventions for patients who are severely ill. Because clinicians have to attend to more than one patient at a time and make decisions in a limited time in acute hospital care environments, the accurate prediction of the in-hospital mortality risk could assist them to pay more attention to patients with a higher in-hospital mortality risk, thereby improving the quality and efficiency of the care. One of the salient features of ICU is the diversity of patients: clinicians are faced by patients with a wide variety of diseases. However, mortality prediction for ICU patients has typically been conducted by building one common predictive model for all the diseases. In this paper, we incorporate disease-specific contexts into mortality modeling by formulating the mortality prediction problem as a multi-task learning problem in which a task corresponds to a disease. Our method effectively integrates medical domain knowledge relating to the similarity among diseases and the similarity among Electronic Health Records (EHRs) into a data-driven approach by incorporating graph Laplacians into the regularization term to encode these similarities. The experimental results on a real dataset from a hospital corroborate the effectiveness of the proposed method. The AUCs of several baselines were improved, including logistic regression without multi-task learning and several multi-task learning methods that do not incorporate the domain knowledge. In addition, we illustrate some interesting results pertaining to disease-specific predictive features, some of which are not only consistent with existing medical domain knowledge, but also contain suggestive hypotheses that could be validated by further investigations in the medical domain.

In many Multi-Agent Systems, under-education agents investigate their environments to discover their target(s). Any agent can also learn its strategy. In multi-task learning, one agent studies a set of related problems together simultaneously, by a common model. In reinforcement learning exploration phase, it is necessary to introduce a process of trial and error to learn better rewards obtained from environment. To reach this end, anyone can typically employ the uniform pseudorandom number generator in exploration period. On the other hand, it is predictable that chaotic sources also offer a random-like series comparable to stochastic ones. It is useful in multi-task reinforcement learning, to use teammate agents' experience by doing simple interactions between each other. We employ the past experiences of agents to enhance performance of multi-task learning in a nondeterministic environment. Communications are created by operators of evolutionary algorithm. In this paper we have also employed the chaotic generator in the exploration phase of reinforcement learning in a nondeterministic maze problem. We obtained interesting results in the maze problem.

In many Multi Agent Systems, under-education agents investigate their environments to discover their targets. Any agent can also learn its strategy. In multitask learning, one agent studies a set of related problems together simultaneously, by a common model. In reinforcement learning exploration phase, it is necessary to introduce a process of trial and error to learn better rewards obtained from environment. To reach this end, anyone can typically employ the uniform pseudorandom number generator in exploration period. On the other hand, it is predictable that chaotic sources also offer a random-like series comparable to stochastic ones. It is useful in multitask reinforcement learning, to use teammate agents' experience by doing simple interactions between each other. We employ the past experiences of agents to enhance performance of multitask learning in a nondeterministic environment. Communications are created by operators of evolutionary algorithm. In this paper we have also employed the chaotic generator in the exploration phase of reinforcement learning in a nondeterministic maze problem. We obtained interesting results in the maze problem.

Depression is a common mood disorder that causes severe medical problems and interferes negatively with daily life. Identifying human behavior patterns that are predictive or indicative of depressive disorder is important. Clinical diagnosis of depression relies on costly clinician assessment using survey instruments which may not objectively reflect the fluctuation of daily behavior. Self-administered surveys, such as the Quick Inventory of Depressive Symptomatology (QIDS) commonly used to monitor depression, may show disparities from clinical decision. Smartphones provide easy access to many behavioral parameters, and Fitbit wrist bands are becoming another important tool to assess variables such as heart rates and sleep efficiency that are complementary to smartphone sensors. However, data used to identify depression indicators have been limited to a single platform either iPhone, or Android, or Fitbit alone due to the variation in their methods of data collection. The present work represents a large-scale effort to collect and integrate data from mobile phones, wearable devices, and self reports in depression analysis by designing a new machine learning approach. This approach constructs sparse mappings from sensing variables collected by various tools to two separate targets: self-reported QIDS scores and clinical assessment of depression severity. We propose a so-called heterogeneous multi-task feature learning method that jointly builds inference models for related tasks but of different types including classification and regression tasks. The proposed method was evaluated using data collected from 103 college students and could predict the QIDS score with an R2 reaching 0.44 and depression severity with an F1-score as high as 0.77. By imposing appropriate regularizers, our approach identified strong depression indicators such as time staying at home and total time asleep.

Previous studies on network mining have focused primarily on learning a single task (such as classification or community detection) on a given network. This paper considers the problem of multi-task learning on heterogeneous network data. Specifically, we present a novel framework that enables one to perform classification on one network and community detection in another related network. Multi-task learning is accomplished by introducing a joint objective function that must be optimized to ensure the classes in one network are consistent with the link structure, nodal attributes, as well as the communities detected in another network. We provide both theoretical and empirical analysis of the framework. We also show that the framework can be extended to incorporate prior information about the correspondences between the clusters and classes in different networks. Experiments performed on both real-world and synthetic data sets demonstrate the effectiveness of the joint framework compared to applying classification and community detection algorithms on each network separately.

Predicting the emotional value of lexical items is a well-known problem in sentiment analysis. While research has focused on polarity for quite a long time, meanwhile this early focus has been shifted to more expressive emotion representation models (such as Basic Emotions or Valence-Arousal-Dominance). This change resulted in a proliferation of heterogeneous formats and, in parallel, often small-sized, non-interoperable resources (lexicons and corpus annotations). In particular, the limitations in size hampered the application of deep learning methods in this area because they typically require large amounts of input data. We here present a solution to get around this language data bottleneck by rephrasing word emotion induction as a multi-task learning problem. In this approach, the prediction of each independent emotion dimension is considered as an individual task and hidden layers are shared between these dimensions. We investigate whether multi-task learning is more advantageous than single-task learning for emotion prediction by comparing our model against a wide range of alternative emotion and polarity induction methods featuring 9 typologically diverse languages and a total of 15 conditions. Our model turns out to outperform each one of them. Against all odds, the proposed deep learning approach yields the largest gain on the smallest data sets, merely composed of one thousand samples.