One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets

Multi-task learning for virtual flow metering

As the need for efficient warehouse logistics has increased in manufacturing systems, the use of automated guided vehicles (AGVs) has also increased to reduce travel time. The AGVs are controlled by a system using laser sensors or floor-embedded wires to transport pallets and their loads. Because such control systems have only predefined palletizing strategies, AGVs may fail to engage incorrectly positioned pallets. In this study, we consider a vision sensor-based method to address this shortcoming by recognizing a pallet’s position. We propose a multi-task deep learning architecture that simultaneously predicts distances and rotation based on images obtained from a visionary sensor. These predictions complement each other in learning, allowing a multi-task model to learn and execute tasks impossible with single-task models. The proposed model can accurately predict the rotation and displacement of the pallets to derive information necessary for the control system. This information can be used to optimize a palletizing strategy. The superiority of the proposed model was verified by an experiment on images of stored pallets that were collected from a visionary sensor attached to an AGV.

The brown rice rate (BRR), milled rice rate (MRR), and head rice rate (HRR) are important indicators of rice milling quality. The simultaneous detection of these three metrics holds significant economic value for rice milling quality assessments. In this study, hyperspectral imaging was employed to estimate the rice milling quality attributes of two rice varieties (Xiushui121 and Zhehujing26). Partial Least Squares Regression (PLSR), Support Vector Regression (SVR), Convolutional Neural Networks (CNNs), and Backpropagation Neural Networks (BPNNs) were used to establish both single-task and multi-task models for the prediction of milling quality attributes. Most multi-task models demonstrated a higher prediction accuracy compared with their corresponding single-task models. Among single-task models, BPNNs outperformed the others in predicting BRR and HRR, with correlation coefficients (r) up to 0.9. SVR excelled in forecasting the MRR. In multi-task learning, BPNNs exhibited relatively better performance, with r values exceeding 0.81 for all three indicators. SHapley Additive exPlanations (SHAP) analysis was used to explore the relationship between wavelength and rice milling quality attributes. This study confirmed that this nondestructive detection method for rice milling quality using hyperspectral imaging combined with machine learning and deep learning algorithms could effectively assess rice milling quality, thus contributing to breeding and growth management in the industry.

Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectures based on large language models that have been pretrained on vast amounts of text. To better leverage that pretraining, recent work has explored a reformulation of semantic parsing whereby the output sequences are themselves natural language sentences, but in a controlled fragment of natural language. This approach delivers strong results, particularly for few-shot semantic parsing, which is of key importance in practice and the focus of our paper. We push this line of work forward by introducing an automated methodology that delivers very significant additional improvements by utilizing modest amounts of unannotated data, which is typically easy to obtain. Our method is based on a novel synthesis of four techniques: joint training with auxiliary unsupervised tasks; constrained decoding; self-training; and paraphrasing. We show that this method delivers new SOTA few-shot performance on the Overnight dataset, particularly in very low-resource settings, and very compelling few-shot results on a new semantic parsing dataset.

In recent years, multitask learning has turned out to be of great success in various applications. Though single task model training has promised great results throughout these years, it ignores valuable information which might help in estimating better learning parameters. By virtue of learning-related tasks, multitask learning has been able to generalize the models even better. In this paper, we try to enhance the feature mapping of multitask deep learning models by sharing features among related tasks and inductive transfer learning. We have explored the learning of task relationship among various tasks for acquiring more benefits from multitask learning. The proposed enhanced model is compared with two state-of-the-art multitask deep learning models, namely Hyperface and FaceNet. The results show better performance of the proposed model in predicting age, gender and ethnicity on the UTK face dataset.KeywordsNeural networksMultitask deep learningDeep learningRecurrent neural networksConvolution neural networks

We present a unified multi-task learning architecture for fast and accurate pedestrian detection. Different from existing methods which often focus on either a new loss function or architecture, we propose an improved multi-task convolutional neural network learning architecture to effectively and efficiently interfuse the task of pedestrian detection and semantic segmentation. To achieve this, we integrate a lightweight semantic segmentation branch to Faster R-CNN detection framework that enables end-to-end hard parameter sharing in order to boost the detection performance and maintain computational efficiency as follows. Firstly, a Semantic Segmentation to Feature Module (SS2FM) refines the convolutional features in RPN stage by integrating the features generated from the semantic segmentation branch. Secondly, a Semantic Segmentation to Confidence Module (SS2CM) refines the classification confidence in RPN stage by fusing it with the semantic segmentation confidence. We also introduce an effective anchor matching point transform to alleviate the problem of feature misalignment for heavily occluded pedestrians. The proposed unified multi-task learning architecture lends itself well to more robust pedestrian detection in diverse scenarios with negligible computation overhead. In addition, the proposed architecture can achieve high detection performance with low resolution input images, which significantly reduces the computational complexity. Experiment results on CityPersons and Caltech datasets show that our method is the fastest among all state-of-the-art pedestrian detection methods while exhibiting competitive detection performance.

Alfalfa is a valuable and widely adapted forage crop, and its nutritive value directly affects animal performance and ultimately affects the profitability of livestock production. Traditional nutritive value measurement method is labor-intensive and time-consuming and thus hinders the determination of alfalfa nutritive values over large fields. The adoption of unmanned aerial vehicles (UAVs) facilitates the generation of images with high spatial and temporal resolutions for field-level agricultural research. Additionally, compared with other imaging modalities, hyperspectral data usually consist of hundreds of narrow spectral bands and allow the accurate detection, identification, and quantification of crop quality. Although various machine-learning methods have been developed for alfalfa quality prediction, they were all single-task models that learned independently for each quality trait and failed to utilize the underlying relatedness between each task. Inspired by the idea of multitask learning (MTL), this study aims to develop an approach that simultaneously predicts multiple quality traits. The algorithm first extracts shared information through a long short-term memory (LSTM)-based common hidden layer. To enhance the model flexibility, it is then divided into multiple branches, each containing the same or different number of task-specific fully connected hidden layers. Through comparison with multiple mainstream single-task machine-learning models, the effectiveness of the model is illustrated based on the measured alfalfa quality data and multitemporal UAV-based hyperspectral imagery.

Healthcare professionals and individual users use wearable devices equipped with various sensors for healthcare management. Recently, the joint usage of artificial intelligence and these wearable sensors has played an essential role in healthcare management by providing a wide range of applications such as fitness tracking, gym activity monitoring, patient rehabilitation monitoring, and disease detection. These tasks eventually aim to enhance personal well-being and better manage the user’s physical health by monitoring different activity types and body weight changes. Here, we present an efficient multi-task learning framework based on commercial smart insoles that can solve three tasks related to physical health management: activity classification, speed estimation, and body weight estimation. Our multi-task framework converts the sensor data from the smart insole to a recurrence plot, which shows significant performance improvement compared to processing the raw time series data. In addition, we utilized a modified MobileNetV2 as our backbone network, which has a total parameter of less than 100K and a computational budget of 0.34G of multiply-accumulate operations. Furthermore, we collected a vast dataset from 72 users carrying out 16 experiments, which contains the largest number of people for multi-task learning purposes using smart insoles. Extensive experiments show that the proposed multi-task learning framework is extremely efficient while outperforming or leading to comparable performance against single-task models.

Biomedical knowledge graph verification with multitask learning architectures.

Historical text normalization often relies on small training datasets. Recent work has shown that multi-task learning can lead to significant improvements by exploiting synergies with related datasets, but there has been no systematic study of different multi-task learning architectures. This paper evaluates 63 multi-task learning configurations for sequence-to-sequence-based historical text normalization across ten datasets from eight languages, using autoencoding, grapheme-to-phoneme mapping, and lemmatization as auxiliary tasks. We observe consistent, significant improvements across languages when training data for the target task is limited, but minimal or no improvements when training data is abundant. We also show that zero-shot learning outperforms the simple, but relatively strong, identity baseline.

A unified modeling framework for lane change intention recognition and vehicle status prediction

Question answering over tables is a very popular semantic parsing task in natural language processing (NLP). However, few existing methods focus on table images, even though there are usually large-scale unstructured tables in practice (e.g., table images). Table parsing from images is nontrivial since it is closely related to not only NLP but also computer vision (CV) to parse the tabular structure from an image. In this demo, we present a question answering system for unstructured table images. The proposed system mainly consists of 1) a table recognizer to recognize the tabular structure from an image and 2) a table parser to generate the answer to a natural language question over the table. In addition, to train the model, we further provide table images and structure annotations for two widely used semantic parsing datasets. Specifically, the test set is used for this demo, from where the users can either choose from default questions or enter a new custom question.

In order to improve the accuracy of the multiple load forecasting of a regional integrated energy system, a short-term multiple load forecasting model based on the quantum weighted GRU and multi-task learning framework is proposed in this paper. Firstly, correlation analysis is carried out using a maximum information coefficient to select the input of the model. Then, a multi-task learning architecture is constructed based on the quantum weighted GRU neural network, and the coupling information among multiple loads is learned through the sharing layer in order to improve the prediction accuracy of multiple loads. Finally, the PSO algorithm is used to optimize the parameters of the quantum weighted GRU. The simulation data of a regional integrated energy system in northern China are used to predict the power and cooling loads on summer weekdays and rest days, and the results show that, compared with the LSTM, GRU and single task learning QWGRU models, the proposed model is more effective in the multiple load forecasting of a regional integrated energy system.

This is an accepted manuscript of an article published by Association for Computational Linguistics in Proceedings of the Joint Workshop on Multiword Expressions and WordNet (MWE-WN 2019), available online: https://www.aclweb.org/anthology/W19-5119
\n
\nThe accepted version of the publication may differ from the final published version.

Advancing screening of aquatic persistent, mobile, and toxic substances via integration of graph convolutional networks and multitask learning.