CPWS: Confident Programmatic Weak Supervision for High-Quality Data Labeling

Obtaining annotations for large training sets is expensive, especially in settings where domain knowledge is required, such as behavior analysis. Weak supervision has been studied to reduce annotation costs by using weak labels from task-specific labeling functions (LFs) to augment ground truth labels. However, domain experts still need to hand-craft different LFs for different tasks, limiting scalability. To reduce expert effort, we present AutoSWAP: a framework for automatically synthesizing data-efficient task-level LFs. The key to our approach is to efficiently represent expert knowledge in a reusable domain-specific language and more general domain-level LFs, with which we use state-of-the-art program synthesis techniques and a small labeled dataset to generate task-level LFs. Additionally, we propose a novel structural diversity cost that allows for efficient synthesis of diverse sets of LFs, further improving AutoSWAP's performance. We evaluate AutoSWAP in three behavior analysis domains and demonstrate that AutoSWAP outperforms existing approaches using only afraction of the data. Our results suggest that AutoSWAP is an effective way to automatically generate LFs that can significantly reduce expert effort for behavior analysis.

Scene text detection is an important and challenging task in computer vision. For detecting arbitrarily-shaped texts, most existing methods require heavy data labeling efforts to produce polygon-level text region labels for supervised training. In order to reduce the cost in data labeling, we study mixed-supervised arbitrarily-shaped text detection by combining various weak supervision forms (e.g., image-level tags, coarse, loose and tight bounding boxes), which are far easier to annotate. Whereas the existing weakly-supervised learning methods (such as multiple instance learning) do not promote full object coverage, to approximate the performance of fully-supervised detection, we propose an Expectation-Maximization (EM) based mixed-supervised learning framework to train scene text detector using only a small amount of polygon-level annotated data combined with a large amount of weakly annotated data. The polygon-level labels are treated as latent variables and recovered from the weak labels by the EM algorithm. A new contour-based scene text detector is also proposed to facilitate the use of weak labels in our mixed-supervised learning framework. Extensive experiments on six scene text benchmarks show that (1) using only 10% strongly annotated data and 90% weakly annotated data, our method yields comparable performance to that of fully supervised methods, (2) with 100% strongly annotated data, our method achieves state-of-the-art performance on five scene text benchmarks (CTW1500, Total-Text, ICDAR-ArT, MSRA-TD500, and C-SVT), and competitive results on the ICDAR2015 Dataset. We will make our weakly annotated datasets publicly available.

These days, there are lots of smart factories with automatic systems that improve the factory’s manufacturing efficiency. One of the systems is product identification number recognition. In this study, we handled Bloom Identification Number (BIN) which is common in steel industries. For our BIN recognition algorithm, we adopted deep learning because it outperforms conventional algorithms in many computer vision tasks. Furthermore, applying a trained deep learning model to another factory is a big issue because data from different factories can look alike to us, but the trained models might confuse them because of the difference in background, light condition, and camera position. For this reason, new label annotations are required to train the model once again. However, label annotations will always be a big burden whenever applying a trained model to different factories. In this paper, we introduce a new method of BIN recognition that does not require data labeling of new data when training. This gives us the advantage of eliminating the time of labeling new collected data when applying the deep learning network to other factories.

Weak supervision has shown promising results in many natural language processing tasks, such as Named Entity Recognition (NER). Existing work mainly focuses on learning deep NER models only with weak supervision, i.e., without any human annotation, and shows that by merely using weakly labeled data, one can achieve good performance, though still underperforms fully supervised NER with manually/strongly labeled data. In this paper, we consider a more practical scenario, where we have both a small amount of strongly labeled data and a large amount of weakly labeled data. Unfortunately, we observe that weakly labeled data does not necessarily improve, or even deteriorate the model performance (due to the extensive noise in the weak labels) when we train deep NER models over a simple or weighted combination of the strongly labeled and weakly labeled data. To address this issue, we propose a new multi-stage computational framework -- NEEDLE with three essential ingredients: (1) weak label completion, (2) noise-aware loss function, and (3) final fine-tuning over the strongly labeled data. Through experiments on E-commerce query NER and Biomedical NER, we demonstrate that NEEDLE can effectively suppress the noise of the weak labels and outperforms existing methods. In particular, we achieve new SOTA F1-scores on 3 Biomedical NER datasets: BC5CDR-chem 93.74, BC5CDR-disease 90.69, NCBI-disease 92.28.

Near real-time fine-resolution land surface phenology (LSP) prediction is essential for understanding surface attributes and ecosystem functions, and solving important ecological processes related to phenology at the landscape scale. In this paper, we applied the Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model (ESTARFM) to fuse image pairs of Landsat 8 and Moderate-resolution Imaging Spectroradiometer (MODIS) as train data, and then applied the first derivative method to retrieve phenophase transition dates from fused time series of satellite data as label data. The convolutional neural network (CNN) model was trained using fusion images as inputs and the label data as targets. The trained model was further used to predict LSP dates from individual Landsat images. As evaluated using the reference data, the predict land surface phenological dates and could match the reference well with the coefficient of determination of 0.77 and root mean squared errors of 3.535, and our study provides an alternative method to predict land surface phenological dates using individual Landsat images.

It is common practice for machine learning systems to rely on crowdsourced label data for training and evaluation. It is also well-known that biases present in the label data can induce biases in the trained models. Biases may be introduced by the mechanisms used for deciding what data should/could be labelled or by the mechanisms employed to obtain the labels. Various approaches have been proposed to detect and correct biases once the label dataset has been constructed. However, proactively reducing biases during the data labelling phase and ensuring data fairness could be more economical compared to post-processing bias mitigation approaches. In this workshop, we aim to foster discussion on ongoing research around biases in crowdsourced data and to identify future research directions to detect, quantify and mitigate biases before, during and after the labelling process such that both task requesters and crowd workers can benefit. We will explore how specific crowdsourcing workflows, worker attributes, and work practices contribute to biases in the labelled data; how to quantify and mitigate biases as part of the labelling process; and how such mitigation approaches may impact workers and the crowdsourcing ecosystem. The outcome of the workshop will include a collaborative publication of a research agenda to improve or develop novel methods relating to crowdsourcing tools, processes and work practices to address biases in crowdsourced data. We also plan to run a Crowd Bias Challenge prior to the workshop, where participants will be asked to collect labels for a given dataset while minimising potential biases.

Aiming at the very challenging problem of motion blur caused by camera shake, object movement, etc. the traditional method using blur kernel estimation easily leads to estimation errors and makes the image restoration effect poor. We propose a deep convolutional neural network solution to restore blurred images. It is based on DeblurGAN to directly obtain deblurred images from end-to-end motion blurred images. and improves the residual network to effectively restore the detailed information of the image, Finally, through the training and testing of the deep convolution neural network model, it is proved that the method can achieve state-of-the-art performance in several commonly used indexes.

Multi-modal fusion methods for movie genre classification have shown to be superior over their single modality counterparts. However, it is still challenging to design a fusion strategy for real-world scenarios where missing data and weak labeling are common. Considering the heterogeneity in different modalities, most existing works design late fusion strategies that process and train models per modality, and combine the results at the decision level. A major drawback in such strategies is the potential loss of across-modality dependencies, which is important for understanding audiovisual contents. In this paper, we introduce a Shot-based Hybrid Fusion Network (SHFN) for movie genre classification. It consists of single-modal feature fusion networks for video and audio, a multi-modal feature fusion network working on a shot-basis, and finally a late fusion part for video-level decisions. An ablation study indicates the major contribution from video and the performance gain from the additional modality, audio. The experimental results on the LMTD-9 dataset demonstrate the effectiveness of our proposed method in movie genre classification. Our best model outperforms the state-of-the-art method by 5.7% on AUPRC(micro).KeywordsMulti-modal fusionMulti-label classificationMovie genre classification

Existing fully supervised deep learning methods usually require a large number of training samples with abundant annotations for the model training, which is extremely expensive and labor-consuming. Therefore, in order to alleviate huge labeling costs, it is highly desirable to develop weakly supervised learning methods. Here, weakly supervised learning refers to that during the model training, the labels of the training data could be inexact, incomplete, or inaccurate. Typically, there are three types of weakly supervised learning scenarios: inexact supervision, incomplete supervision, and inaccurate supervision. In this thesis, we study all three types of weak supervision scenarios based on three different fundamental 2D and 3D recognition tasks including weakly supervised object detection (WSOD), few-shot image segmentation (FSS), and weakly supervised point cloud segmentation (WSPCS). Specifically, for WSOD, we only have the image-level annotations for the novel class images in the web domain, corresponding to inexact supervision. For FSS, we only have a few pixel-level labeled images (e.g., one or five images) for the novel classes, corresponding to incomplete supervision. For WSPCS, we consider partially labeled samples as weak annotations for the model training, i.e., only a few sparse points inside the whole scene are labeled and all other points are unlabeled, corresponding to incomplete supervision. Moreover, we observe one major limitation in existing consistency-based weakly supervised point cloud segmentation methods, i.e., unsatisfied pseudo labels due to the conventional confidence-based selection, which further leads to inaccurate supervision. For weakly supervised object detection, in Chapter 3, we propose a novel webly supervised object detection (WebSOD) method for novel class detection, which only requires the web images retrieved via the internet using class names as the keywords. Here, we only have the image-level annotations for web images during the model training. Our proposed method combines bottom-up and top-down cues. Within our approach, we introduce a bottom-up mechanism based on the well-trained fully supervised object detector (e.g., Faster RCNN) as an object region estimator for web images by recognizing the common objectiveness shared between base and novel classes. With the estimated regions on the web images, we then use the top-down attention cues as the guidance for region classification. Furthermore, we propose a residual feature refinement (RFR) block to tackle the domain mismatch between the web domain and the target domain. For few-shot image segmentation, currently, the state-of-the-art methods treat this task as a conditional foreground-background segmentation problem, assuming each class is independent. Different from existing methods, in Chapter 4, we introduce the concept of meta-class, which is the meta-information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations, we propose a novel Meta-class Memory-based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer it to novel classes during the inference stage. For weakly supervised point cloud segmentation, we only have a few sparse labeled points as well as a large number of unlabeled points. To exploit the unlabeled data, we design two different methods from the aspect of adversarial training and consistency training. Firstly, in Chapter 5, considering the smoothness-based methods have achieved promising progress, we advocate applying the consistency constraint under various perturbations to effectively regularize unlabeled 3D points. In particular, we propose a novel Dual Adaptive Transformations (DAT) model for weakly supervised point cloud segmentation, where the dual adaptive transformations are performed via an adversarial strategy at both point-level and region-level, aiming at enforcing the local and structural smoothness constraints on 3D point clouds. Secondly, in Chapter 6, we observe that the straightforward way of applying consistency constraints to weakly supervised point cloud segmentation has two major limitations: unsatisfied pseudo labels due to the conventional confidence-based selection and insufficient consistency constraints due to discarding unreliable pseudo labels. Therefore, we propose a novel Reliability-Adaptive Consistency Network (RAC-Net) to use both prediction confidence and model uncertainty to measure the reliability of pseudo labels and apply consistency training on all unlabeled points while with different consistency constraints for different points based on the reliability of corresponding pseudo labels.

SUMMARYA major bottleneck in developing clinically impactful machine learning models is a lack of labeled training data for model supervision. Thus, medical researchers increasingly turn to weaker, noisier sources of supervision, such as leveraging extractions from unstructured text reports to supervise image classification. A key challenge in weak supervision is combining sources of information that may differ in quality and have correlated errors. Recently, a statistical theory of weak supervision called data programming has shown promise in addressing this challenge. Data programming now underpins many deployed machine-learning systems in the technology industry, even for critical applications. We propose a new technique for applying data programming to the problem of cross-modal weak supervision in medicine, wherein weak labels derived from an auxiliary modality (e.g., text) are used to train models over a different target modality (e.g., images). We evaluate our approach on diverse clinical tasks via direct comparison to institution-scale, hand-labeled datasets. We find that our supervision technique increases model performance by up to 6 points area under the receiver operating characteristic curve (ROC-AUC) over baseline methods by improving both coverage and quality of the weak labels. Our approach yields models that on average perform within 1.75 points ROC-AUC of those supervised with physician-years of hand labeling and outperform those supervised with physician-months of hand labeling by 10.25 points ROC-AUC, while using only person-days of developer time and clinician work—a time saving of 96%. Our results suggest that modern weak supervision techniques such as data programming may enable more rapid development and deployment of clinically useful machine-learning models.

A face annotation has many applications the main part of based face annotation is to management of most same facial images and their weak data labels. These problem, different methods are adopted. The efficiency of annotating systems are improved by using these methods. This paper proposes a review on various techniques used for detection and analysis of each technique. Combine techniques are used in retrieving facial images based on query. So it is effective to label the images with their exact names. The detected face recognition techniques can annotate the faces with exact data labels which will help to improve the detection more efficiently.

Multilabel image classification has been a hot topic in the field of computer vision and image understanding in recent years. To achieve better classification performance with fewer labeled images, multilabel active learning is used for this scenario. Several active learning methods have been proposed for multilabel image classification. However, all of them assume that either all training images have complete labels or label correlations are given at the beginning. These two assumptions are unrealistic. In fact, it is very difficult to obtain complete labels for each example, in particular when the size of labels in a multilabel dataset is very large. Typically, only partial labels are available. This is one type of “weak label” problem. To solve this weak label problem inside multilabel active learning, this paper proposes a novel solution called AE-WLMAL. AE-WLMAL explores conditional label correlations on the weak label problem with the help of input features and then utilizes label correlations to construct a unified sampling strategy and evaluate the informativeness of each example-label pair in a multilabel dataset for active sampling. In addition, a pruning strategy is adopted to further improve its computation efficiency. Moreover, AE-WLAML exploits label correlations to infer labels for unlabeled images, which further reduces human labeling cost. Our experimental results on seven real-world datasets show that AE-WLMAL consistently outperforms existing approaches.

In the era of the Internet of Things, the construction of edge computing platform has become more and more important, which has led lots of object detection applications being deployed on embedded devices. However, traditional object detection algorithms require lots of engery and a large amount of well labeled samples for training. The time spent on model training and data labeling also slows down the upgrade iteration of applications. Therefore, an object detection algorithm that requires only few energy and a few samples to update parameters could help the long-term benign development of IoT technology. In this paper, we propose an effective object detection method based on the few-shot learning, which could achieve considerable performance with few data for novel(new) classes. Our well-designed strategies could alleviate the impact of scale variation in support set under few-shot setting. Through extensive experiments, we prove that our model is superior to well-recognized baselines on few-shot object detection task.

Machine learning (ML) has recently been adopted in vehicular networks for applications such as autonomous driving, road safety prediction and vehicular object detection, due to its model-free characteristic, allowing adaptive fast response. However, most of these ML applications employ centralized learning (CL), which brings significant overhead for data trans-mission between the parameter server and vehicular edge devices. Federated learning (FL) framework has been recently introduced as an efficient tool with the goal of reducing transmission overhead while achieving privacy through the transmission of model updates instead of the whole dataset. In this paper, we investigate the usage of FL over CL in vehicular network applications to develop intelligent transportation systems. We provide a comprehensive analysis on the feasibility of FL for the ML based vehicular applications, as well as investigating object detection by utilizing image-based datasets as a case study. Then, we identify the major challenges from both learning perspective, i.e., data labeling and model training, and from the communications point of view, i.e., data rate, reliability, transmission overhead, privacy and resource management. Finally, we highlight related future research directions for FL in vehicular networks.

Optimal, efficient, and accurate drug stock management at hospital depots is crucial for ensuring the smooth operation of medical and operational services. Therefore, the use of machine learning is currently essential for managing drug stocks at hospital depots more optimally. This optimization process involves stages such as data collection, data pre-processing, attribute selection, data labeling, classification algorithm selection, model training, model eval_uation, and result interpretation. The data used in this research includes information on drug stocks at hospital depots with details on drug items, quantities, prices, depot origins, demand trends, and types of transactions. The aim of using these algorithms is to classify drug stock items into categories such as "sufficient," "deficient," and "excess" based on historical data patterns and relevant attributes. Model eval_uation is carried out by comparing classification results with actual data and measuring eval_uation metrics such as accuracy, precision, recall, and F1-score. It is hoped that the classification results will indicate the need for optimization in the previously implemented algorithms and provide new solutions for managing drug stocks at hospital depots. The Shuffle Frog Leaping algorithm (SFLA) implemented will help drug stock management staff identify demand patterns more optimally, efficiently, and accurately. Thus, this research has the potential to make significant contributions to optimizing drug stock management and decision-making at hospital depots, which will also positively impact the progress of hospital services.