Automating Data Science Pipelines with Tensor Completion

Data science has become prevalent in a large variety of domains. Inherent in its practice is an exploratory, probing, and fact finding journey, which consists of the assembly, adaptation, and execution of complex data science pipelines. The trustworthiness of the results of such pipelines rests entirely on their ability to be reproduced with fidelity, which is difficult if pipelines are not documented or recorded minutely and consistently. This difficulty has led to a reproducibility crisis and presents a major obstacle to the safe adoption of the pipeline results in production environments. The crisis can be resolved if the provenance for each data science pipeline is captured transparently as pipelines are executed. However, due to the complexity of modern data science pipelines, transparently capturing sufficient provenance to allow for reproducibility is challenging. As a result, most existing systems require users to augment their code or use specific tools to capture provenance, which hinders productivity and results in a lack of adoption. In this paper, we present Ursprung, 1 a transparent provenance collection system designed for data science environments. 2 The Ursprung philosophy is to capture provenance and build lineage by integrating with the execution environment to automatically track static and runtime configuration parameters of data science pipelines. Rather than requiring data scientists to make changes to their code, Ursprung records basic provenance information from system-level sources and combines it with provenance from application-level sources (e.g., log files, stdout), which can be accessed and recorded through a domain-specific language. In our evaluation, we show that Ursprung is able to capture sufficient provenance for a variety of use cases and only adds an overhead of up to 4%.

Increasingly larger number of software systems today are including data science components for descriptive, predictive, and prescriptive analytics. The collection of data science stages from acquisition, to cleaning/curation, to modeling, and so on are referred to as data science pipelines. To facilitate research and practice on data science pipelines, it is essential to understand their nature. What are the typical stages of a data science pipeline? How are they connected? Do the pipelines differ in the theoretical representations and that in the practice? Today we do not fully understand these architectural characteristics of data science pipelines. In this work, we present a three-pronged comprehensive study to answer this for the state-of-the-art, data science in-the-small, and data science in-the-large. Our study analyzes three datasets: a collection of 71 proposals for data science pipelines and related concepts in theory, a collection of over 105 implementations of curated data science pipelines from Kaggle competitions to understand data science in-the-small, and a collection of 21 mature data science projects from GitHub to understand data science in-the-large. Our study has led to three representations of data science pipelines that capture the essence of our subjects in theory, in-the-small, and in-the-large.

Data science pipelines inform and influence many daily decisions, from what we buy to who we work for and even where we live. When designed incorrectly, these pipelines can easily propagate social inequity and harm. Traditional solutions are technical in nature; e.g., mitigating biased algorithms. In this vision paper, we introduce a novel lens for promoting responsible data science using theories of behavior change that emphasize not only technical solutions but also the behavioral responsibility of practitioners. By integrating behavior change theories from cognitive psychology with data science workflow knowledge and ethics guidelines, we present a new perspective on responsible data science. We present example data science interventions in machine learning and visual data analysis, contextualized in behavior change theories that could be implemented to interrupt and redirect potentially suboptimal or negligent practices while reinforcing ethically conscious behaviors. We conclude with a call to action to our community to explore this new research area of behavior change interventions for responsible data science.

In the era of data science, a huge amount of data has emerged in the form of tensors. In many applications, the collected tensor data are incomplete with missing entries, which affects the analysis process. In this paper, we investigate a new method for tensor completion, in which a low-rank tensor approximation is used to exploit the global structure of data, and sparse coding is used for elucidating the local patterns of data. Regarding the characterization of low-rank structures, a weighted nuclear norm for the tensor is introduced. Meanwhile, an orthogonal dictionary learning process is incorporated into sparse coding for more effective discovery of the local details of data. By simultaneously using the global patterns and local cues, the proposed method can effectively and efficiently recover the lost information of incomplete tensor data. The capability of the proposed method is demonstrated with several experiments on recovering MRI data and visual data, and the experimental results have shown the excellent performance of the proposed method in comparison with recent related methods.

Successful data-driven science requires complex data engineering pipelines to clean, transform, and alter data in preparation for machine learning, and robust results can only be achieved when each step in the pipeline can be justified, and its effect on the data explained. In this framework, we aim at providing data scientists with facilities to gain an in-depth understanding of how each step in the pipeline affects the data, from the raw input to training sets ready to be used for learning. Starting from an extensible set of data preparation operators commonly used within a data science setting, in this work we present a provenance management infrastructure for generating, storing, and querying very granular accounts of data transformations, at the level of individual elements within datasets whenever possible. Then, from the formal definition of a core set of data science preprocessing operators, we derive a provenance semantics embodied by a collection of templates expressed in PROV, a standard model for data provenance. Using those templates as a reference, our provenance generation algorithm generalises to any operator with observable input/output pairs. We provide a prototype implementation of an application-level provenance capture library to produce, in a semi-automatic way, complete provenance documents that account for the entire pipeline. We report on the ability of that reference implementation to capture provenance in real ML benchmark pipelines and over TCP-DI synthetic data. We finally show how the collected provenance can be used to answer a suite of provenance benchmark queries that underpin some common pipeline inspection questions, as expressed on the Data Science Stack Exchange.

This year, due to the lift of travel restrictions and the improvement of the Covid-19 pandemic world-wide, SIGMOD is being held in a hybrid format, with a special emphasis on bringing back much-needed face-to-face interactions among our community. We are excited to host SIGMOD in person again after the past two online editions. We have an excellent technical program with outstanding research, industrial and demonstration track presentations, keynotes, tutorials, panels, and the awards session. Building upon the lessons we learned in the online editions during the pandemic, SIGMOD combined asynchronous and synchronous elements. Every technical talk was pre-recorded and posted for viewing ahead of time. Questions and discussions were handled asynchronously through Slack channels. But in addition, synchronous sessions were held at the conference where presentations were viewed inperson and live-streamed, and the audience and speakers could interact. This year, we had three categories of papers in the Research Track, (a) Data Management (b) Data Science and (c) Data-centric Applications. Data Science papers focused on data intensive components of data science pipelines, solving problems in areas of interest to the community inspired by real applications. Data-centric applications papers presented novel applications of data management systems in other neighbouring domains beyond Computer Science in order to inspire future research in our community. In the Research Track this year, we received 514 research submissions (259 for Round 1 and 255 for Round 2), which were extensively reviewed by 191 program committee members, 26 associate editors, and several external reviewers. We accepted 151 submissions (a 29.3% acceptance rate), most of them after a revision phase that gave authors 2 months to revise and resubmit their papers in response to the reviewer comments. This year, we expanded on the detailed reviewing instructions provided to the reviewers to promote constructive reviewing. Authors were given the opportunity to provide structured feedback directly to the reviewers, associate editors and the program chairs. Furthermore, papers that were selected for revision were given an extra page in their final camera-ready version to allow for more space to address reviewer comments and suggestions.

Xel: A cloud-agnostic data platform for the design-driven building of high-availability data science services

BackgroundIn response to the COVID-19 pandemic, a new oral antiviral called nirmatrelvir-ritonavir (PaxlovidTM) was authorized for use in Canada in January 2022. In vitro studies have reported mutations in Mpro protein that may be associated with the development of nirmatrelvir resistance.ObjectivesTo survey the prevalence, relevance and temporal patterns of Mpro mutations among SARS-CoV-2 Omicron lineages in Ontario, Canada.MethodsA total of 93,082 Mpro gene sequences from December 2021 to September 2023 were analyzed. Reported in vitro Mpro mutations were screened against our database using in-house data science pipelines to determine the nirmatrelvir resistance. Negative binomial regression was conducted to analyze the temporal trends in Mpro mutation counts over the study time period.ResultsA declining trend was observed in non-synonymous mutations of Mpro sequences, showing a 7.9% reduction (95% CI: 6.5%–‬9.4%; p<0.001) every 30 days. The P132H was the most prevalent mutation (higher than 95%) in all Omicron lineages. In vitro nirmatrelvir-resistant mutations were found in 3.12% (n=29/929) Omicron lineages with very low counts, ranging from one to 19. Only two mutations, A7T (n=19) and M82I (n=9), showed temporal presence among the BA.1.1 in 2022 and the BQ.1.2.3 in 2022, respectively.ConclusionThe observations suggest that, as of September 2023, no significant or widespread resistance to nirmatrelvir has developed among SARS-CoV-2 Omicron variants in Ontario. This study highlights the importance of creating automated monitoring systems to track the emergence of nirmatrelvir-resistant mutations within the SARS-CoV-2 virus, utilizing genomic data generated in real-time.

VeridicalFlow is a Python package that simplifies building reproducible and trustworthy data science pipelines using the PCS (predictability-computability-stability) framework It provides users with a simple interface for stability analysis, i.e., checking the robustness of results from a data science pipeline to various judgement calls made during modeling. This ensures that arbitrary judgement calls made by data practitioners (e.g., specifying a default imputation strategy) do not dramatically alter the final conclusions made in a modeling pipeline. In addition to wrappers facilitating stability analysis, VeridicalFlow also automates many cumbersome coding aspects of Python pipelines, including experiment tracking and saving, parallelization, and caching, all through integrations with existing Python packages. Overall, the package helps to code using the PCS framework by screening models for predictive performance, helping automate computation, and facilitating stability analysis.

In the machine learning and data science pipelines, feature extraction is considered the most crucial component according to researchers, where generating a discriminative feature matrix is the utmost challenging task to achieve high classification accuracy. Generally, the classical feature extraction techniques are sensitive to the noisy component of the signal and need more time for training. To deal with these issues, a comparatively new feature extraction technique, referred to as a wavelet scattering transform (WST) is utilized, and incorporated with ML classifiers to design a framework for bearing fault classification in this paper. The WST is a knowledge-based technique, and the structure is similar to the convolution neural network. This technique provides low-variance features of real-valued signals, which are usually necessary for classification tasks. These signals are resistant to signal deformation and preserve information at high frequencies. The current signal data from a publicly available dataset for three different bearing conditions are considered. By combining the scattering path coefficients, the decomposition coefficients from the 0th and 1st layers are considered as features. The experimental results demonstrate that WST-based features, when used with ensemble ML algorithms, could achieve more than 99% classification accuracy. The performance of ANN models with these features is similar. This work exhibits that utilizing WST coefficients for the motor current signal as features can improve the bearing fault classification accuracy when compared to other feature extraction approaches such as empirical wavelet transform (EWT), information fusion (IF), and wavelet packet decomposition (WPD). Thus, our proposed approach can be considered as an effective classification method for the fault diagnosis of rotating machinery.

Most advanced Internet of Things solutions today are more than just technology. This chapter first introduces crowdsourcing and human‐in‐the‐loop (HITL), two related approaches for realizing use cases and devises data science pipelines that seamlessly combine machine with human and collective intelligence. Broadly speaking, there are two main categories of crowdsourcing activity: microtask and macrotask crowdsourcing. These can be distinguished based on task granularity, or the amount of work required for – and by extension, the complexity of – the task assigned to individual workers. HITL and crowdsourcing are related, but there are important differences: crowdsourcing is a distributed problem‐solving approach. An important part of a crowdsourcing project is how to allocate the tasks to participants and how to validate and aggregate their contributions. HITL does not necessarily involve decentralization. The chapter also discusses two instances of crowdsourcing for location data: spatial crowdsourcing and citizen sensing, which are particularly relevant in a smart city context.