• Home
  • Search
  • Real-Time Regression Analysis of Streaming Clustered Data With Possible Abnormal Data Batches
  • Open Access IconOpen Access
  • Cite Icon24
  • https://doi.org/10.1080/01621459.2022.2026778Copy DOI Icon

Real-Time Regression Analysis of Streaming Clustered Data With Possible Abnormal Data Batches

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This article develops an incremental learning algorithm based on quadratic inference function (QIF) to analyze streaming datasets with correlated outcomes such as longitudinal data and clustered data. We propose a renewable QIF (RenewQIF) method within a paradigm of renewable estimation and incremental inference, in which parameter estimates are recursively renewed with current data and summary statistics of historical data, but with no use of any historical subject-level raw data. We compare our renewable estimation method with both offline QIF and offline generalized estimating equations (GEE) approach that process the entire cumulative subject-level data all together, and show theoretically and numerically that our renewable procedure enjoys statistical and computational efficiency. We also propose an approach to diagnose the homogeneity assumption of regression coefficients via a sequential goodness-of-fit test as a screening procedure on occurrences of abnormal data batches. We implement the proposed methodology by expanding existing Spark’s Lambda architecture for the operation of statistical inference and data quality diagnosis. We illustrate the proposed methodology by extensive simulation studies and an analysis of streaming car crash datasets from the National Automotive Sampling System-Crashworthiness Data System (NASS CDS). Supplementary materials for this article are available online.

Loading PDF

Similar Papers
  • Research Article

Efficient parameter estimation for multivariate accelerated failure time model via the quadratic inference functions method

  • Oct 01, 2019
  • Random Matrices: Theory and Applications
  • Liya Fu +3
  • Research Article
  • Citations1

Similarity scoring methodology for comparing real-world cases to crash test standards

  • Oct 04, 2013
  • International Journal of Crashworthiness
  • Kathryn L Loftis +4
  • Research Article

Risks and patterns of serious injury in motor vehicle crashes with pole and car side impacts

  • Jan 01, 2011
  • Apostolos A Alexandridis
  • PDF
  • Research Article
  • Citations4

A bias-reduced generalized estimating equation approach for proportional odds models with small-sample longitudinal ordinal data

  • Jun 28, 2024
  • BMC Medical Research Methodology
  • Yukio Tada +1
  • Research Article
  • Citations1

Generalized Empirical Likelihood Inference in Partially Linear Errors-in-Variables Models with Longitudinal data

  • Aug 17, 2016
  • Hacettepe Journal of Mathematics and Statistics
  • Juanfang Liu +2
  • Research Article
  • Citations31

Partial Linear Models for Longitudinal Data Based on Quadratic Inference Functions

  • Feb 04, 2008
  • Scandinavian Journal of Statistics
  • Yang Bai +2
  • Research Article
  • Citations54

Model Selection for Generalized Estimating Equations Accommodating Dropout Missingness

  • Mar 29, 2012
  • Biometrics
  • Chung‐Wei Shen +1
  • Research Article
  • Citations4

Variable selection for longitudinal varying coefficient errors-in-variables models

  • Aug 08, 2020
  • Communications in Statistics - Theory and Methods
  • Mingtao Zhao +2
  • Research Article
  • Citations9

A partially linear additive model for clustered proportion data.

  • Dec 15, 2017
  • Statistics in medicine
  • Weihua Zhao +2
  • Research Article
  • Citations8

%QLSSASMacro: ASASMacro for Analysis of Correlated Data Using Quasi-Least Squares

  • Jan 01, 2010
  • Journal of Statistical Software
  • Hanjoo Kim +1
  • Research Article

The Development of Web-based Graphical User Interface for Learning and Fitting Generalized Estimating Equation with Spline Smoothers

  • Feb 23, 2018
  • Jurnal ILMU DASAR
  • Made Tirta +1
  • Research Article

Investigating the association between school substance programs and student substance use: accounting for informative cluster size.

  • Jan 01, 2025
  • Epidemiologic methods
  • Aya A Mitani +3
  • Research Article

Correlated and misclassified binary observations in complex surveys

  • May 30, 2020
  • Canadian Journal of Statistics
  • Hon Yiu So +2
  • Research Article

Functional regression with intensively measured longitudinal outcomes: a new lens through data partitioning

  • May 06, 2025
  • Canadian Journal of Statistics
  • Cole Manschot +1
  • Research Article

A Modified Normalized Power Prior Approach for Bayesian Adaptive Borrowing in Item Response Theory Models.

  • May 01, 2026
  • Statistics in medicine
  • Qiang Zhang +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.