In machine learning, the selection of a promising model from a potentially large number of competing models and the assessment of its generalization performance are critical tasks that need careful consideration. Typically, model selection and evaluation are strictly separated tasks, splitting the sample at hand into training, validation, and evaluation sets, and only computing a single confidence interval for the prediction performance of the final selected model. We however regard the selection problem as a simultaneous inference problem and propose an algorithm to compute valid lower confidence bounds for multiple models that have been selected based on their prediction performance in the evaluation set. For this, we use bootstrap tilting and a maxT-type multiplicity correction. Various simulation experiments show that this leads to lower confidence bounds for the conditional performance that are at least as good as bounds from standard methods, and that reliably reach the nominal coverage probability. Also, a better performing final prediction model is selected this way, especially when the sample size is small. The approach is universally applicable for any combination of prediction models, any model selection strategy, and any prediction performance measure that accepts weights.

The more extensive access to time-series data, especially for biomedical purposes, raises new methodological challenges, particularly regarding missing values. Functional linear discriminant analysis (FLDA) extends Linear Discriminant Analysis (LDA)-mediated multiclass classification and dimension reduction to data in the form of fragmented observations of a univariate function. For large multivariate and partially-observed data, there are two challenges: (i) statistical dependencies between different components of a multivariate function and (ii) heterogeneous sampling times with missing features. We here develop a multivariate version of FLDA, called MUDRA, to tackle these challenges and describe a computationally efficient expectation/conditional-maximisation (ECM) algorithm to infer its parameters without any tensor inversions. We assess its predictive power on the “Articulary Words” dataset and show its improvement over the state-of-the-art, especially in the case of missing data. This advancement in dimension reduction of multivariate functional data holds promise for enhancing classification accuracy in scenarios like partially observed short multivariate time series analysis.

Social interactions are prevalent in our lives. These can be observed, e. g., online using social media, however, also offline specifically using sensors. In such contexts, typically time-stamped interactions are recorded, which can also be inferred from real-time location of humans. Such interaction data can then be modeled as so-called social interaction networks. For their analysis, a variety of different approaches can be applied. A prominent research direction is then the detection of patterns describing specific subgroups with exceptional behavioral characteristics, given some measure of interest. In the standard case of plain graphs modeling the interaction networks, methods for identifying such subgroups mainly focus on structural characteristics of the network and/or the induced subgraph. For attributed social networks, then additional attributive information can be exploited. This paper proposes to focus on the dyadic structure of the attributed social interaction networks, thus enabling a compositional perspective for identifying interesting subgroup patterns. Specifically, we can then analyze spatio-temporal data modeled as attributed social interaction networks for identifying exceptional social behavior. The presented approach adapts local pattern mining using subgroup discovery to the dyadic setting, exploiting attribute information of the spatio-temporal attributed interaction networks. With this, specific characteristics of social interactions are considered, i. e., duration and frequency, for identifying subgroups capturing social behavior that deviates from the norm. For subgroup discovery, we propose according interestingness measures in the form of seven novel quality functions and discuss their properties. In our experimentation, we perform an evaluation demonstrating the efficacy of the presented approach using four real-world datasets on face-to-face interactions in academic conferencing as well as school playground contexts. Our results indicate that the proposed method returns interesting, meaningful, and valid findings and results.

Deep neural networks are increasingly employed in high-stakes medical applications, despite their tendency for shortcut learning in the presence of spurious correlations, which can have potentially fatal consequences in practice. Whereas a multitude of works address either the detection or mitigation of such shortcut behavior in isolation, the Reveal2Revise approach provides a comprehensive bias mitigation framework combining these steps. However, effectively addressing these biases often requires substantial labeling efforts from domain experts. In this work, we review the steps of the Reveal2Revise framework and enhance it with semi-automated interpretability-based bias annotation capabilities. This includes methods for the sample- and feature-level bias annotation, providing valuable information for bias mitigation methods to unlearn the undesired shortcut behavior. We show the applicability of the framework using four medical datasets across two modalities, featuring controlled and real-world spurious correlations caused by data artifacts. We successfully identify and mitigate these biases in VGG16, ResNet50, and contemporary Vision Transformer models, ultimately increasing their robustness and applicability for real-world medical tasks. Our code is available at https://github.com/frederikpahde/medical-ai-safety.

Over the last few decades, machine learning (ML) applications have grown exponentially, yielding several benefits to society. However, these benefits are tempered with concerns of discriminatory behaviours exhibited by ML models. In this regard, fairness in machine learning has emerged as a priority research area. Consequently, several fairness metrics and algorithms have been developed to mitigate against discriminatory behaviours that ML models may possess. Yet still, very little attention has been paid to the problem of naturally occurring changes in data patterns (aka data distributional drift), and its impact on fairness algorithms and metrics. In this work, we study this problem comprehensively by analyzing 4 fairness-unaware baseline algorithms and 7 fairness-aware algorithms, carefully curated to cover the breadth of its typology, across 5 datasets including public and proprietary data, and evaluated them using 3 predictive performance and 10 fairness metrics. In doing so, we show that (1) data distributional drift is not a trivial occurrence, and in several cases can lead to serious deterioration of fairness in so-called fair models; (2) contrary to some existing literature, the size and direction of data distributional drift is not correlated to the resulting size and direction of unfairness; and (3) choice of, and training of fairness algorithms is impacted by the effect of data distributional drift which is largely ignored in the literature. Emanating from our findings, we synthesize several policy implications of data distributional drift on fairness algorithms that can be very relevant to stakeholders and practitioners.