Dynamic Statistical Learning in Massive Datastreams

Recent research highlighted the interest in 1) investigating the effect of variable practice on the dynamics of learning and 2) modeling the dynamics of motor skill learning to enhance understanding of individual pathways learners. Such modeling has not been suitable for predicting future performance, both in terms of retention and transfer to new tasks. The present study attempted to quantify, by means of a machine learning algorithm, the prediction of skill transfer for three practice conditions in a climbing task: constant practice (without any modifications applied during learning), imposed variable practice (with graded contextual modifications, i.e., the variants of the climbing route), and self-controlled variable practice (participants were given some control over their variant practice schedule). The proposed pipeline allowed us to measure the fitness of the test to the dataset, i.e., the ability of the dataset to be predictive of the skill transfer test. Behavioral data are difficult to model with statistical learning and tend to be 1) scarce (too modest data sample in comparison with the machine learning standards) and 2) flawed (data tend to contain voids in measurements). Despite these adversities, we were nevertheless able to develop a machine learning pipeline for behavioral data. The main findings demonstrate that the level of learning transfer varies, according to the type of practice that the dynamics pertain: we found that the self-controlled condition is more predictive of generalization ability in learners than the constant condition.

Chapter 7 Learning dynamics

Federated learning is applied in scenarios where organisations lack sufficient data volume for modelling their business logic and cannot share their data with external parties. Moreover, Industry 4.0 and IoT scenarios generate massive data streams, which normally are fed to ML/AI solutions for model training and prediction. However, in most cases, ML/AI frameworks are not prepared to work with these streaming pipelines. In this paper, we present an asynchronous federated learning solution based on the Kafka-ML data stream framework, which is able to combine federated learning and data stream capabilities within ML/AI applications. While most federated learning approaches are tailored to a specific ML model or a use case, the solution provided adapts itself to the availability of both data and ML models, achieving a flexible and dynamic federated learning solution. To validate its performance, an evaluation of the federated learning solution is carried out on different scenarios in a multi-node state-of-the-art infrastructure. Results show that this framework can work with multiple federated clients, being the resulting accuracy dependent on the amount of data and the behaviour of clients during training.

Lazy learning, such as k-nearest neighbor learning, has been widely applied to many applications. Known for well capturing data locality, lazy learning can be advantageous for highly dynamic and complex learning environments such as data streams. Yet its high memory consumption and low prediction efficiency have made it less favorable for stream oriented applications. Specifically, traditional lazy learning stores all the training data and the inductive process is deferred until a query appears, whereas in stream applications, data records flow continuously in large volumes and the prediction of class labels needs to be made in a timely manner. In this paper, we provide a systematic solution that overcomes the memory and efficiency limitations and enables fast lazy learning for concept drifting data streams. In particular, we propose a novel Lazy-tree (Ltree for short) indexing structure that dynamically maintains compact high-level summaries of historical stream records. L-trees are M-Tree [5] like, height-balanced, and can help achieve great memory consumption reduction and sub-linear time complexity for prediction. Moreover, L-trees continuously absorb new stream records and discard outdated ones, so they can naturally adapt to the dynamically changing concepts in data streams for accurate prediction. Extensive experiments on real-world and synthetic data streams demonstrate the performance of our approach.

摘要: 确定学习运用自适应控制和动力学系统的概念与方法, 研究未知动态环境下的知识获取、表达、存储和利用等问题. 针对产生周期或回归轨迹的连续 非线性动态系统, 确定学习可以对其未知系统动态进行局部准确建模, 其基本要 素包括: 1)使用径向基函数(Radial basis function, RBF)神经网络; 2)对于周期(或回归)状态轨迹 满足部分持续激励条件; 3)在周期(或回归)轨迹的邻域内实现对非线性系统动态的局部准确神经网络逼近(局部准确建模); 4)所学的知识以时不变且空间分布的方式表达、以常值神经网络权值的方式存储, 并可在动态环境下用于动态模式的快速识别或者闭环神经网络控制. 本文针对离散动态系统, 扩展了确定学习理论, 提出一个根据时态数据序列对离散动态系统进行建模与控制的框架. 首先, 运用确定学习原理和离散系统的自适应辨识方法, 实现对产生时态数据的离散非线性系统的未知动态进行局部准确的神经网络建模, 并利用此建模结果对时态数据序列进行时不变表达. 其次, 提出时态数据序列的基于动力学的相似性定义, 以及对离散动态系统产生的时态数据序列(亦可称为动态模式)进行快速识别方法. 最后, 针对离散非线性控制系统, 实现了基于时态数据序列对控制系统动态的闭环辨识(局部准确建模). 所学关于闭环动态的知识可用于基于模式的智能控制. 本文表明确定学习可以为时态数据挖掘的研究提供新的途径, 并为基于数据的建模与控制等问题提供新的研究思路. 关键词: 确定学习 / 时态数据序列 / 离散动态系统 / 基于数据的建模 / 部分持续激励条件 / 时态数据挖掘 / 动态模式识别 / 基于模式的控制

Dynamics of brain activation during learning of syllable-symbol paired associations

With the explosive growth of data scale, the high-dimensional sparsity, dynamic evolution and multi-source heterogeneity of complex data sets have brought great challenges to traditional statistical modeling methods. Machine learning provides a new paradigm for complex data modeling with its ability of nonlinear function approximation and automatic feature learning. However, the pure data-driven machine learning model has some problems such as poor interpretability and insufficient robustness. This article proposes a Dynamic Statistical Constrained Deep Learning (DSC-DL), which integrates techniques such as graph neural networks (GNN), Bayesian dynamic regularization, and multimodal variational encoders to address the challenges of complex data. DSC-DL deals with high-dimensional sparsity by structural feature selection and graph embedding dimension reduction technology, captures dynamic evolution by time-aware hidden variable modeling, and realizes multi-source heterogeneous fusion through joint variational inference and adaptive fusion mechanism. In this paper, the generalization error bound of DSC-DL under dependent identically distributed data is derived, which provides theoretical guarantee for its robustness. Experimental results on financial fraud detection, social topic prediction and cancer typing diagnosis show that DSC-DL can effectively handle complex data sets, and shows excellent prediction performance, robustness and interpretability.

Water availability plays a critical role in shaping terrestrial ecosystems, particularly in low- and mid-latitude regions. The sensitivity of vegetation growth to precipitation strongly regulates global vegetation dynamics and their responses to drought, yet sensitivity changes in response to climate change remain poorly understood. Here we use long-term satellite observations combined with a dynamic statistical learning approach to examine changes in the sensitivity of vegetation greenness to precipitation over the past four decades. We observe a robust increase in precipitation sensitivity (0.624% yr−1) for drylands, and a decrease (−0.618% yr−1) for wet regions. Using model simulations, we show that the contrasting trends between dry and wet regions are caused by elevated atmospheric CO2 (eCO2). eCO2 universally decreases the precipitation sensitivity by reducing leaf-level transpiration, particularly in wet regions. However, in drylands, this leaf-level transpiration reduction is overridden at the canopy scale by a large proportional increase in leaf area. The increased sensitivity for global drylands implies a potential decrease in ecosystem stability and greater impacts of droughts in these vulnerable ecosystems under continued global change.

In this manuscript, we develop a multi-party framework tailored for multiple data contributors seeking machine learning insights from combined data sources. Grounded in statistical learning principles, we introduce the Multi-Key Homomorphic Encryption Logistic Regression (MK-HELR) algorithm, designed to execute logistic regression on encrypted multi-party data. Given that models built on aggregated datasets often demonstrate superior generalization capabilities, our approach offers data contributors the collective strength of shared data while ensuring their original data remains private due to encryption. Apart from facilitating logistic regression on combined encrypted data from diverse sources, this algorithm creates a collaborative learning environment with dynamic membership. Notably, it can seamlessly incorporate new participants during the learning process, addressing the key limitation of prior methods that demanded a predetermined number of contributors to be set before the learning process begins. This flexibility is crucial in real-world scenarios, accommodating varying data contribution timelines and unanticipated fluctuations in participant numbers, due to additions and departures. Using the AI4I public predictive maintenance dataset, we demonstrate the MK-HELR algorithm, setting the stage for further research in secure, dynamic, and collaborative multi-party learning scenarios.

Humans are highly sensitive to statistical regularities in their environment. This phenomenon, usually referred as statistical learning, is most often assessed using post-learning behavioural measures that are limited by a lack of sensibility and do not monitor the temporal dynamics of learning. In the present study, we used magnetoencephalographic frequency-tagged responses to investigate the neural sources and temporal development of the ongoing brain activity that supports the detection of regularities embedded in auditory streams. Participants passively listened to statistical streams in which tones were grouped as triplets, and to random streams in which tones were randomly presented. Results show that during exposure to statistical (vs. random) streams, tritone frequency-related responses reflecting the learning of regularities embedded in the stream increased in the left supplementary motor area and left posterior superior temporal sulcus (pSTS), whereas tone frequency-related responses decreased in the right angular gyrus and right pSTS. Tritone frequency-related responses rapidly developed to reach significance after 3min of exposure. These results suggest that the incidental extraction of novel regularities is subtended by a gradual shift from rhythmic activity reflecting individual tone succession toward rhythmic activity synchronised with triplet presentation, and that these rhythmic processes are subtended by distinct neural sources.

IntroductionBased on the schema theory advanced by Rumelhart and Norman, we shed light on the individual variability in brain dynamics induced by hybridization of learning methodologies, particularly alternating unsupervised learning and supervised learning in language acquisition. The concept of “schema” implies a latent knowledge structure that a learner holds and updates as intrinsic to his or her cognitive space for guiding the processing of newly arriving information.MethodsWe replicated the cognitive experiment of Onnis and Thiessen on implicit statistical learning ability in language acquisition but included additional factors of prosodic variables and explicit supervised learning. Functional magnetic resonance imaging was performed to identify the functional network connections for schema updating by alternately using unsupervised and supervised artificial grammar learning tasks to segment potential words.ResultsRegardless of the quality of task performance, the default mode network represented the first stage of spontaneous unsupervised learning, and the wrap‐up accomplishment for successful subjects of the whole hybrid learning in concurrence with the task‐related auditory language networks. Furthermore, subjects who could easily “tune” the schema for recording a high task precision rate resorted even at an early stage to a self‐supervised learning, or “superlearning,” as a set of different learning mechanisms that act in synergy to trigger widespread neuro‐transformation with a focus on the cerebellum.ConclusionsInvestigation of the brain dynamics revealed by functional connectivity imaging analysis was able to differentiate the synchronized neural responses with respect to learning methods and the order effect that affects hybrid learning.