Real-time robot learning with locally weighted statistical learning

Simplified Techniques for Adaptive Control of Robotic Systems

The complexity of the kinematic and dynamic structure of humanoid robots make conventional analytical approaches to control increasingly unsuitable for such systems. Learning techniques offer a possible way to aid controller design if insufficient analytical knowledge is available, and learning approaches seem mandatory when humanoid systems are supposed to become completely autonomous. While recent research in neural networks and statistical learning has focused mostly on learning from finite data sets without stringent constraints on computational efficiency, learning for humanoid robots requires a different setting, characterized by the need for real-time learning performance from an essentially infinite stream of incrementally arriving data. This paper demonstrates how even high-dimensional learning problems of this kind can successfully be dealt with by techniques from nonparametric regression and locally weighted learning. As an example, we describe the application of one of the most advanced of such algorithms, Locally Weighted Projection Regression (LWPR), to the on-line learning of three problems in humanoid motor control: the learning of inverse dynamics models for model-based control, the learning of inverse kinematics of redundant manipulators, and the learning of oculomotor reflexes. All these examples demonstrate fast, i.e., within seconds or minutes, learning convergence with highly accurate final peformance. We conclude that real-time learning for complex motor system like humanoid robots is possible with appropriately tailored algorithms, such that increasingly autonomous robots with massive learning abilities should be achievable in the near future.

The development and maturation of multisensory processing and integration depend on the brain’s ability to detect patterns and adjust behaviour based on feedback. However, the developmental dynamics of these mechanisms are not yet fully understood. This study examines neural and behavioural differences in multisensory statistical and reinforcement learning processes by comparing adults and children, and further investigates these processes across two learning tasks of varying difficulty in children. Using a discriminative choice and a match recognition multisensory associative learning task with embedded statistical stimulus regularities, 28 adults (19.0–30.9 years) and 2 groups of children (N = 28 and N = 29, 8.5–12.8 years) learned novel, non-linguistic associations of audio–visual or tactile–visual stimuli. Behaviourally, accuracy increased and reaction time decreased over time in all groups, and these learning effects were stronger in adults than in children and for audio–visual than for tactile–visual learning. Computational modelling revealed that adults employed more advanced learning strategies, with greater sensitivity to value differences, and more deterministic choice patterns. Behaviourally, reaction times were slower for more surprising trials across all groups. Neuroimaging showed that statistical surprise modulated activity in the lateral prefrontal cortex, intraparietal sulcus, and anterior cingulate cortex, while reward prediction errors were linked to striatal, medial frontal, and hippocampal regions. Reward prediction errors were consistent between adults and children and between task variants, whereas neural surprise processing emerged only in the match recognition task and showed developmental differences, indicating immaturity in children’s ability to detect statistical regularities. These findings highlight the importance of task design when comparing learning differences across developmental groups and showcase how reinforcement and statistical learning can be studied in parallel.

Large and complex data sets are increasingly available for research in critical care. To analyze these data, researchers use techniques commonly referred to as statistical learning or machine learning (ML). The latter is known for large successes in the field of diagnostics, for example, by identification of radiological anomalies. In other research areas, such as clustering and prediction studies, there is more discussion regarding the benefit and efficiency of ML techniques compared with statistical learning. In this viewpoint, we aim to explain commonly used statistical learning and ML techniques and provide guidance for responsible use in the case of clustering and prediction questions in critical care. Clustering studies have been increasingly popular in critical care research, aiming to inform how patients can be characterized, classified, or treated differently. An important challenge for clustering studies is to ensure and assess generalizability. This limits the application of findings in these studies toward individual patients. In the case of predictive questions, there is much discussion as to what algorithm should be used to most accurately predict outcome. Aspects that determine usefulness of ML, compared with statistical techniques, include the volume of the data, the dimensionality of the preferred model, and the extent of missing data. There are areas in which modern ML methods may be preferred. However, efforts should be made to implement statistical frameworks (e.g., for dealing with missing data or measurement error, both omnipresent in clinical data) in ML methods. To conclude, there are important opportunities but also pitfalls to consider when performing clustering or predictive studies with ML techniques. We advocate careful valuation of new data-driven findings. More interaction is needed between the engineer mindset of experts in ML methods, the insight in bias of epidemiologists, and the probabilistic thinking of statisticians to extract as much information and knowledge from data as possible, while avoiding harm.

Stress is widely considered to negatively impact hippocampal function, thus impairing episodic memory. However, the hippocampus is not merely the seat of episodic memory. Rather, it also (via distinct circuitry) supports statistical learning. On the basis of rodent work suggesting that stress may impair the hippocampal pathway involved in episodic memory while sparing or enhancing the pathway involved in statistical learning, we developed a behavioral experiment to investigate the effects of acute stress on both episodic memory and statistical learning in humans. Participants were randomly assigned to one of three conditions: stress (socially evaluated cold pressor) immediately before learning, stress ∼15 min before learning, or no stress. In the learning task, participants viewed a series of trial-unique scenes (allowing for episodic encoding of each image) in which certain scene categories reliably followed one another (allowing for statistical learning of associations between paired categories). Memory was assessed 24 hr later to isolate stress effects on encoding/learning rather than retrieval. We found modest support for our hypothesis that acute stress can amplify statistical learning: Only participants stressed ∼15 min in advance exhibited reliable evidence of learning across multiple measures. Furthermore, stress-induced cortisol levels predicted statistical learning retention 24 hr later. In contrast, episodic memory did not differ by stress condition, although we did find preliminary evidence that acute stress promoted memory for statistically predictable information and attenuated competition between statistical and episodic encoding. Together, these findings provide initial insights into how stress may differentially modulate learning processes within the hippocampus.

Shill attacks are a serious threat to the stability of filtering and recommendation systems. These attacks involve the injection of fake profiles into the system, which can compromise the reliability of system output. Several shilling attack detection techniques have been proposed, but they often have limitations in terms of accuracy. This works presents a enhanced method for detecting attacks in collaborative recommender systems. The proposed method is based on a combination of statistical and machine learning techniques. The statistical techniques are used to identify anomalous user behavior, while the machine learning techniques are used to classify users as either malicious or benign. The main contribution of the proposed method is the use of a hybrid approach that combines the strengths of statistical and machine learning techniques. The statistical techniques are able to identify anomalous user behavior that is not easily detected by machine learning techniques. The machine learning techniques are able to classify users as either malicious or benign with a high degree of accuracy. The proposed method was evaluated on a real-world dataset. The results showed that the proposed method was able to detect attacks with a high degree of accuracy. The proposed method uses a combination of ensemble learning and feature selection to achieve better accuracy than previous methods. The results of the experiments show that the proposed method can achieve an accuracy of up to 99%.

The neural basis of statistical learning as it occurs over time was explored with stimuli drawn from a natural language (Russian nouns). The input reflected the “rules” for marking categories of gendered nouns, without making participants explicitly aware of the nature of what they were to learn. Participants were scanned while listening to a series of gender-marked nouns during four sequential scans, and were tested for their learning immediately after each scan. Although participants were not told the nature of the learning task, they exhibited learning after their initial exposure to the stimuli. Independent component analysis of the brain data revealed five task-related sub-networks. Unlike prior statistical learning studies of word segmentation, this morphological learning task robustly activated the inferior frontal gyrus during the learning period. This region was represented in multiple independent components, suggesting it functions as a network hub for this type of learning. Moreover, the results suggest that subnetworks activated by statistical learning are driven by the nature of the input, rather than reflecting a general statistical learning system.

Humans observe and infer things in a disentanglement way. Instead of remembering all pixel by pixel, learn things with factors like shape, scale, colour etc. Robot task learning is an open problem in the field of robotics. The task planning in the robot workspace with many constraints makes it even more challenging. In this work, a disentanglement learning of robot tasks with Convolutional Variational Autoencoder is learned, effectively capturing the underlying variations in the data. A robot dataset for disentanglement evaluation is generated with the Selective Compliance Assembly Robot Arm. The disentanglement score of the proposed model is increased to 0.206 with a robot path position accuracy of 0.055, while the state-of-the-art model (VAE) score was 0.015, and the corresponding path position accuracy is 0.053. The proposed algorithm is developed in Python and validated on the simulated robot model in Gazebo interfaced with Robot Operating System.

ABSTRACTPrevious studies found a relationship between performance on statistical learning (SL) tasks and reading ability and developmental dyslexia. Thus, it has been suggested that the ability to implicitly learn patterns may be important for reading acquisition. Causal mechanisms behind this relationship are unclear: Although orthographic sensitivity to letter bigrams may emerge through SL and facilitate reading, there is no empirical support for this link. We test 84 adults on two SL tasks, reading tests, and a bigram sensitivity task. We test for correlations using Bayes factors. This serves to test the prediction that SL and reading ability are correlated and to explore sensitivity to bigram legality as a potential mediator. We find no correlations between SL tasks and reading ability, SL and bigram sensitivity, or between the SL tasks. We conclude that correlating SL with reading ability may not yield replicable results, partly due to low correlations between SL tasks.

Robot Learning from Demonstration (RLfD) seeks to enable lay users to encode desired robot behaviors as autonomous controllers. Current work uses a human’s demonstration of the target task to initialize the robot’s policy, and then improves its performance either through practice (with a known reward function), or additional human interaction. In this article, we focus on the initialization step and consider what can be learned when the humans do not provide successful examples. We develop probabilistic approaches that avoid reproducing observed failures while leveraging the variance across multiple attempts to drive exploration. Our experiments indicate that failure data do contain information that can be used to discover successful means to accomplish tasks. However, in higher dimensions, additional information from the user will most likely be necessary to enable efficient failure-based learning.

One of the major problems in managing large-scale distributed systems is the prediction of the application performance. The complexity of the systems and the availability of monitored data have motivated the applicability of machine learning and other statistical techniques to induce performance models and forecast performance degradation problems. However, there is a stringent need for additional experimental and comparative studies, since there is no optimal method for all cases. In addition to a deeper comparison of different statistical techniques, studies lack on two important dimensions: resilience to transient failures of the statistical techniques, and diagnostic abilities. In this work, we address these issues, presenting three main contributions: first, we establish the capability of different statistical learning techniques for forecasting the resource needs of component-based distributed systems, second, we investigate an analysis engine that is more robust to false alarms, introducing a novel algorithm that augments the predictive power of statistical learning methods by combining them with a statistical test to identify trends in resources usage, third, we investigate the applicability of statistical tests for identifying the nature and cause of performance problems in component-based distributed systems.

In this work a stable task space neuro algorithm for set-point control of robot manipulators with uncertain parameters is proposed. A depart from current approaches is the fact that a Wavelet Neural Network with on-line real-time learning seeks to explicitly compensate both the unknown gravity terms and the mismatch between the true and the estimated Jacobian matrix and the fact that it does not need velocity measurements. Linear position filtering is used to estimated the robot joint velocity in the control law and the properties of the Wavelet Neural Network are employed for avoiding velocity measurements in the learning rule. It is shown that all the closed loop signals are uniformly ultimately bounded. Experimental results in a two degrees of freedom robot are presented to evaluate the proposed controller.

Nowadays, more and more data is being produced, processed and stored at a rapid pace. This explosion in the volume of data has clearly affected the field of sports, and more specifically the field of basketball. Data analytics and sports are going hand-in-hand for some time now. Statistical learning refers to a set of tools for modeling and understanding complex datasets. Sports analytics are a collection of relevant, historical, statistics that when properly applied can provide a competitive advantage to a team or individual. Through the collection and analyzation of these data, sports analytics inform players, coaches and other staff in order to facilitate decision making both during and prior to sporting events. In this study, using actual data from the most prestigious European Basketball Tournament, we will analyze with statistical and machine learning techniques, which statistics in the box-score determine the team’s season-long success. Additionally, we will perform a descriptive analysis and visualize the results through graphs, charts and tables. Afterwards, we will employ logit, probit, cauchit and cloglog models aiming to found the key characteristics affecting the final outcome of a game. Finally, we will predict the accuracy and evaluate the performance of those models through different criteria, such as confusion matrix, area under curve (AUC) and random forest algorithm.

ABSTRACTThe detection of regularities in the sensory environment, known as statistical learning, is an important brain function that has been observed in many experimental contexts. In these experiments, statistical learning of patterned sensory stimulation leads to improvements in the speed and/or accuracy with which subsequent stimuli are recognized. That is, statistical learning facilitates the transformation of sensory stimuli into motor responses, but the mechanism by which this occurs is unclear. Statistical learning could improve the efficiency of sensory processing, or it could bias responses toward particular outcomes. The distinction is important, as these different hypotheses imply different functions and different neural substrates for statistical learning. Here we address this problem by studying statistical learning as a decision-making process, which allows us to leverage the extensive computational literature on this topic. Specifically we describe a method for applying the Diffusion Decision Model (DDM) to isolate different sensory and cognitive processes associated with decision-making. The results indicate that statistical learning improves performance on a visual learning task in two distinct ways: by altering the efficiency of sensory processing and by introducing biases in the decision-making process. By fitting the parameters of the DDM to data from individual subjects, we find that the prominence of these two factors differed substantially across the population, and that these differences were predictive of individual performance on the psychophysical task. Overall, these results indicate that different cognitive processes can be recruited by statistical learning, and that the DDM is a powerful framework for detecting these influences.

Robots and virtual agents need to adapt existing and learn novel behavior to function autonomously in our society. Robot learning is often in interaction with or in the vicinity of humans. As a result the learning process needs to be transparent to humans. Reinforcement Learning (RL) has been used successfully for robot task learning. However, this learning process is often not transparent to the users. This results in a lack of understanding of what the robot is trying to do and why. The lack of transparency will directly impact robot learning. The expression of emotion is used by humans and other animals to signal information about the internal state of the individual in a language-independent, and even species-independent way, also during learning and exploration. In this article we argue that simulation and subsequent expression of emotion should be used to make the learning process of robots more transparent. We propose that the TDRL Theory of Emotion gives sufficient structure on how to develop such an emotionally expressive learning robot. Finally, we argue that next to such a generic model of RL-based emotion simulation we need personalized emotion interpretation for robots to better cope with individual expressive differences of users.