Model-Based Learning for Location-to-Channel Mapping

T124. Effect of subthreshold paired associative stimulation during voluntary contraction on different forms of motor learning

Reinforcement learning (RL) models have been influential in characterizing human learning and decision making, but few studies apply them to characterizing human spatial navigation and even fewer systematically compare RL models under different navigation requirements. Because RL can characterize one’s learning strategies quantitatively and in a continuous manner, and one’s consistency of using such strategies, it can provide a novel and important perspective for understanding the marked individual differences in human navigation and disentangle navigation strategies from navigation performance. One-hundred and fourteen participants completed wayfinding tasks in a virtual environment where different phases manipulated navigation requirements. We compared performance of five RL models (3 model-free, 1 model-based and 1 “hybrid”) at fitting navigation behaviors in different phases. Supporting implications from prior literature, the hybrid model provided the best fit regardless of navigation requirements, suggesting the majority of participants rely on a blend of model-free (route-following) and model-based (cognitive mapping) learning in such navigation scenarios. Furthermore, consistent with a key prediction, there was a correlation in the hybrid model between the weight on model-based learning (i.e., navigation strategy) and the navigator’s exploration vs. exploitation tendency (i.e., consistency of using such navigation strategy), which was modulated by navigation task requirements. Together, we not only show how computational findings from RL align with the spatial navigation literature, but also reveal how the relationship between navigation strategy and a person’s consistency using such strategies changes as navigation requirements change.

Evaluating a software design is an important practice of expert software designers. They spend significant time evaluating their solution, by developing an integrated mental model of the software design and the requirements. However, sufficient emphasis has not been given on teaching and learning of evaluation practices in software design courses, and hence, graduating students find it difficult to critically analyse an existing design and improve upon it. In this paper, we describe a model-based learning pedagogy for teaching–learning of software design evaluation. Model-based learning has been extensively used in science education and entails helping students construct, refine, revise, evaluate, and validate scientific models. We argue that modelling practices in software design evaluation are analogous to these practices. We adapted the model-based learning paradigm and operationalised it into a technology-enhanced learning environment (TELE) for fostering software design evaluation skills in computer science undergraduates. We conducted a research study with 22 undergraduate students to explore how the TELE and its features help students effectively evaluate a given software design. Students attempted a pre-test and post-test which asked them to identify defects in the design. We used the content analysis method to identify categories of defects from student responses in the pre-test and post-test. We also analysed student interaction logs and conducted focus group interviews to identify how features in the TELE contributed towards student learning. Findings from the study showed that students’ understanding of evaluation improved, from merely adding new functionalities and requirements, to a process which involved identifying alternate scenarios in the design which violate the given requirements. Students perceived that pedagogical features of the TELE were useful in helping them effectively evaluate software designs. Findings from the study provide evidence for the model-based learning paradigm as an appropriate pedagogy for software design and also opens the space for researchers to investigate model-based learning in other aspects of software design, such as designs of different types and varying complexities.

Aim: This study aims to elevate the modelling skills of Grade 10 students in understanding the circulatory system, using model-based learning, to a level where they can achieve at least a 70% passing score on a relevant unit test.Method: The study involved 23 students and implemented a model-based learning approach. Research tools included a design for model-based learning, assessments of the modelling process, a modelling ability exam, structural interviews, and student diaries. Data analysis was conducted using percentages and averages, and the action research was structured into two iterative rounds.Results: Initial findings from the first cycle revealed an average modelling ability score of 18.21 out of 24 (75.87%), with 14 students surpassing the 70% threshold. The second cycle showed marked improvement, with an average score of 19.94 out of 24, translating to an 83.09% success rate. Notably, all 23 students exceeded the 70% benchmark in this cycle.Conclusion: The implementation of model-based learning significantly enhanced the students' modelling skills in understanding the circulatory system. The method proved effective in not only achieving but surpassing the targeted 70% success threshold, demonstrating its potential as a valuable educational tool in biology.

Objective: The fundamental purpose of education is to cultivate learners’ capacity to think critically as an essential component of human development. Critical thinking aligns with higher-order cognitive processes in Bloom’s taxonomy, particularly analyzing, evaluating, and creating, which are central to achieving Sustainable Development Goals (SDGs), especially SDG 4 on quality and inclusive education. This study aims to examine the effect of Model-Based Learning, operationalized through Problem-Based Learning (PBL), on students’ critical thinking skills in Islamic Studies at SMP Muhammadiyah 1 Kartasura. Theoretical framework: The theoretical framework integrates constructivist learning theory, Bloom’s cognitive taxonomy, and the SDGs-oriented education paradigm, emphasizing learner-centered instruction and cognitive empowerment. Literature review: Prior studies highlight the effectiveness of PBL in promoting critical reasoning; however, empirical evidence within Islamic education contexts linked explicitly to SDGs remains limited. Methods: This research employed an experimental method using a pretest–posttest non-equivalent control group design. The participants consisted of 53 ninth-grade students, with 18 students assigned to the experimental group and 35 students to the control group. Data were collected through structured pre-tests and post-tests designed to measure critical thinking indicators within the cognitive domain of Bloom’s taxonomy. Results: The findings reveal a statistically significant effect of PBL on students’ critical thinking skills in Islamic Studies. The Mann–Whitney test yielded an Asymp. Sig. (2-tailed) value of 0.001, indicating a significant improvement in the experimental group compared to the control group. These results demonstrate that Model-Based Learning contributes meaningfully to developing students’ higher-order thinking skills while supporting the goals of sustainable and quality education. Implications: The implications of this study suggest that integrating SDGs-oriented PBL into Islamic education can enhance cognitive competence, foster critical awareness, and prepare students for complex social challenges. Novelty: The novelty of this research lies in its empirical linkage between Model-Based Learning, Islamic Studies, and SDGs within a junior high school context, offering a replicable framework for sustainable educational practices.

Background It is crucial to support students in better understanding water and sustainability issues because water plays a vital role in maintaining global ecosystems, including human life. A wide range of curricular and instructional supports like those embodied in model-based learning (MBL) are necessary for teachers to engage students in the core epistemic commitments of the Next-Generation Science Standards (NGSS). Purpose The main purpose of this study is to inform theory about students’ early attempts to engage in the complex kinds of sensemaking experiences inherent in the Framework for K-12 Science Education and the NGSS. Sample Data for this study was collected from 74 10th grade students in a high school in the Northwest region of the New England state. Design and Methods An explanatory sequential mixed-method research design was used to examine students’ learning outcomes, and to better understand these outcomes in connection to their experiences engaging in modeling in the MBL curriculum unit. Results The results indicated that students’ model scores, the number of concepts in models, and the coherence and sophistication of models improved between their initial and final models. Additionally, the following patterns emerged related to ways in which students engaged in the practice of modeling: (1) students attempted to directly represent what they observed, (2) they struggled to pictorially express complex patterns or mechanisms, and (3) students experienced difficulties representing models from a diverse range of perspectives. Conclusion The patterns identified across student models, as well as their reports of experiences related to the MBL unit implementation, provided insight into student experiences with models, while also providing meaningful implications for the refinement of the MBL curriculum unit investigated in this research specifically, while informing approaches MBL curricular units aimed at supporting NGSS implementation efforts more generally.

How to improve efficiency of learning is always the key issue for implementation of reinforcement learning. This paper makes use of advantages of both hierarchical learning and model-based learning, so that an improved algorithm, named Bayesian-MAXQ learning, is introduced, in which several modifications are developed to solve the value update of hierarchy, while the possible performance damages brought by prioritized sweeping is reduced to trivial. The simulation results show that, Bayesian-MAXQ learning performs with high efficiency, and it can serve as a good framework for further study on hierarchical model-based learning.

The majority of model-based learned image reconstruction methods in medical imaging have been limited to uniform domains, such as pixelated images. If the underlying model is solved on nonuniform meshes, arising from a finite element method typical for nonlinear inverse problems, interpolation and embeddings are needed. To overcome this, we present a flexible framework to extend model-based learning directly to nonuniform meshes, by interpreting the mesh as a graph and formulating our network architectures using graph convolutional neural networks. This gives rise to the proposed iterative Graph Convolutional Newton-type Method (GCNM), which includes the forward model in the solution of the inverse problem, while all updates are directly computed by the network on the problem specific mesh. We present results for Electrical Impedance Tomography, a severely ill-posed nonlinear inverse problem that is frequently solved via optimization-based methods, where the forward problem is solved by finite element methods. Results for absolute EIT imaging are compared to standard iterative methods as well as a graph residual network. We show that the GCNM has good generalizability to different domain shapes and meshes, out of distribution data as well as experimental data, from purely simulated training data and without transfer training.

The Wisconsin Card Sorting Test (WCST) is considered a gold standard for the assessment of cognitive flexibility. On the WCST, repeating a sorting category following negative feedback is typically treated as indicating reduced cognitive flexibility. Therefore such responses are referred to as ‘perseveration’ errors. Recent research suggests that the propensity for perseveration errors is modulated by response demands: They occur less frequently when their commitment repeats the previously executed response. Here, we propose parallel reinforcement-learning models of card sorting performance, which assume that card sorting performance can be conceptualized as resulting from model-free reinforcement learning at the level of responses that occurs in parallel with model-based reinforcement learning at the categorical level. We compared parallel reinforcement-learning models with purely model-based reinforcement learning, and with the state-of-the-art attentional-updating model. We analyzed data from 375 participants who completed a computerized WCST. Parallel reinforcement-learning models showed best predictive accuracies for the majority of participants. Only parallel reinforcement-learning models accounted for the modulation of perseveration propensity by response demands. In conclusion, parallel reinforcement-learning models provide a new theoretical perspective on card sorting and it offers a suitable framework for discerning individual differences in latent processes that subserve behavioral flexibility.

To behave adaptively, we must learn from the consequences of our actions. Doing so is difficult when the consequences of an action follow a delay. This introduces the problem of temporal credit assignment. When feedback follows a sequence of decisions, how should the individual assign credit to the intermediate actions that comprise the sequence? Research in reinforcement learning provides 2 general solutions to this problem: model-free reinforcement learning and model-based reinforcement learning. In this review, we examine connections between stimulus-response and cognitive learning theories, habitual and goal-directed control, and model-free and model-based reinforcement learning. We then consider a range of problems related to temporal credit assignment. These include second-order conditioning and secondary reinforcers, latent learning and detour behavior, partially observable Markov decision processes, actions with distributed outcomes, and hierarchical learning. We ask whether humans and animals, when faced with these problems, behave in a manner consistent with reinforcement learning techniques. Throughout, we seek to identify neural substrates of model-free and model-based reinforcement learning. The former class of techniques is understood in terms of the neurotransmitter dopamine and its effects in the basal ganglia. The latter is understood in terms of a distributed network of regions including the prefrontal cortex, medial temporal lobes, cerebellum, and basal ganglia. Not only do reinforcement learning techniques have a natural interpretation in terms of human and animal behavior but they also provide a useful framework for understanding neural reward valuation and action selection.

Deep reinforcement learning is the technology of artificial neural networks in the field of decision-making and control. The traditional model-free reinforcement learning algorithm requires a large amount of environment interactive data to iterate the algorithm. This model’s performance also suffers due to low utilization of training data, while the model-based reinforcement learning (MBRL) algorithm improves the efficiency of the data, MBRL locks into low prediction accuracy. Although MBRL can utilize the additional data generated by the dynamic model, a system dynamics model with low prediction accuracy will provide low-quality data and affect the algorithm’s final result. In this paper, based on the A3C (Asynchronous Advantage Actor-Critic) algorithm, an improved model-based deep reinforcement learning algorithm using a learning degree network (MBRL-LDN) is presented. By comparing the differences between the predicted states outputted by the proposed multidynamic model and the original predicted states, the learning degree of the system dynamics model is calculated. The learning degree represents the quality of the data generated by the dynamic model and is used to decide whether to continue to interact with the dynamic model during a particular episode. Thus, low-quality data will be discarded. The superiority of the proposed method is verified by conducting extensive contrast experiments.

Although model-free reinforcement learning methods are illustrated to own abilities to learn a wide range of control and decision-making tasks, it is time-consuming and needs a huge number of samples to learn optimal policy. Therefore, model-based reinforcement learning algorithms are always treated as the most efficient ways to settle robotic control problems. In this work, a model-based reinforcement learning tracking control method with free energy is proposed. The control objective is to make sure that the proposed controller can let a robotic manipulator with parameters uncertainties and disturbance follow the desired trajectory and obtain good tracking performance. The main contribution of this work is to employ the free energy principle to optimize the problem of robotic control. The probability distribution of robotic dynamic model is learnt by a evolving neural network, a free energy based reinforcement learning control method uses the learnt model to find optimal control policy. The experiment results demonstrate that the proposed tracking control method is effective in solving robotic tracking control problems.

The Robot with nonlinear and stochastic dynamic challenges optimal control that relying on an analytical model. Model-free reinforcement learning algorithms have shown their potential in robot learning control without an analytical or statistical dynamic model. However, requiring numerous samples hinders its application. Model-based reinforcement learning that combines dynamic model learning with model predictive control provides promising methods to control the robot with complex dynamics. Robot exploration generates diverse data for dynamic model learning. Model predictive control exploits the approximated model to select an optimal action. There is a dilemma between exploration and exploitation. Uncertainty provides a direction for robot exploring, resulting in better exploration and exploitation trade-off. In this paper, we propose Model Predictive Control with Posterior Sampling (PSMPC) to make the robot learn to control efficiently. Our PSMPC does approximate sampling from the posterior of the dynamic model and applies model predictive control to achieve uncertainty directed exploration. In order to reduce the computational complexity of the resulting controller, we also propose a PSMPC guided policy optimization algorithm. The results of simulation in the high fidelity simulator "MuJoCo" show the effectiveness of our proposed robot learning control scheme.

Nowadays, applications of ultrasonic proximity sensors are limited to a post-processing of the acquired signals with a pipeline of filters and threshold comparators. This article proposes two different and novel processing methodologies, based on machine learning algorithms, that outperform classical approaches. Indeed, noisy signals and presence of thin or soundproofing objects are likely sources of false positive detections that can make traditional approaches useless and unreliable. In order to take advantage of correlations among the data, multiple parallel signals, coming from a cluster of ultrasonic sensors, have been exploited, producing a number of different features that allowed to achieve more accurate and precise predictions for object detection. Firstly, model-based learning as well as instance-based learning systems have been investigated for an independent time correlation analysis of the different signals. Particular attention has been given to the training and testing of the deep fully connected network that showed, since the beginning, more promising results. In the second part, a recurrent neural network, based on long short term memory cells, has been devised. As a result of its intrinsic nature, time correlations between successive samples are not more overlooked, further improving the overall prediction capability of the system. Finally, cutting edge training methodologies and strategies to find the different hyperparameters have been adopted in order to obtain the best results and performance from the available data.

Traditional dynamic models of continuum robots are in general computationally expensive and not suitable for real-time control. Recent approaches using learning-based methods to approximate the dynamic model of continuum robots for control have been promising, although real data hungry—which may cause potential damage to robots and be time consuming—and getting poorer performance when trained with simulation data only. This paper presents a modelbased learning framework for continuum robot closed-loop control that, by combining simulation and real data, shows to require only 100 real data to outperform a real-data-only controller trained using up to 10000 points. The introduced data-efficient framework with three control policies has utilized a Gaussian process regression (GPR) and a recurrent neural network (RNN). Control policy A uses a GPR model and a RNN trained in simulation to optimize control outputs for simulated targets; control policy B retrains the RNN in policy A with data generated from the GPR model to adapt to real robot physics; control policy C utilizes policy A and B to form a hybrid policy. Using a continuum robot with soft spines, we show that our approach provides an efficient framework to bridge the sim-to-real gap in model-based learning for continuum robots.