Did You Forget Your Password?

Categorization of Emergent Processes by Students at Different Levels of Expertise Randi A. Engle (RAEngle@pitt.edu) Michelene T. H. Chi (chi@pitt.edu) Learning Research and Development Center, University of Pittsburgh 3939 O’Hara St., Pittsburgh, PA 15260 USA Introduction Chi and Roscoe (2002) proposed that one reason for the persistence of scientific misconceptions is that students classify scientific phenomena into incorrect ontological categories. In particular, Chi (submitted) has hypothesized that many commonly misunderstood science concepts (like evolution and electric current) are emergent processes in which a macro-level phenomena emerges from complex interactions between entities at a micro-level. Rather than correctly categorizing them this way, students are thought to miscategorize them as non-emergent processes in which there is a more direct relationship between the two levels. To test this hypothesis, we are conducting a study comparing how participants at different levels of expertise categorize science problems across domains, using a card sorting task modeled on Chi, Feltovich & Glaser (1981). The prediction of Chi’s theory is that participants with more expertise will be more likely to distinguish emergent from non-emergent problems while those with less expertise will often conflate them. Experts will also be more likely to refer to the 11 features of the emergent schema (e.g., disjointness, parallelism; see Chi, submitted) in defining their categories. Method Participants Participants consisted of 10 undergraduate and 9 doctoral students from the biology, chemistry, or physics departments at a local university. Undergraduates had all completed 1 st year courses in biology, chemistry, and physics. Eight were single majors in one of these disciplines and two were double majors in biology and chemistry. Doctoral students were in their third year or above. Materials Participants sorted 24 science problems, 8 drawn from each discipline’s 1 st year course. Within each discipline, half were emergent and half were non-emergent processes. Procedure Participants were asked to sort the problems into piles with similar mechanisms for linking the macro and micro levels. In the 1 st sort, they were allowed to make as many piles as they wished. In the 2 nd sort, they were asked to divide the cards into just two piles. In the 3 rd sort, the experimenter sorted the cards into emergent vs. non-emergent processes, and participants were asked to infer the distinction. In all cases, participants were asked to explain the explanatory mechanism each pile represented and why each problem fit. Results As a preliminary measure of the degree to which participants distinguished emergent from non-emergent processes, we calculated a weighted average (by pile size) of the absolute difference between the number of emergent versus non-emergent processes in each pile by the number of cards in the pile. For example, 3 piles—one with 5 emergent & 5 non-emergent problems, a 2 nd with 1 emergent & 7 non-emergent problems, and a 3 rd with 6 emergent & 0 non-emergent problems—would get a score of [10(0/10) + 8(6/8) + 6(6/6)] / 24 = 12/24 = .50. A value of 1 on this score represents perfect separation of emergent versus non-emergent processes while 0 represents perfect 50/50 mixing. Mean separation scores were .58 for doctoral students, .53 for single majors, and .73 for double majors, although none of these differences are statistically reliable. A complicating factor is that participants sometimes put, for example, only emergent problems in a given pile for reasons unrelated to their emergence. Thus, we analyzed participants’ definitions of their categories to determine whether they referred to any of the 11 features of the emergent schema. Doctoral students’ categories referred to more emergent features (1.78) than single majors (0.13; t(9) = 3.08, p < .05). Double majors referred to even more of them (3.50), but more data is needed to see if this is reliable. Discussion Doctoral students and double majors used more features of the emergent schema in sorting science problems than single majors, although they were no more likely than single majors to create piles that distinguished emergent and non- emergent processes. In future work, we will include in our sample more double majors as well as professors to further investigate the nature of expertise in emergent processes. References Chi, M. T. H. (submitted). “Causal” and “Emergent” Explanatory Mechanisms: Potential Schemas for Overcoming Misunderstandings in Science. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152. Chi, M. T. H., & Roscoe, R. (2002). The processes and challenges of conceptual change. In M. Limon & L. Mason (Eds.), Reconsidering conceptual change: Issues in theory and practice. Dordrecht, NL: Kluwer.

Icon-function relationship in toolbar icons

Interacting to reach a shared decision is an omnipresent component of human collaboration. We explored the interaction between dyads of individuals with different levels of expertise. The members of the dyads completed a number line task privately, jointly and privately again. In the joint condition, dyad members shared their private estimates and then negotiated a joint estimate. Both dyad members averaged their private individual estimates to determine joint estimates, thereby showing a strong equality bias. Their performance in the joint condition exceeded the performance of the dyad's best estimator, demonstrating interaction benefit, only when the dyad members had similar levels of expertise and when the averaged dyad performance was sufficiently accurate. At the end of the task, participants rated their level and their partner's level of competence. Participants were accurate in classifying themselves as the expert or the novice within the dyad. Nevertheless, novices tended to overestimate their ability as they admitted to being less competent but only slightly worse than their expert partner. Experts, instead, believed themselves to be more competent but were humble and considered their performance only marginally better than their partner's. Overall, these results have important implications for settings in which people with different levels of expertise interact. (PsycINFO Database Record

Training models in laparoscopic surgery allow the surgical team to practice procedures in a safe environment. We have proposed the use of a 4-task, low-cost inert model to practice critical steps of laparoscopic common bile duct exploration. The performance of 3 groups with different levels of expertise in laparoscopic surgery, novices (A), intermediates (B), and experts (C), was evaluated using a low-cost inert model in the following tasks: (1) intraoperative cholangiography catheter insertion, (2) transcystic exploration, (3) T-tube placement, and (4) choledochoscope management. Kruskal-Wallis and Mann-Whitney tests were used to identify differences among the groups. A total of 14 individuals were evaluated: 5 novices (A), 5 intermediates (B), and 4 experts (C). The results involving intraoperative cholangiography catheter insertion were similar among the 3 groups. As for the other tasks, the expert had better results than the other 2, in which no significant differences occurred. The proposed model is able to discriminate among individuals with different levels of expertise, indicating that the abilities that the model evaluates are relevant in the surgeon's performance in CBD exploration. Construct validity for tasks 2 and 3 was demonstrated. However, task 1 was no capable of distinguishing between groups, and task 4 was not statistically validated.

To meet the rising demand for comprehensive bioinformatics tools in the life sciences, the German Network for Bioinformatics Infrastructure (de.NBI) was established in 2015 as a national bioinformatics consortium offering high quality bioinformatics services, comprehensive training and powerful computing capacities. The de.NBI Cloud is one of the flagship services for the de.NBI &amp; ELIXIR Germany network with more than 4000 registered users, over 1350 projects, of which more than 620 are currently active, and over 1380 publications citing its use. Eight federated cloud centers adhere to a unified governance structure and utilize the centralized project application and management workflow available through the de.NBI Cloud Portal. Single sign-on and user management are provided via LifeScience Login and eduGAIN. PIs from German universities or research institutes may submit project applications, which are reviewed by the Cloud Access Committee and, upon approval, resources are allocated to one of the cloud centers. Regular staff training, mutual internal audits and external ISMS certification (ISO27001 or BSI C5) ensure secure operations of the data centers. The de.NBI Cloud offers various project types for your computing needs, covering different levels of expertise and experience in cloud computing: From the simple and fast start of pre-configured VMs with easy access (SimpleVM), to the use of bioinformatics tools and workflows (Galaxy), to the possibility of running your own infrastructures and services (OpenStack &amp; Kubernetes). We offer pre-configured virtual research environments (VRE), which can be accessed easily when using our SimpleVM platform. They include integrated development environments, data science notebooks and virtual desktop environments. The VREs can be accessed in the web browser or by secure shell (ssh). VREs and images tailored for specific needs can also be used to provide uniform VMs for trainings and workshops. The cluster module makes it easy to set up a SLURM-based grid cluster for resource-intensive computations. Users can also provide recipes for custom VREs. The de.NBI Cloud provides computational resources for a diverse portfolio of projects within Europe (ELIXIR, EOSC, Horizon Europe) and Germany (NFDI projects), ranging from (web)service development and hosting, to execution of memory and CPU-intensive bioinformatics workflows, GPU resources for machine learning, resources for hackathons, workshops and trainings, as well as resources for proof-of-concept development, and secure environments for sensitive data processing. Regular trainings and workshops teach you how to use the cloud infrastructure effectively for your research. The annual Cloud User Meeting offers users the opportunity for networking and to get in touch with the administrators of the cloud locations and the cloud governance. Researchers present their work in the cloud and training courses expand the range of expertise for experienced cloud users, while also providing a comprehensive introduction for beginners. The de.NBI Cloud offers various project types for your computing needs, covering different levels of expertise and experience in cloud computing: From the simple and fast start of pre-configured VMs with easy access (SimpleVM), to the use of bioinformatics tools and workflows (Galaxy), to the possibility of running your own infrastructures and services (OpenStack &amp; Kubernetes). Selection of training course topics: • VM basics: Linux &amp; command line • SimpleVM &amp; Research Environments • BioConda &amp; BioContainers • CWL, Nextflow, Snakemake • Infrastructure as Code: Ansible, Terraform • Grid computing: BiBiGrid / SLURM

To assess the construct validity of the URO Mentor (Simbionix Corp., Cleveland, OH, USA) virtual reality training model for several variables of skills training in cysto-urethroscopy, addressing two research questions: (i) Does training on the URO Mentor significantly improve novices' performance in terms of time, trauma, areas inspected and Global Rating Scale (GRS) score?; (ii) is discrimination between different levels of expertise possible using the URO Mentor? Thirty experts and 50 novices performed seven tasks on the URO Mentor during one training session. The first, fourth and seventh tasks were 'test tasks' to evaluate participants' performance. The simulator recorded procedure time and trauma; a supervisor scored which areas were inspected and gave scores on the GRS. A two-way analysis of variance with repeated-measures test was used to analyse experts' and novices' performances, with P < 0.05 considered to indicate statistical significance. Effect sizes (ES) were calculated to quantify the practical significance of the results; ES of 0.10, 0.30, and 0.50 were considered small, medium and large, respectively. Novices' performances showed a significant improvement with large ES in time (linear trend of learning curve P < 0.001, ES 0.66) and mean GRS score (linear trend P < 0.001, ES 0.84, quadratic trend P = 0.018, ES 0.24). There was a medium improvement for trauma (linear trend P < 0.001, ES 0.40) and a small improvement in areas inspected (linear trend P = 0.032, ES 0.21). That the 95% confidence intervals of the measures on the first task of experts and novices did not coincide indicates that differentiation between experts and novices on the four variables measured can be achieved using the URO Mentor. Training on the URO Mentor appears to result in a medium to large improvement of novices' performances for time, trauma, areas inspected and GRS scores. Moreover, discrimination between different levels of expertise is possible using this simulator.

Kiosk 8R-FB-07 - Quantitative Segmentation Analysis in CMR Across Different Levels of Expertise – FBom Beginners to Experts

Bigliani classification is used for determination of acromial morphology, but poor inter-observer reliability has been reported on conventional radiographs. This study aims to assess inter- and intra-observer reliability using magnetic resonance imaging (MRI). Forty consecutive patients diagnosed with subacromial impingement syndrome were included to study. All subjects underwent standard shoulder MRI scan and acromial shape was evaluated by nine observers of different level of expertise (three attending surgeons, three senior orthopaedic residents and three radiologists). A second set of evaluation was performed in order to assess intra-observer reproducibility. Kappa (κ) coefficient analyses both for interobserver reliability and intra-observer reproducibility were then performed. Overall inter-observer agreement among nine observers was fair (κ=0.323). κ values for all 4 individual types ranged from 0.234 to 0.720 with highest agreement for type 4 and lowest agreement for type 3. Second evaluation did not result with an increase of inter-observer agreement (κ=0.338, fair). The κ coefficients for intra-observer reproducibility of nine observers ranged from 0.496 to 0.867. Overall intra-observer reproducibility was substantial. Comparison of inter- and intra-observer reliability among three groups showed no significant difference (p=0.92 and 0.22, respectively). Results showed that MRI did not show superior reliability compared to conventional radiographs. Moreover, inter- and intra-observer agreement did not differ between observers of different level of expertise. Findings of present study suggest that despite a sophisticated imaging modality like MRI, Bigliani's classification apparently lacks accuracy and additional criteria, or different assessment methods are required to assess acromial morphology for clinical guidance.

Cognitive science has long shown interest in expertise, in part because prediction and control of expert development would have immense practical value. Most studies in this area investigate expertise by comparing experts with novices. The reliance on contrastive samples in studies of human expertise only yields deep insight into development where differences are important throughout skill acquisition. This reliance may be pernicious where the predictive importance of variables is not constant across levels of expertise. Before the development of sophisticated machine learning tools for data mining larger samples, and indeed, before such samples were available, it was difficult to test the implicit assumption of static variable importance in expertise development. To investigate if this reliance may have imposed critical restrictions on the understanding of complex skill development, we adopted an alternative method, the online acquisition of telemetry data from a common daily activity for many: video gaming. Using measures of cognitive-motor, attentional, and perceptual processing extracted from game data from 3360 Real-Time Strategy players at 7 different levels of expertise, we identified 12 variables relevant to expertise. We show that the static variable importance assumption is false - the predictive importance of these variables shifted as the levels of expertise increased - and, at least in our dataset, that a contrastive approach would have been misleading. The finding that variable importance is not static across levels of expertise suggests that large, diverse datasets of sustained cognitive-motor performance are crucial for an understanding of expertise in real-world contexts. We also identify plausible cognitive markers of expertise.

There has been considerable research considering how instrumental practice changes as expertise develops. Much of that research has been relatively small scale and restricted in the range of instrumentalists included. This paper aimed to explore the development of practising strategies and motivation to practise as expertise develops with a large sample of participants at different levels of expertise playing a wide range of different instruments. A total of 3,325 young people ranging in level of expertise from beginner to the level required for entry to higher education conservatoires completed a questionnaire which consisted of a number of statements relating to practising strategies, organization of practice, and motivation to practise with a seven-point rating scale. Data were analyzed in relation to nine levels of expertise. Factor analysis revealed seven factors: adoption of systematic practice strategies; organization of practice; use of recordings for listening and feedback and use of the metronome; use of analytic strategies; adoption of ineffective strategies; concentration; and immediate correction of errors. There were statistically significant linear relationships between grade level and four of the factors but not for organization of practice; use of analytic strategies; and concentration. The findings are discussed in relation to the educational implications.

We present two studies that investigated the adoption of visual/spatial and analytic strategies by individuals at different levels of expertise in the area of organic chemistry, using the Visual Analytic Chemistry Task (VACT). The VACT allows the direct detection of analytic strategy use without drawing inferences about underlying mental processes. The first study examined the psychometric properties of the VACT and revealed a structure consistent with the hypothesis that it consists of two sub-scales: visual/spatial and analytic. The second study investigated the performance of 285 participants with various levels of expertise in organic chemistry on the VACT. The results showed that the adoption of analytic strategies in organic chemistry, and specifically in molecular structure, was difficult and was systematically used only by the more expert participants. The implications of this research for the teaching of chemistry are discussed.

Knowledge acquisition (KA) encompasses working with the expert to model the domain and a suitable problem solving method as preconditions for building a knowledge based system (KBS) and secondly working with the expert to populate the knowledge base. Ripple Down Rules (RDR) focuses on the second of these activities and allows an expert to populate a knowledge base (KB) without any knowledge engineering assistance. It is based on the idea that since the knowledge an expert provides is a justification of his or her judgment given in a specific context, this knowledge should only be used in the same context. Although the approach has been used for large single classification systems, it has the potential problem that the local nature of the knowledge may result in much repeated knowledge in the KB and much repeated knowledge acquisition. The study here attempts to quantitate and compare KB size and performance for systems built by experts with various levels of expertise and also inductively. The study also proposes a novel way of conducting such studies in that the different levels of expertise were achieved by using simulated experts. The conclusion from this study is that experts are likely to produce reasonably compact and efficient knowledge bases using the Ripple-Down Rule approach.

Interobserver agreement in the diagnosis of pulmonary embolism with helical CT

Taekwondo (TKD) is a popular sport among adolescents. This study aims to (a) compare the balance performance between adolescent TKD practitioners at different levels of expertise with nonpractitioners and (b) determine the sensory system(s) that contributed to the balance function in adolescents with and without TKD training. Subjects with >5 years of TKD training (n = 11), <4 years of training (n = 10), and no training (n = 10) participated in this study. The sway velocity, somatosensory, vestibular and visual ratios were recorded during standing on a balance testing system. Both short- and long-term TKD practitioners swayed slower than control subjects when standing on one leg (p = .016 and 0.012, respectively). However, only short-term practitioners have better visual ratio (p = .018) and vestibular ratio (p = .029) than control subjects. There was no significant difference in the somatosensory ratio among the 3 groups. We conclude that adolescents undertaking TKD training may have better balance performance than untrained subjects.

Response suppression and decision-making in association football referees.