Beyond Programming: A Computer-Based Assessment of Computational Thinking Competency

Computational thinking has already been acknowledged as an important learning objective for all students in compulsory education. Our research interests focus on the assessment of fundamental computational thinking competencies, such as collection, organisation and analysis of data, algorithmic thinking and abstraction. For the needs of our research, we implemented the digital platform PhysGramming, which constitutes the backbone of a relevant assessment tool we have built. PhysGramming is developmentally appropriate for first and second grade primary school students and provides the opportunity to create their own digital games, transforming them from passive consumers to active creators of digital technology. The assessment tool has been applied to 450 students in the first and second grade of primary schools on Crete, Greece from February to June 2019. The research was conducted attuned to the ethical guidelines of educational research, within the framework of the physical and natural science courses and evolved around three axes: (a) Checking the validity and the reliability of the results that come out when applying the proposed assessment tool. (b) Designating the levels of computational thinking skills of students at first stages of schooling. (c) Checking the correlation between the levels of computational thinking skills and the comprehension of the lesson's content. In other words, we check whether high learning performance is a necessary and sufficient condition for diagnosing high levels of basic computational thinking skills. In this paper, we shortly describe the assessment tool we propose.

Formative evaluation of immersive virtual reality expedition mini-games to facilitate computational thinking

<span lang="EN-US">Based on the study, computational thinking skills are influenced by science, technology, engineering and mathematics (STEM) skills, and there is a relationship between computational thinking skills and 21st-century skills. However, studies related to STEM attitudes, computational thinking and their impact on 21st-century skills are still very few and limited. The purpose of our study was to examine the impact of STEM attitudes and computational thinking on 21st-century. This research uses a quantitative approach. The participants of this study were students of a vocational school in Bantul Regency, Yogyakarta, Indonesia (N=290). Research data in STEM attitude, computational thinking, and 21st-century skills using a questionnaire. The data were analyzed using structural equation modeling techniques using the Smart PLS application. The results of the study obtained several findings, including: the model proposed in this study was valid; STEM attitude has a positive and significant effect on computational thinking; and computational thinking has a positive and significant effect on 21-st century skills. It can be argued that when STEM attitudes and computational thinking are more positive, 21-st century skills will improve. These findings have implications that curriculum development and STEM learning practices have to develop students’ computational thinking skills and 21st-century skills, especially in vocational schools.</span>

Two studies were conducted to examine the use of grounded embodied pedagogy, construction of Imaginary Worlds (Study 1), and context of instructional materials (Study 2) for developing learners' Computational Thinking (CT) Skills and Concept knowledge during the construction of digital artifacts using Scratch, a block-based programming language. Utilizing a conceptual framework for grounded embodied pedagogy called Instructional Embodiment, learners physically enacted (Direct Embodiment) and mentally simulated (Imagined Embodiment) the actions and events as presented within pre-defined Scripts. Instructional Embodiment utilizes action, perception, and environment to create a dynamic, interactive teaching & learning scenario that builds upon previous research in embodied teaching and learning. The two studies described herein examined the effects of Instructional Embodiment, Imaginary World Construction, and Context on the development of specific Computational Thinking Concepts and Skills. In particular, certain CT Concepts, such as Conditionals, Variables, Thread Synchronization, Collision Detection, & Events, and CT Skills, such as abstraction and pattern recognition, were identified and measured within the learners' individual digital artifacts. Presence and/or frequency of these Concepts and Skills were used to determine the extent of Computational Thinking development. In Study 1, fifty-six sixth- and seventh-grade students participated in a fifteen-session curricular program during the academic school day. This study examined the type of instruction and continuity of Imaginary World Construction on the development of certain CT Skills and Concepts used in a visual novel created in Scratch. Main effects were found for learners who physically embodied the pre-defined instructional materials: embodying the pre-defined Scripts led to the learners using significantly more ‘speech’ Blocks in their projects and more Absolute Positioning Blocks for ‘motion’ than those who did not physically embody the same Scripts. Significant main effects were also found for continuity of Imaginary World Construction: learners who were instructed to continue the premise of the first digital artifact (Instructional Artifact) implemented significantly more computational structures in their second digital artifact (Unique Artifact) than those who were instructed to create a Unique Artifact with a premise of their own design. In Study 2, seventy-eight sixth- and seventh-grade students participated in a seventeen-session curricular program during the academic school day. This study examined the type of instruction and context of instructional materials on the development of CT Skills and Concepts during the construction of a video game using Scratch. Similar to Study 1, findings suggest that physically embodying the actions presented within the pre-defined instructional materials leads to greater implementation of many of these same structures during individual artifact construction. The study also showed that as the pre-defined Scripts become more complex (e.g. single-threaded to multi-threaded), the effect of physical embodiment on the development of CT Skills and complex CT Concept structures becomes less pronounced. Findings from this study also suggest that Context has a significant effect on identifying & implementing the CT Skill pattern recognition: learning CT Concepts from an Unfamiliar Context had a significant positive effect on the implementation of both Broadcast/Receive couplings and Conditional Logic & Operator patterns. In sum, the findings suggest that the type of instruction, the continuity of the Imaginary World being constructed, and the context of the instructional materials all play a significant role in the learners' ability to develop certain Computational Thinking Skills and Concept knowledge. The findings also suggest that a physically embodied approach to teaching abstract concepts that is grounded in an unfamiliar context is the most effective way to integrate a grounded embodied approach to pedagogy within a formal instructional setting.

This research aims to examine the effect of Mind Games activities on problem-solving skills and computational thinking skills of 5th grade students. Sequential explanatory design, one of the mixed research methods, was used in the research. The research was carried out in a quasi-experimental design pretest-posttest without a control group with 17 5th grade students in a secondary school in Adıyaman. The research was carried out with an experimental period of 6 weeks. Before the experimental study, the problem-solving skills and computational thinking skills scales were applied to the students as a pre-test. Kapsul game activity was performed in the first week of the experimental study. ABC Baglama activity was held during the week. Sudoku activity was held in the 3rd week. In the 4th week, Three Stone game activities were held. Kendoku game activity was held in the 5th week. In the 6th week, Kakuro game activity was held. Statistical values in the study were calculated using the SPSS program. To collect quantitative data, a pre-test of problem-solving skills and computational thinking skills was applied to the students before the experimental process. After the experimental process, problem-solving skills and computational thinking skills scales were applied as a posttest. The applied pre-test and post-test results were analyzed with the paired sample t-test. Additionally, to collect qualitative data to strengthen the research, an interview form was applied to obtain the opinions of the students about the experimental process. In line with the findings obtained because of the experimental study, it can be said that the mind game activities have a positive effect on the students' computational thinking and problem-solving skills, and the lesson turns into more fun.

In recognition of the integration of Information and Communications Technology (ICTs) along with Artificial Intelligence (AI) into education, several educational organizations have recommended cultivating 21st-century digital skills as part of every educational curriculum. The computational thinking skill, which is at the core of 21st-century digital skills, has gained research in several fields, such as physics and psychology (Barkela et al., 2024). However, applied linguistics and computer-assisted language learning (CALL) have overlooked this crucial skill, which might stem from the lack of language teachers’ computational thinking competency. For this reason, in the present study the researchers developed a new theoretical framework, and it’s corresponding scale specifically designed for applied linguistics and CALL, namely the Language Teachers’ Computational Thinking Competency in Computer Assisted Language Learning (LTCCTCALL). Using deductive and inductive methods, the researchers developed the items, followed by a validation process that used the Rasch-Andrich rating scale model (RSM) to assess the scale’s item difficulty and validity using Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). LTCCTCALL was validated with five factor structures that included 15 items in the Iranian EFL context, involving 273 Iranian in-service language teachers. Based on this result, the study introduced a new theoretical framework and its scale to the fields of CALL, applied linguistics, and the Common European Framework of Reference for Languages (CEFR) to lead the community’s continued growth and to prevent the community from falling behind other fields that incorporate computational thinking skills into our classrooms to have problem-solving approaches with CALL, AI, and chatbots in language class.

This research aims to create a tool for assessing students' computational thinking (CT) skills in relation to biology concepts. The study followed a research and development approach, utilizing a modified 4D design model, with the dissemination phase excluded. Various data collection techniques were employed, including observations, literature reviews, documentation, surveys, interviews, and tests. The findings showed that the developed instrument was valid, having been evaluated by three experts. The material expert rated it at 95%, the assessment and evaluation expert at 88%, and the linguist at 90%. In terms of readability, the instrument scored 75%, indicating a "good" rating. The overall validity of the instrument was found to be 0.70, placing it in the high validity range, while its reliability score was 0.82, reflecting a very high level of dependability. The assessment of students' computational thinking skills in biology revealed that 25% of students scored in the low range, 28% in the medium range, and 57% in the high range. From these findings, it can be concluded that the CT test tool is both valid and appropriate for evaluating students' computational thinking skills.

Computational thinking skill is an important skill individuals should acquire to meet the requirements of the digital age. The aim of the study is to predict the computational thinking skills of middle school students through ANFIS approach, which is an adaptive neural network-based fuzzy logic. Students' computational thinking skill scores were predicted by creating a model based on grade level and academic achievement variables. Grade level and academic achievement served as the model's input variables, and computational thinking skill scores served as the model's output variable. Data were collected using personal information form and computational thinking scale. A comparison was made between students' real and artificial computational thinking skill scores using statistical methods. In the study, a strong and favorable association between the artificial scores produced using the ANFIS technique and the actual scores was discovered. Furthermore, there was no statistically significant difference between the real and artificial scores for computational thinking skills. These results indicate that the ANFIS approach is a suitable alternative analysis method for predicting students' computational thinking skills. The study provides a good example in the field of education where artificial intelligence can be used to predict students' educational characteristics.

Problem-based learning integrated with Digital Argumentation (PBL-DA) is a learning strategy for optimizing innovative learning in the digital era. This research aimed to investigate whether the application of PBL-DA can foster the Computational Thinking (CT) and Higher Order Thinking Skills (HOTs). A quasi-experimental design measured three aspects: skill (decomposition, algorithm design, evaluation), attitude (confidence, communication, flexibility), and approach (tinkering, creating, collaborating). The students' HOTs were measured through eight aspects: critical thinking, argumentation, problem-solving, problem-identifying, understanding concepts, analysing, making decisions, and creative thinking. The students' CT and HOTs scores of control and experimental classes were analyzed using Hotelling’s T² test and Tukey’s post hoc test. The Hotelling’s T² test revealed a significant difference between the experimental and control classes for both CT and HOTs (T² = 0.340, p < 0.001 for CT; T² = 0.718, p < 0.001 for HOTs). Tukey’s test further showed that PBL-DA significantly impacted the CT skill and attitude aspects (p < 0.01), while the approach aspect was not significant (p > 0.05). For HOTs, critical thinking, argumentation, problem-identifying and analyzing were significantly improved (p < 0.01), but problem-solving, understanding concepts, making decisions, and creative thinking showed no significant improvement (p > 0.05). Pearson’s correlation analysis indicated a strong positive correlation (r = 0.651, p < 0.001) between students' CT and HOTs skills. These findings provide evidence of the effectiveness of the PBL-DA model in improving students' CT and HOTs, demonstrating its potential for fostering critical and higher-order thinking skills in the digital era.

This study attempts to enhance high school learners' computational thinking and mathematical resilience skills. This study employs quantitative approaches with a quasi-experimental research design, and the type of research is Non-Equivalent Group Design. The tools used in this study were computational thinking skills tests and mathematical resilience questionnaires. From the results of the survey conducted, it is known that: 1) the Project Learning model accompanied by GeoGebra significantly improves computational thinking skills, and Project-Based Learning model has moderate improvement; 2) computational thinking skills of high school students who were taught through Project-Based Learning model aided by GeoGebra improved significantly compared to those taught with the Project-Based Learning model; 3) Project-Based Learning model aided by GeoGebra has a positive effect on the mathematical resilience of high school students, but only to a moderate extent.

This study aims to provide an overview of the computational mathematical thinking abilities of male and female students in solving mathematical problems and the aspects (abstraction, decomposition, algorithmic thinking, and generalization) that influence these differences. This study uses a qualitative method with a case study approach in a junior high school in Jakarta for the academic year 2023/2024. The participants in this study are 36 students. Data were collected through written examinations and interviews and analyzed using the triangulation technique. The results of the study showed different patterns of computational thinking skills between male and female students. Male students excel in decomposition and algorithmic thinking but are weak in abstraction and generalization. Female students excel in abstraction and decomposition but are weak in algorithmic thinking and generalization. Both male and female students master the context of decomposition for computational thinking skills but lack mastery in the context of generalization. In conclusion, gender differences influence specific aspects of computational thinking skills, indicating the need for tailored educational strategies to address these differences.

The graphics calculator (GC) is a hand-held technology. With it, students visualize and understand concepts in mathematics and science better. It also allows them to make real-world connections, binding together advanced graphing functionality and conceptual science, technology, engineering, arts and mathematics (STEAM). As they gain a deeper understanding of abstract concepts, they acquire creative and critical thinking and problem- solving skills they need to reach greater academic successes. In learning, GC plays several roles in increasing students’ computational thinking and procedural skills. Despite being a learning tool that satisfies science and mathematics curriculum needs, the calculator often has been used as an add-on to traditional teaching methodology rather than central and in more integrated ways. This study looks into the potential of developing computational thinking skills using GC technology. The data of the study was obtained from secondary sources through a systematic review method. The study showed that GC seems to be effective tools to practice STEAM education as well as to develop students’ computational thinking through analysing and decompose real problem as well as engaging in the abstraction process and developing an algorithm using coding. The findings provides justifications on the impact of GC in developing computational thinking skills thereby maximizing the pedagogical benefits to students.

Computational thinking has become a crucial skill for students because it gives them the tools to solve complicated issues using organized methods. Implementing Problem-based learning and Design-based learning in science learning has significantly enhanced students' skills. This research aims to ascertain which of the two models is better for facilitating computational thinking skills among middle school students in science learning. The study used a sample of 69 middle school students selected through random sampling. This research employs a quasi-experimental, multi-group, post-test-only design. The instrument utilized in this research is a computational thinking skills test comprising five questions. The data were analyzed using one-way ANOVA to test the hypotheses. Tukey test results indicate that the mean difference score between the PBL and DBL groups is 1.2609, with a significance value greater than 0.05. The study confirms that Problem-based learning is more effective than Design-based learning in facilitating students' computational thinking skills. However, the difference between the two is not particularly noteworthy. PBL and DBL represent viable pedagogical approaches that can enhance middle school students' computational thinking skills in science learning. Keywords: problem-based learning, design-based learning, computational thinking skill. DOI: http://dx.doi.org/10.23960/jpmipa/v25i3.pp1214-1223

The study examined the effect of teaching text-based programming with a physical programming tool on secondary school students’ computational thinking skills and self-efficacy perceptions. The study was conducted according to a sequential explanatory design as a mixed method research. The study group consisted of 85 secondary school students. Within the scope of the study, a physical programming tool called Micro:bit was used to teach Python programming for a period of 6 weeks. Data were collected using the Self-Efficacy Perception Scale for Computational Thinking Skill, Bebras: International Challenge on Informatics and Computational Thinking Tasks, tests focused on programming tool, concepts, and processes, and through semi-structured interview questioning. According to the findings obtained from pretests and posttests, a significant and positive difference was found in the students’ computational thinking skills and self-efficacy perceptions towards computational thinking skill. As a result of having received instruction in programming, the students satisfactorily learnt the required programming concepts and processes. Through learning Python programming with a physical programming tool, the students not only gained the skills required to write appropriate syntax, and to test and debug code, but they also learnt programming concepts such as variables, conditional expressions, loops, and functions.

The PISA data results show that computational thinking abilities are still lacking. Computational thinking ability is influenced by gender. This research aims to analyze patterns of computational thinking skills of prospective elementary school teachers based on gender at 8 universities in Indonesia. In this research, the components of computational thinking skills analyzed are abstraction, algorithmic, decomposition, and pattern recognition. This research is a mix method research with research subjects as many as 234 prospective elementary school teachers at 8 higher educational institutions. The instruments used were test and interviews. The data analysis technique used is a quantitative data analysis technique using SEM PLS and for qualitative data analysis using miles and Huberman. The research results show that computational thinking skills are still low on the decomposition and pattern recognition components. Based on the SEM PLS test results, it shows that computational thinking abilities are related to gender. In general, the computational thinking ability of female students is slightly higher in all sub-indicators than men and there are differences in the pattern of computational thinking ability between male and female elementary school teacher prospective. The ability of prospective female elementary school teachers to answer in more detail and more structured, while the answers of male prospective teachers are shorter and less comprehensive. Recommendations for developing computational thinking skills by developing problem-based learning, contextual project-based learning and STEAM based learning.