A Pedagogical Framework for Integrating Generative AI into Interactive Learning Environments

Artificial intelligence (AI), as a smart and connected technology, has significantly expanded the educational landscape.As more educators and learners begin to rely on GAI to assist with tasks such as instructional design and information generation, its potential to support problem-solving has gained increasing attention. However, the extent to which GAI contributes to the development of higher-order thinking skills-such as creative thinking, critical thinking, and metacognitive awareness-and how these thinking processes interact to influence problem-solving ability remains underexplored. A more comprehensive understanding of this relationship is needed to guide the effective integration of GAI in educational practice. The study encompassed 473 pre-service teachers from three distinct higher education institutions, with specialties in science, computer science, and mathematics, and included a four-week generative AI-supported instructional design training program. And a semistructured interview comprising open-ended questions was administered to 50 pre-service teachers within the experimental group to present their views on generative AI-assisted teaching. Assessments were conducted before and after this program using a higher-order thinking skills survey. The relationship between thinking and problem-solving ability development of pre-service teachers using generative artificial intelligence was analyzed with a moderated model. The study encompassed 473 pre-service teachers from three distinct higher education institutions, with specialties in science, computer science, and mathematics, and included a four-week generative AI-supported instructional design training program. And a semistructured interview comprising open-ended questions was administered to 50 pre-service teachers within the experimental group to present their views on generative AI-assisted teaching. Assessments were conducted before and after this program using a higher-order thinking skills survey. The relationship between thinking and problem-solving ability development of pre-service teachers using generative artificial intelligence was analyzed with a moderated model. This study confirms that training pre-service teachers using generative AI to foster creative thinking can elevate their critical thinking, thereby impacting their problem-solving abilities. Moreover, metacognitive thinking amplifies the impact of creative thinking on critical thinking, resulting in a moderated mediation effect.

Aim/Purpose: The study investigates the factors influencing the acceptance and utilisation of large language models (LLMs) (predictor variables of LLM usage), such as ChatGPT, in Learning design by instructional designers and university-teaching academics from various countries. Background: Large language models (LLMs) have exploded onto the scene, transforming the landscape of learning design. Instructional designers and university teaching academics have been overburdened with content creation for their teaching programmes, and the arrival of LLM models will help in this regard by developing more interactive content that drives student engagement and, in turn, contributes to student success. Since LLMs are a relatively new phenomenon, little is known about the factors influencing their acceptance in learning design; therefore, this research is needed, as learning design principles are the bedrock of student engagement and success. Methodology: A cross-sectional correlational quantitative study was employed. Data was collected using an online questionnaire posted on social media, including LinkedIn, from 203 instructional designers and university teaching academics. Purposive and snowball sampling methods were used to target instructional designers and university teaching academics at colleges and universities worldwide. Participants were asked to share the survey link with fellow instructional designers and university-teaching academics in their communities. The factor structure of the data was determined using exploratory factor analysis. Nonetheless, the factor structure derived from the LLMs did not entirely reflect the original configuration of the Unified Theory of Acceptance and Use of Technology (UTAUT3), as certain predictors appeared to coalesce, indicating LLMs’ unique nature in learning design. Confirmatory factor analysis was used to verify the fit of the data on the measurement model. First-order and second-order structural modelling were used to identify the structural relationships among the variables. Contribution: The study determines significant factors for the acceptance of LLMs by instructional designers and academic teaching staff in learning design, enabling possible opportunities for best practices in the field through interventions to optimize LLM usage. The study applies the technology acceptance model to the emerging LLM technology and extends the technology acceptance model by adding the trust construct as a predictor variable. Findings: The structural analysis results indicated that the ingrained LLM practices, LLM peer-driven expectations, innovative propensity towards LLM adoption, reliability and provider trust in LLMs, and ease of use and support influenced perceived LLM benefits and usage, but community standards and infrastructure had no influence. The second-order structural equation modelling indicated that perceived LLM benefits and usage and ingrained LLM habits contributed most to the learning design. Recommendations for Practitioners: Teaching academics and instructional designers must use LLMs in designing content, assessments, and interactive learning activities, and attend LLM training workshops on prompting and best practices in integrating LLMs into learning and teaching to see their benefits; hence, regular use of LLMs will then lead to trust and innovation in LLMs usage, enhancing learning design and improving student learning outcomes. Recommendation for Researchers: Researchers must use mixed methods approaches to have a deeper understanding of the factors influencing LLMs. Since habit and perceived LLM benefits and usage contributed the most variance to learning design, researchers must investigate strategies that optimise these factors in learning design, such as effective intervention strategies that can help form positive LLM habits. In addition, the findings provide researchers with a starting point for future research. Further researchers must investigate interventions that optimise the influence of personal innovativeness and trust that contributed the least variance to learning design, hence unlocking the potential of LLMs in learning design through innovation, responsible, and ethical use. Impact on Society: The use of LLMs in learning design has a high possibility of transforming education, specifically the learning design landscape. Using LLMs will free up more time for teaching academics and instructional designers so that they spend more time on higher-order thinking skill demands. Consequently, the students will be exposed to more engaging and interactive content, resulting in improved learning outcomes. Future Research: Future research must include context-derived external variables in technology acceptance models, such as levels of prompting competencies, to provide a deeper understanding of LLMs. In addition, future research must be based on the application and impact of LLMs on student engagement and success, and their attainment of 21st-century skills.

Contextual Teaching and Learning (CTL), a learning approach which connects learning materials with real life applications within the context of students’ lives, encourages students to find the meaning of what they have learned on their own and the lecturer only act as a guide or facilitator, so the students are motivated to think critically. This study aims to describe the implementation of CTL approach to enhance students’ higher order thinking skill (HOTS) in writing ability of STKIP PGRI Sidoarjo, which has been found in preliminary observations to be lacking in this aspect. This quantitative research uses a quasi-experimental design through a pretest-posttest pattern. The research sample was 22 students of 2021 A class (Experimental Class) and 23 students of 2021 B class (Control Class). The data were collected through observation, test, and questionnaires. The higher order thinking skill includes: understanding problems (24,20%), making assumptions (28,17%), explaining in depth (25,13%), solving problems (23,25 %), and making conclusions (20,63%). From the table of t-test results, the results (α) = 0.004. It can be stated that (α) < 0.05, so it can be concluded that there is a significant difference in the students' higher order thinking skill in writing ability by using contextual teaching and learning approach. The findings revealed that the contextual teaching and learning approach was successful to enhance students’ higher order thinking skill in writing ability. Moreover, it is recommended that CTL and higher order thinking skill have significant influence toward writing ability.

This study aims to investigate the analysis of the implementation of multi techniques based learning media in improving the elementary school students’ higher order thinking skill in solving exponential problem. This is caused by the lesson plans cannot motivate students to improve higher order thinking skills. The method used in this study is a mixed method which is a combination of quantitative and qualitative methods. The research subjects were the fifth grade students of elementary school consisting of 25 students of experimental class and 24 students of the control class. The instruments of this research are an interview, a questionnaire, observation, and documentation. Quantitative method is applied to analyze the difference of student achievement result among two classes, while the qualitative method is applied to analyze the students' higher order thinking skills. The results show that there are significant differences between the two classes that applied multi techniques based learning and conventional learning. The statistical result indicates that the (2-tailed) significance of the independent sample t-test in the pre-test was 0.000 or α ≤ 0.05. It implies that the implementation of multi techniques media affects the students' higher order thinking skill in solving exponential problem.

Concept attainment is a method of learning a new concept by contrasting and comparing certain aspects of the concept. In order to effectively use higher order thinking skills to improve the performance, there was a dire need for teachers to re-evaluate their teaching strategies and lay more responsibility on prospective teachers to learn and understand concepts of research. The researcher studied the effect of concept attainment model (CAM) based teaching on prospective teachers’ higher order thinking skills. The objective of the study was to examine the effect of concept attainment model (CAM) based teaching on prospective teachers’ higher order thinking skills. The investigation was quantitative in nature. The researcher used a quasi-experimental approach with non-equivalent control groups design. The subject of this study were all prospective teachers of Master of Education, two year programme. There were 30 participants in each intact group. The researcher first developed a table of specification then multiple choice questions (MCQs) test based on higher order thinking. There were 30 items present in the achievement test. After data collection from 16 weeks of intervention, the researcher used the paired sample t-test and independent samples t-test to examine the data gathered from pre-tests and post-tests pertaining to CAM and conventional methods. The results indicated that the achievement level of higher order thinking skills of the prospective teachers who received Concept Attainment Model (CAM) based teaching was higher than those who received conventional instruction methods.

Abstrak: Salah satu tujuan pendidikan khususnya di Perguruan Tinggi adalah agar mahasiswa memiliki kemampuan berpikir kreatif dan kritis yang dikenal dengan istilah HOTS (Higher Order of Thinking Skills). Tujuan yang ingin dicapai dalam penelitian ini adalah untuk mengetahui peningkatan HOTS (Higher Order of Thinking Skills) mahasiswa pada mata kuliah Pengembangan Materi Ajar Bahasa dan Sastra Indonesia Universitas Muhammadiyah Bulukumba. Penelitian ini menggunakan jenis penelitian tindakan kelas dengan objek penelitian adalah mahasiswa Semester 5 (INA 17 A) Program Studi Pendidikan Bahasa Indonesia yang berjumlah 32 orang. Teknik pengumpulan data berupa hasil belajar dan aktivitas mahasiswa dalam perkuliahan. Hasil belajar dengan menggunakan tes dan aktivitas mahasiswa berupa lembar observasi. Analisis data dilakukan dengan mendeskripsikan hasil pengamatan dan analisis statistik sederhana untuk menghitung hasil tes mahasiswa. Hasil tes pada siklus I menunjukkan bahwa 31% (10 orang) mahasiswa yang memeroleh ketuntasan sedangkan 69% (22 orang) mahasiswa yang belum mencapai ketuntasan. Berdasarkan hal tersebut, penelitian dilanjutkan ke siklus kedua. Hasil tes pada siklus II menunjukkan peningkatan yang signifikan dimana sebesar 87% (28 orang) mahasiswa yang berhasil mencapai ketuntasan belajar sedangkan 13% (4 orang) mahasiswa belum tuntas. Dengan demikian, dapat disimpulkan pemanfaatan teknik SCAMPER terbukti efektif dalam meningkatkan HOTS (Higher Order of Thinking Mahasiswa) mahasiswa pada mata kuliah Pengembangan Materi Ajar Bahasa dan Sastra Indonesia. Abstract: One of the education goals especially in higher education is that students have creative and critical thinking skills known as higher order thinking skills or hots. The aim of this research is to find out the improvement of the students’ higher order thinking skills in the subject of Teaching Materials Development of Indonesian Language and Literature at Universitas Muhammadiyah Bulukumba. This study used a classroom action research with research objects that consist of 32 students in the 5th semester students of class 17 A from Indonesian Language Education Department. Techniques of collecting data used are students’ learning outcomes and students’ activities during lectures. Students’ learning outcomes were measured with tests and students’ activities were taken by using observation sheets. Data analysis was carried out by describing the results of observations and simple statistical analysis to calculate students’ test results. The test results in the first cycle show that 31 percent of students or 10 students can reach the learning completeness criteria, while 69 percent of students or 22 students cannot reach the completeness criteria of learning. Based on this research result, the actions were continoud to the second cycle. The second cycle test results show a significant increase in the students’ higher order thinking skills in that 87 percent of students or 28 students are able to achieve the learning completeness criteria and 13 percent of students or 4 students are not able to gain the completeness. Thus, it can be concluded that the SCAMPER technique proves effective in increasing the students’ higher order thinking skills in the subject of Teaching Materials Development of Indonesian Language and Literature.

Developing Higher Order Thinking Skills and Team Commitment via Group Problem Solving: A Bridge to the Real World

Higher order thinking skills are skills needed in studying chemistry. Student higher order thinking skills are currently still relatively low, so appropriate learning models are needed to overcome them. This research aimed at finding out the effect of Discovery Learning (DL) model toward student higher order thinking skills on Buffer Solution lesson. This research was conducted to the eleventh-grade students of Chemistry class at the second semester in the Academic Year of 2023/2024 at State Senior High School 2 Tambang. Quantitative method was used in this research with pretest-posttest non-equivalent control group design. The samples consisted of two classes selected by using cluster random sampling technique. The techniques of analyzing data were t-test to find out whether there was an effect of Discovery Learning model toward student higher order thinking skills and determination coefficient test to find out the effect. Based on the research findings, (1) t-test showed that the significant score 0.002 was lower than 0.05, it meant that there was a significant difference between experiment and control groups, the experiment group higher order thinking skills posttest mean was 80.21, and the control group was 72.64, so H0 was rejected and Ha was accepted; (2) determination coefficient test showed that the score of r2 was 0.130, so it could be concluded that DL learning model affected student higher order thinking skills on Buffer Solution lesson 13%.

This study dives into how Generative Artificial Intelligence (GenAI) can boost higher-order thinking skills among vocational college students in Guangxi, China. The researchers used a quasi-experimental mixed-methods approach, involving 70 students from information technology and engineering programs. They split the students into two groups: one that experienced GenAI-enhanced interactive teaching and another that followed traditional instruction. The experimental group utilized GenAI in specific roles like “virtual debater” and “decision consultant,” which helped them engage more actively in debates, case analyses, and reflective decision-making. The quantitative results from pre- and post-tests showed a notable improvement in critical thinking, logical reasoning, and reflective skills for the students in the experimental group. Meanwhile, qualitative data gathered from observations, student journals, and interviews highlighted an increase in metacognitive awareness, deeper engagement with problem-solving tasks, and a positive view of AI as a helpful partner in learning. However, there were also concerns about students becoming too reliant on AI tools and the necessity for guided facilitation. The findings indicate that when implemented thoughtfully, GenAI tools can significantly enhance higher-order thinking in vocational education, especially when paired with teaching strategies that promote active learning and reflection. This study underscores the importance of integrating AI ethically and calls for customized instructional designs to ensure fair access and ongoing cognitive growth. These insights add to the ongoing conversation about human-machine collaboration in education and provide valuable guidance for future curriculum innovations in tech-driven learning environments.

Please Many studies have described meta-analysis on the overall effect of e-learning influence in learning. However, there has been no meta-analysis of the overall effect of STEM-based e-learning on Higher Order Thinking Skills (HOTS). The purpose of this study investigates the effect of STEM-based e-learning on students' critical thinking skills in Indonesia. This type of research is a Systematic Literature Review (SL) and meta-analysis. Data sources came from 12 national and international journals published in 2018-2023. The process of searching for data sources through Google Scholar, ScienceDirect, Wiley, Eric, and Springer. Data analysis used the Hedge formula to calculate effect size with the help of the JSAP application. The findings show that the effect size value (ES = 0.84) is very high. This explains that STEM-based e-learning provides a very high positive impact and is effective in improving students' Higher Order Thinking Skills (HOTS). Furthermore, HOTS characteristics do not cause heterogeneity in the results of STEM-based e-learning research on students' Higher Order Thinking Skills. The findings suggest that educators should choose STEM-based e-learning as one of the learning models to improve students' Higher Order Thinking skills in Indonesia

Aim/Purpose: This study aims to develop a theoretical framework for enhancing students’ higher-order thinking skills (HOTS) by integrating massive open online courses (MOOCs) with gamification elements. Background: There is a growing demand to develop students’ innovative thinking abilities through MOOCs, focusing on higher-order thinking skills (HOTS), which are essential for 21st-century challenges. While gamification has shown potential in enhancing HOTS, its integration within MOOCs to improve these skills remains underexplored. Enhancing students’ HOTS through MOOCs combined with gamification is crucial for developing advanced skills like analysis, evaluation, and creativity. Therefore, there is an urgent need for a robust theoretical framework that effectively merges MOOCs and gamification to enhance students’ HOTS. Methodology: This research used a qualitative research approach employing critical analysis techniques. The research procedures were guided by the SALSA framework. A total of 19 articles from the SCOPUS and Google Scholar databases were selected based on specific criteria: articles published between 2013-2023, articles with keywords such as MOOCs, gamification, higher-order thinking, or engagement, and articles written in English. Thematic analysis was conducted to identify common themes in the selected articles. The proposed framework was developed by drawing upon well-established theories in the fields of educational technology, online learning, collaborative learning, connectivism, student engagement, and Bloom’s taxonomy. Contribution: This study not only synthesizes existing research on MOOCs but also presents a holistic and integrated framework for leveraging learning theories, gamification elements, student engagement dimensions, and HOTS to enhance the effectiveness of MOOC-based education. The proposed framework aims to provide researchers and educators with a comprehensive model for integrating gamification elements into MOOCs to enhance students’ higher-order thinking skills. By utilizing this framework, educators can design more engaging and effective online courses, while researchers can further investigate the impact of gamification on learning outcomes and student engagement. Findings: This study proposes a framework that integrates three main components: connectivism, online collaborative learning, and gamification principles. Implementing these components in the MOOC learning environment aims to enhance digital higher-order thinking as proposed by Churches and improve students' feelings and perceptions towards MOOC learning. Recommendations for Practitioners: By recognizing the unique challenges of maintaining students’ attention in the context of MOOC learning, practitioners can incorporate gamification elements into MOOC learning environments to enhance students’ HOTS. Recommendation for Researchers: Researchers can further explore the understanding and measurement of the dynamics of interactions and engagement within MOOCs. Additionally, they should aim to identify which gamification elements effectively capture students’ attention and contribute to their overall engagement. Impact on Society: By focusing on HOTS, especially through gamification, society can anticipate a generation of individuals with improved critical thinking, problem-solving, and innovative capabilities. Furthermore, the implementation of connectivism in MOOCs can promote a global exchange of knowledge, resulting in diverse perspectives and a shared pool of information. This, in turn, will contribute to a more interconnected and collaborative world to address complex challenges. Future Research: The future direction of research in MOOC learning contexts should prioritize guaranteeing and fostering student engagement. It should also involve exploring the potential of gamification within MOOCs and refining instructional designs to specifically enhance higher-order thinking skills. By addressing these critical aspects, researchers can contribute to the ongoing evolution of online education and ensure its effectiveness and relevance in the ever-changing landscape of digital learning.

The aim of this study was to examine the application of artificial intelligence (AI) and generative AI (GAI) in dental education through a comprehensive bibliometric analysis, with the goal of identifying potential future research areas in this field. Data were obtained from the Web of Science and Scopus databases, with no date restrictions. The inclusion criteria encompassed research articles, reviews and proceedings papers focused on the use of AI in dental education. The Bibliometrix R package was used for data analysis. A total of 54 documents were analysed, revealing a significant increase in publications from 2021 to 2024. The Journal of Dental Education was identified as the most prolific source, with the United States leading in terms of contributions. The findings showed a growing interest in AI's potential to transform dental education, with significant contributions from a diverse set of global authors and institutions. Several key themes were identified, including 'large language models', 'chatbots' and 'clinical decision support systems'. The citation analysis highlighted influential papers, including those by Thurzo etal. (2023) and Yüzbaşıoğlu (2021). There is a growing interest in the use of AI and GAI in dental education. However, further research, particularly experimental studies, is essential to fully understand their impact on educational outcomes in dentistry. Key areas for exploration include the application of AI for personalised learning, the integration of chatbots as educational tools to support students, and the use of AI for training and assessing practical skills.

<p>Higher-order thinking skills are 21st-century competencies that must be mastered by students. Higher-order thinking skills can be seen in student learning outcomes. Students' higher-order thinking skills can be improved by choosing the right learning media, one of which is by using animated video media. The purpose of this study was to measure the effectiveness of animated videos to improve the high-order thinking skills of SMA Negeri 1 Silat Hulu students. The method used in this research is the experimental method. The form of study used a quasi-experimental design with a nonequivalent control group design. The research subjects were all students of class XI IPA. The instrument used is a question of pretest-posttest. Data analysis used a t-test and N-Gain Score. The conclusions of this study are: (1) there is a significant difference between the students' higher-order thinking skills in the experimental group and the control group. This is evidenced by the independent sample t-test which obtained a significance level of 0.000 <0.05. (2) animation video media is effective in improving students' higher-order thinking skills as seen from the results of the calculation of the N-gain score of 0.61 with a moderate interpretation.</p>

This study aims to determine efforts to improve students' HOTS abilities through the Problem Based Learning learning model in social science subject, based on the learning carried out in the previous cycle, there are still a lot of students who have not reached the minimum completeness criteria when answering questions at the HOTS cognitive level. The solution to overcome these problems is using the Problem Based Learning (PBL) learning model. In carrying out this classroom action research, the researchers focused on research that aimed to determine whether there was a change in students' higher-order thinking skills through the application of problem-based learning models, and to find out how much the students' higher-order thinking skills increased using problem-based learning models. This classroom action research was conducted in two cycles of activity with each cycle consisting of two learning meetings. The data collection techniques used in this research are observation and test techniques. Based on the results of the classroom action research that has been carried out, it is concluded that the use of problem-based learning models can be used as an effort to improve participants' Higher Order Thinking Skills (HOTS). students in Social Studies class IX.9 SMP Negeri 2 Singaraja for the 2022/2023 academic year. Based on the results of the data analysis conducted, it is known that after being given action, there was a significant increase in HOTS in students from previously only 46.4% of students increased to 71.4% of students who were at the level of thinking skills. ABSTRAKPenelitian ini bertujuan untuk mengetahui upaya untuk meningkatkan kemampuan HOTS peserta didik melalui model pembelajaran Problem Based Learning pada mata pelajaran IPS. Berdasarkan pembelajaran yang dilakukan siklus sebelumnya, masih banyak peserta didik yang belum mencapai kriteria ketuntasan minimum saat menjawab soal-soal dalam level kognitif HOTS. Solusi untuk mengatasi permasalahan tersebut adalah penggunaan model pembelajaran Problem Based Learning (PBL). Keterlaksanaan penelitian tindakan kelas ini, peneliti memfokuskan penelitian yang bertujuan untuk mengetahui adanya perubahan keterampilan berpikir tingkat tinggi peserta didik melalui penerapan model pembelajaran problem based learning, serta mengetahui seberapa besar peningkatan keterampilan berpikir tingkat tinggi peserta didik dengan menggunakan model pembelajaran problem based learning. Penelitian tindakan kelas ini dilakukan dalam dua siklus kegiatan dengan masing-masing siklus terdiri dari dua pertemuan pembelajaran. Adapun teknik pengumpulan data yang digunakan dalam penelitian ini adalah teknik observasi dan tes. Berdasarkan hasil penelitian tindakan kelas yang telah dilaksanakan diperoleh kesimpulan bahwa pemanfaatan model pembelajaran problem based learning dapat digunakan sebagai salah satu upaya untuk meningkatkan Higher Order Thinking Skills (HOTS) peserta didik pada mata pelajaran IPS kelas IX.9 SMP Negeri 2 Singaraja tahun pelajaran 2022/2023. Berdasarkan hasil analisis data yang dilakukan diketahui bahwa setelah diberikan tindakan terjadi peningkatan HOTS yang cukup signifikan pada peserta didik dari yang sebelumnya hanya 46,4% peserta didik meningkat menjadi 71,4% peserta didik yang berada pada keterampilan berpikir tingkat tinggi.

Narwianta, Nanang. (2019). The Evaluation of Higher Order Thinking Skills in English School Nationally Standardized Examination at State Senior High School 6 Semarang. Thesis, English Language Education Pascasarjana Universitas Negeri Semarang. Supervisor I. Dr. Dwi Anggani Linggar Bharati, M.Pd. Supervisor II. Prof. Dr. Dwi Rukmini, M.Pd Keywords: Evaluation, HOTS, English Nationally Standardized Examination The English school nationally standardized examination as an assessment of the learning outcomes conduct by educational unit is designed to measure the student`s achievement competencies, to determine student`s graduation from the school, it is very significant to conduct test items evaluation since it gives a clear portrait of the quality of the items and the test as a whole. The purpose of this study was to explain the realization of higher order thinking skills reflected in English school nationally standardized examination at state senior high school 6 Semarang academic year of 2018/2019. Then this research evaluate whether the items in English school nationally standardized examination met the requirement of content validity, reliability and practicality. The researcher use the mixed method using analysis card to analyze the realization of higher order thinking skills in English school nationally standardized examination. The content validity analyze through matching method between the basic competensies and the question in paper test.While to ensure the reliability of higher order thinking skills in English school nationally standardized examination, use the KR-21 for multiple choice test items and alpha cronbach for essay test items. And the practiality obtained through checklist instrument. The results indicated that HOTS realized in listening, reading, and writing questions. The listening in the form of spoken written, because it is to test listening simultaneously with speaking. There are one listening question categorized into HOTS of analyzing level, in reading the questions belonging to HOTS reach 8 questions that consist of 5 questions of analyzing level and 3 questions of evaluating level. Meanwhile, there is one question of HOTS into creating level in writing questions. The results also showed the proportion of HOTS comprised 22,22% met the requirement from BNSP that stated in academic year 2018/2019 there is approximately 10-15% of higher order thinking skills test items. Meanwhile, the content validity of the questions is high, and then the reliability of those test items categorized as unreliable and the practicality meet the requirements of the practical test items. Accordingly, this finding reveals that there is still much room for the variation of HOTS levels implemented all skills in English school nationally standardized examination. In the light of these result, this study recommends modifying the proportion of HOTS in four skills by providing them with more representation of the content validity.