Designing LLM-supported informal learning in social reading: a stage-based co-creation approach

We introduce a novel writing method called Probing Chain-of-Thought, which potentially prevents students from cheating using a large language model while enhancing their critical thinking. large language models have disrupted education and many other fields. For fear of students cheating, many educationists have resorted to banning their use. We conduct studies in two different courses with 65 students using qualitative research design primarily (i.e. phenomenological) and quantitative methods. The students in each course were asked to prompt a large language model of their choice with one question from a set of four (random) questions and required to affirm or refute statements in the large language model output by using peer-reviewed references as evidence. In addition, the rubric for assessing the students writing included 5 more criteria: focus, logic, content, style and correctness. The average success rate of the writing of students based on the criteria for the two cases is 79.49% (±12.82%). The results of the rubric assessment show two things: (1) Probing Chain-of-Thought stimulates critical thinking and writing of students through engagement with large language models when we compare the large language models-only output to Probing Chain-of-Thought output and (2) Probing Chain-of-Thought may prevent cheating because of clear limitations in the concerned large language models when we compare students’ Probing Chain-of-Thought output to large language models’ Probing Chain-of-Thought output. In quantitative analysis, we also discover that most students prefer to give answers in fewer words than large language models, which are typically verbose. The average word counts for students in the first course, ChatGPT 3.5, and Phind (v8) are 208, 391 and 383, respectively, while it is 405, 356, and 315 for students, ChatGPT 3.5, and BingAI, respectively, in the second course, where we enforced a minimum word-count of 300 for the students. We provide access to the outputs for possible assessments (available after review).

As generative artificial intelligence (AI) tools and large language models (LLMs)-powered applications develop rapidly in the era of algorithms, it should be integrated thoughtfully to enhance English as a Foreign Language (EFL) teaching and learning without replacing learners’ critical thinking (CT). This study systematically analyzes the impact of generative AI tools and LLMs on language learners’ CT in EFL education using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) framework to identify, evaluate, and synthesize relevant studies from 2022 to 2025. A thorough review of 15 selected studies focuses on generative AI tools and LLMs’ dual nature, research methods, main focuses, theory and models, limitations and challenges, and future directions in the field based on Web of Science (WoS), SCOPUS, ERIC, ProQuest, and Google Scholar. The findings identified generative AI tools and LLMs possessed both the potential to nurture and the risk of hindering CT in EFL education. 66.67% of studies reported generative AI tools and LLMs’ positive role in CT, while 33.33% of studies reported its negative role in CT. Furthermore, 3 types of research methods, 3 key themes of research focus, and 4 groups of theoretical perspectives were examined. However, 4 kinds of limitations in this field remain, including research scope, user dependency, generative AI reliability, and pedagogical integration. Future research can focus on assessing long-term effects, broadening research scope, promoting responsible AI use, and refining pedagogical strategies. Finally, Limitations, implications and future direction of this study were discussed.

Journal clubs (JCs) play an important role in medical education by promoting critical appraisal and evidence-based practice. However, residents often face barriers to effective participation. Some of the issues that are commonly faced include limited time and difficulty understanding complex concepts and statistics. This study was conducted during March-September 2024 and explored the use of a custom-trained Large Language Model (LLM) as a supportive tool for JC preparation and participation among postgraduate dental residents. Using a design-based research approach, researchers implemented the LLM integrated with relevant literature. Six JC sessions were conducted with sixteenresidents across two subspecialties, accompanied by structured observations, feedback forms, and pre-/and post-focus groups with residents and faculty (n=16). Findings revealed that the LLM improved residents' comprehension of complex content, enhanced confidence, and increased engagement during discussions. Residents used the tool for summarizing articles, clarifying statistical methods, and generating discussion points. Fiftythree percent reported a positive experience of using the LLM for JC preparation, Forty-three percentwere neutral, and only one response was negative. However, challenges included the need for precise prompt construction, occasional content inaccuracies, and limited depth in some specialized areas. Faculty observed enhanced participation but stressed the need for critical evaluation of LLM outputs. Both groups identified prompt-writing skills, critical thinking, and AI literacy as key competencies for effective LLM use. LLMs can complement traditional teaching by supporting deeper engagement in JCs. As generative AI evolves, further research should examine its broader implications on learners' cognitive processes, epistemic trust, and educational equity.

The world is experiencing an AI revolution, with large language models (LLMs) transforming various industries, including education. Academics are striving to harness the potential of LLMs while also contending with their risks. This paper presents the first bibliometric analysis focused on LLM research in programming education, identifying leading countries, authors, and institutions while analyzing key terms and popular keywords in this field. Additionally, it highlights influential studies on topics such as introductory programming, computer science, computing, programming education, and prompt engineering, discussing key insights from these works. Findings indicate that LLMs could play a significant role in programming education and may be integrated into computer science curricula. However, careful consideration is needed to ensure their benefits outweigh their risks across various use cases. This study specifically examines ChatGPT as a representative LLM, exploring its benefits and limitations as both a learning aid for students and a support tool for professionals. It also evaluates the quality of ChatGPT-generated code and its effectiveness in simplifying programming concepts for beginners. Furthermore, the ethical implications of increasing reliance on LLMs for programming tasks, including concerns about dependency, plagiarism, and potential effects on critical thinking, are addressed. By contributing to the ongoing discourse on integrating AI tools like ChatGPT in programming education, this research emphasizes the importance of responsible and ethical usage to maximize benefits for students, educators, and the broader educational community.

IntroductionJournal clubs are integral to continuing medical education, promoting critical thinking and evidence-based learning. However, inconsistent engagement, reliance on faculty expertise, and the complexity of research articles can limit their effectiveness. Generative Artificial Intelligence (Gen AI), particularly Large Language Models (LLMs) offers a potential solution, but general-purpose LLMs may generate inaccurate responses (“hallucinations”). Retrieval-Augmented Generation (RAG) mitigates this by integrating AI-generated content with curated knowledge sources, ensuring more accurate and contextually relevant responses. This study explores the development and preliminary evaluation of a RAG-enhanced LLM to support journal club discussions.Materials and methodsA specialized LLM was deployed using Microsoft Azure’s GPT-4o. A vector database was created by embedding journal club articles using text-embedding-ada-002 (Version 2) for efficient information retrieval. A dedicated website provided user-friendly access. The study followed a design-based research (DBR) approach, engaging residents and faculty who interacted with the LLM before and during journal club sessions. Data collection included focus group discussions (FGDs) and questionnaires assessing engagement, usability, and impact.ResultsThe study involved a total of 13 residents and three faculty members as participants. 50% of residents reported a positive experience, while the rest had a neutral response, citing both advantages and limitations. The LLM improved article summarization, query responses, and engagement as reported by residents. Moreover, the faculty observed enhanced discussion quality and preparation whereas overall challenges included the need for precise prompts and occasional misleading responses.ConclusionThe study highlights the potential of a RAG-enhanced LLM to improve journal club engagement and learning. Future advancements in AI and open-source models may enhance accessibility, warranting further research.

Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM‐based applications to understand and generate natural language can potentially improve instructional effectiveness and learning outcomes, but the integration of LLMs in education technology has renewed concerns over algorithmic bias, which may exacerbate educational inequalities. Building on prior work that mapped the traditional machine learning life cycle, we provide a framework of the LLM life cycle from the initial development of LLMs to customizing pre‐trained models for various applications in educational settings. We explain each step in the LLM life cycle and identify potential sources of bias that may arise in the context of education. We discuss why current measures of bias from traditional machine learning fail to transfer to LLM‐generated text (eg, tutoring conversations) because text encodings are high‐dimensional, there can be multiple correct responses, and tailoring responses may be pedagogically desirable rather than unfair. The proposed framework clarifies the complex nature of bias in LLM applications and provides practical guidance for their evaluation to promote educational equity. Practitioner notes What is already known about this topic The life cycle of traditional machine learning (ML) applications which focus on predicting labels is well understood. Biases are known to enter in traditional ML applications at various points in the life cycle, and methods to measure and mitigate these biases have been developed and tested. Large language models (LLMs) and other forms of generative artificial intelligence (GenAI) are increasingly adopted in education technologies (EdTech), but current evaluation approaches are not specific to the domain of education. What this paper adds A holistic perspective of the LLM life cycle with domain‐specific examples in education to highlight opportunities and challenges for incorporating natural language understanding (NLU) and natural language generation (NLG) into EdTech. Potential sources of bias are identified in each step of the LLM life cycle and discussed in the context of education. A framework for understanding where to expect potential harms of LLMs for students, teachers, and other users of GenAI technology in education, which can guide approaches to bias measurement and mitigation. Implications for practice and/or policy Education practitioners and policymakers should be aware that biases can originate from a multitude of steps in the LLM life cycle, and the life cycle perspective offers them a heuristic for asking technology developers to explain each step to assess the risk of bias. Measuring the biases of systems that use LLMs in education is more complex than with traditional ML, in large part because the evaluation of natural language generation is highly context‐dependent (eg, what counts as good feedback on an assignment varies). EdTech developers can play an important role in collecting and curating datasets for the evaluation and benchmarking of LLM applications moving forward.

The application of deep learning to tabular data remains an ongoing challenge, with tree-based models such as XGBoost consistently outperforming neural network methods in most real-world scenarios. Recent advances in Large Language Models (LLMs) highlight their ability to generalize to new tasks via few-shot or one-shot prompting, yet questions remain about their utility in structured, tabular domains—particularly for resource-efficient Small Language Models (SLMs). In this work, we systematically evaluate the effectiveness of both large and small transformer-based language models for tabular classification tasks using few-shot strategies. Our approach investigates input serialization schemes and prompt engineering to maximize performance in low-data regimes. Experiments on benchmark datasets—including Diabetes, Heart Failure, and German Credit Risk—demonstrate that well-designed SLMs can approach, and occasionally match, the performance of much larger models, substantially reducing the need for extensive labeled data and retraining. However, further advances are required for language models to consistently rival the strongest tree-ensemble baselines. Our findings support the idea that SLMs, when properly prompted, offer a promising, flexible, and label-efficient alternative for automating and dynamizing machine learning pipelines on tabular data.

Tools based on the use of Large Language Models (LLMs) have improved the computer programming teaching process, automated feedback processes, facilitated program repair, and enabled personalized learning experiences. This research examines which and how LLM-based opportunities are applied in the computer programming teaching assessment process and how LLMs are applied to improve evaluation accuracy, their impact on student learning outcomes, and the challenges in scaling these technologies. Key opportunities arise from prompt engineering, which optimizes precision and LLM-generated feedback relevance, and feedback propagation techniques, which offer scalable solutions for large-scale programming courses. LLMs are also applied effectively in debugging assistance to detect and repair syntactic and semantic errors in student code. This review identifies several research directions, including prompt engineering refinement, improved feedback system scalability, and deeper exploration of the long-term educational impacts of LLM. The study concludes that LLMs are effective in enhancing the assessment process, but a balanced approach combining human oversight with automated feedback is crucial to fostering critical thinking and ensuring long-term learning success in programming education.

Research has revealed that critical thinking (CT) can predict undergraduate academic performance well. It is essential for nurturing students’ unique voice and creativity, especially in the era of Large Language Models (LLMs). LLMs can generate linguistically complex texts, potentially overshadowing students’ authorial voices—particularly those still developing language skills and domain-specific knowledge. Despite this, research on CT within subject education contexts remains scarce, leaving disciplinary lecturers’ expectations of CT in disciplinary writing largely unexplored. Our study analysed a 160,527-word corpus of lecturer feedback on 230 assignments from three education-related postgraduate programs to reveal their shared expectations of CT in disciplinary writing. We identified nine CT constructs, each illustrated with relevant feedback examples. Their intricate interrelationships were revealed through hierarchical cluster analyses. Drawing inspiration from Bloom’s Taxonomy, we established a CT taxonomy to guide effective and consistent assessment of CT. Further studies and practical applications of the CT taxonomy within local educational contexts are needed to develop students’ CT skills and empower them to retain their voice and creativity in an AI-empowered era.

Large Language Models (LLMs) have demonstrated a remarkable capacity for generating human-quality text across diverse applications. In light of this, the use of these models is becoming increasingly widespread. Education is no exception and so literature is starting to explore the potential of LLMs in educational contexts: ranging from teaching assistant chatbot to personalized tutoring. Although the versatility of these models is evident, the critical question arises whether they can truly replicate the creativity and nuances of human languages, giving students the variability and richness of expression necessary for effective learning, and for nurturing critical and divergent thinking. This is particularly important in the pre-school or primary education context, where novel and insightful content is crucial for effective engagement. Framing this problem from an abstract linguistic perspective, it translates into wondering how effectively the “creativity” of LLM-generated text can be quantified and how it compares to human one. So, recognizing the inherent ambiguity in defining creativity this paper investigates the variability of LLM outputs as a potential proxy, and compares it with observed text variability in a large human-authored dataset. Specifically focusing on early childhood education (ECE), we compare the semantic and syntactic diversity of responses from LLMs and humans prompted to answer like 5-year-olds. Our empirical results show LLMs have lower variability than humans, particularly on repeated questions, though the gap narrows for different questions. This limitation raises concerns for educational suitability, where diverse explanations of concepts are crucial, suggesting a need to enhance LLM expressive range and diversity.

Distribution network design is one of the most essential strategic decisions in logistics systems and supply chains, as it structures the network by determining the number of echelons and specifying the types, quantities, and locations of various facilities. This chapter presents decision-support models for distribution and logistics network design, covering both continuous location models (gravity location models) and discrete network design models (capacitated/uncapacitated and their relevant variants). Comprehensive numerical experiments based on the same case study offer a direct comparison of these different network design approaches. Furthermore, as one of the most disruptive technologies in Industry 5.0, the impact of generative AI (GenAI) and large language models (LLMs) on the research and education of distribution network design is thoroughly analyzed. Our experiments demonstrate that GenAI and LLMs are powerful tools in providing structured introductions to existing problems, guiding model selection and explanation, assisting with coding and code clarification, facilitating model comparison, and generating initial ideas for new research questions in distribution and logistics network design. However, they also face reliability and accuracy challenges, exhibit limited critical thinking capabilities, and struggle to address emerging research questions. Our findings offer valuable guidance for incorporating GenAI tools and LLMs into research and educational processes in distribution and logistics network design.

Artificial intelligence, especially large language models (LLMs), is reshaping how we learn, communicate, and create. These systems provide students and professionals immediate access to fluent, context-aware language that can support learning, increase productivity, and spark creativity. But what happens when their use becomes excessive? This paper explores the potential long-term consequences of over-relying on LLMs—particularly memory, critical thinking, creativity, and motivation. Drawing on research in cognitive psychology, education, neuroscience, and media studies, it argues that LLMs are best understood as cognitive prostheses: incredibly valuable when used wisely, but risky when they replace rather than support human imagination.

Applications driven by large language models (LLMs) are reshaping higher education by offering innovative tools that enhance learning, streamline administrative tasks, and support scholarly work. However, their integration into education institutions raises ethical concerns related to bias, misinformation, and academic integrity, necessitating thoughtful institutional responses. This article explores the evolving role of LLMs in midwifery higher education, providing historical context, key capabilities, and ethical considerations. Using insights from a US-based midwifery program, it highlights strategies for responsible LLM integration, including faculty development, classroom applications, and policy updates. The discussion addresses challenges such as mitigating bias, preventing plagiarism, and fostering critical thinking while ensuring that LLM-fueled applications remain a tool rather than a substitute for student learning. Practical approaches, including faculty training and student guidance, offer a replicable framework for leveraging LLM tools in professional health education while maintaining academic standards and equity.

Decision-making requires examining underlying assumptions and concepts, considering diverse perspectives, and weighing potential consequences with clear, accurate reasoning. Recent large language models (LLMs) show promise for assisting decision-makers by combining reasoning capabilities with the ability to retrieve relevant information from large documents. However, our formative study with five professional decision-makers revealed key limitations of using LLM in workflow: time-consuming alignment of user goals, lack of evidence-based grounding, overwhelmingly long outputs, and unsurfaced assumptions undermined user trust in the LLM output and the validity of the final decision. We introduce Criticality, a system that operationalizes the Paul-Elder Critical Thinking framework to structure reasoning into interactive Elements of Thought (e.g., purpose, assumptions, perspectives, implications), and evaluates and guides reasoning using Intellectual Standards (e.g., clarity, fairness, logic). It also retrieves evidence for each claim, classifies it as supporting, neutral, or contradictory, and explains the claim-evidence link. A within-subjects study (n=13) comparing Criticality to ChatGPT 5 Pro, a state-of-the-art reasoning model in conversational interface, found that Criticality improved user interaction of steering and repairing through the decision-making process, producing better decision rationales compared to the baseline.

Integrated STEM projects and experiments promote interdisciplinary learning, enhance critical thinking, and prepare students for real-world problem-solving by bridging science, technology, engineering, and mathematics. While numerous research studies have explored the STEM education domain, few have focused on supporting educators in identifying and designing effective integrated STEM projects and experiments. This research addresses this gap by introducing a novel system that leverages machine learning (ML) and large language models (LLMs) to assist educators in classifying and creating integrated STEM projects.We created and labeled a dataset of 1,071 project descriptions in this research work, mapped to one or more STEM areas (Science, Technology, Engineering, Mathematics). The dataset was developed using manually curated records, harvesting online sources, and synthetic data generation. Natural Language Processing (NLP) techniques were applied to prepare project descriptions for analysis. Various ML models, including Random Forest, Support Vector Machines (SVM), Logistic Regression, and Naive Bayes, were used for multi-label classification. The Random Forest model achieved the best performance, with a Precision of 91.99%, Recall of 94.59%, F1 Score of 93.27%, and Hamming Loss of 6.74%.The system operates in two phases: In Phase 1, the instructor proposes a project title with a description, and in response, the ML model identifies one or more STEM areas as the most relevant. In Phase 2, ChatGPT, specifically GPT-4o, assists educators by defining experiment objectives, generating ideas, designing procedures, and suggesting tools and resources. This human-in-the-loop system enhances collaboration between educators, ML models, and LLMs, streamlining the design of effective integrated STEM projects.This work's implications extend to improving the accessibility and effectiveness of integrated STEM education by reducing the barriers educators face in designing interdisciplinary projects. It provides a scalable framework for integrating advanced AI techniques into educational practices and fostering classroom innovation.