• Home
  • Search
  • Comparative Analysis of Large Language Models for Automated Question Generation from Video-Based Learning Content
  • https://doi.org/10.1109/tai.2025.3620274Copy DOI Icon

Comparative Analysis of Large Language Models for Automated Question Generation from Video-Based Learning Content

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

The increasing adoption of video-based learning has presented a significant challenge in effectively assessing learner’s comprehension and engagement. This paper addresses this gap by conducting a comparative analysis of ten state-of-the-art large language models (LLMs) for Automated Question Generation (AQG) from video transcripts. Our evaluation centres on assessing the quality of four question types: short-answer, multiple-choice, boolean, and gap-fill. Our major contributions are three-fold. First, we propose an evaluation framework for comparing multiple LLMs in their ability to generate different question types, identifying the models best suited for each based on five key evaluation metrics–relevance, reading comprehension, question difficulty, clarity, and context utilisation–that holistically assess question quality. Second, we release the prompts used for both question generation and evaluation, providing a valuable resource for future AQG research. Third, we uncover the strengths and weaknesses of each LLM, identifying which question types require further improvement, and uncovering statistically significant differences in AQG performance across the LLMs. By providing an in-depth comparative analysis, this study provides important insight into the development of more effective AQG systems, contributing to the broader integration of AI in automated assessment in video-based learning environments.

Similar Papers
  • Research Article
  • Citations713

Practical and ethical challenges of large language models in education: A systematic scoping review

  • Aug 06, 2023
  • British Journal of Educational Technology
  • Lixiang Yan +8
  • Book Chapter
  • Citations78

Towards Human-Like Educational Question Generation with Large Language Models

  • Jan 01, 2022
  • Zichao Wang +3
  • Research Article
  • Citations2

GPT-4o and OpenAI o1 Performance on the 2024 Spanish Competitive Medical Specialty Access Examination: Cross-Sectional Quantitative Evaluation Study

  • Jan 12, 2026
  • JMIR Medical Education
  • Pau Benito +9
  • Research Article
  • Citations11

Ontology Mapping for Retrieval Augmented Modelling to Reduce Factual Hallucinations in Pretrained Language Model-Based Auto-Generated Questions

  • Feb 01, 2025
  • Applied Ontology
  • R Tharaniya Sairaj +1
  • PDF
  • Research Article
  • Citations7

AUTOMATIC GENERATION OF ENGLISH LANGUAGE TEST QUESTIONS USING MATHEMATICA

  • Sep 17, 2017
  • CBU International Conference Proceedings
  • Anna Malinova +1
  • Research Article

A large comprehensive comparison of large language models on ophthalmology board exams.

  • May 14, 2026
  • Graefe's archive for clinical and experimental ophthalmology = Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie
  • Alon Moore Galindo +6
  • Research Article
  • Citations125

Performance of Large Language Models on a Neurology Board–Style Examination

  • Dec 07, 2023
  • JAMA network open
  • Marc Cicero Schubert +2
  • Research Article
  • Citations3

Optimizing Automated Question Generation for Educational Assessments

  • Jun 04, 2025
  • Engineering, Technology & Applied Science Research
  • Sumayyah Alamoudi +2
  • Research Article
  • Citations7

Education Research: Can Large Language Models Match MS Specialist Training?

  • Nov 20, 2025
  • Neurology: Education
  • Hernan Inojosa +6
  • Research Article
  • Citations8

MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback.

  • Jan 01, 2025
  • Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting
  • Zonghai Yao +6
  • Research Article
  • Citations3

Enhancing Diversity in Difficulty-Controllable Question Generation for Reading Comprehension via Extended T5

  • Nov 25, 2024
  • International Conference on Computers in Education
  • Teruyoshi Goto +2
  • Research Article
  • Citations15

A novel framework for Automatic Chinese Question Generation based on multi-feature neural network model

  • Jan 01, 2018
  • Computer Science and Information Systems
  • Hai-Tao Zheng +3
  • PDF
  • Conference Article
  • Citations113

Towards a Better Metric for Evaluating Question Generation Systems

  • Jan 01, 2018
  • Preksha Nema +1
  • Research Article

Вдосконалена класифікація великих мовних моделей

  • Jan 01, 2025
  • Scientific papers of Donetsk National Technical University. Series: Informatics, Cybernetics and Computer Science
  • С.М Бабчук
  • Research Article

MCQ Generator

  • Mar 19, 2026
  • INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Kakumanu Amani +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.