• Home
  • Search
  • Evaluating AI-Generated Questionnaires Using LDA Topic Modeling and KMeans Clustering: A Comparative Study with Human-Designed Instruments
  • https://doi.org/10.1145/3759275.3759297Copy DOI Icon

Evaluating AI-Generated Questionnaires Using LDA Topic Modeling and KMeans Clustering: A Comparative Study with Human-Designed Instruments

  • Jun 13, 2025
  • Menghan Cheng +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

The rapid development of large language models (LLMs) has opened up new opportunities for automated questionnaire design in academic research. However, questions remain about the validity and quality of AI-generated tools, especially when applied to theory-driven frameworks. The aim of this study is to assess the performance of AI-generated questionnaires and human-designed questionnaires in single and integrated theoretical models. To this end, we employ a hybrid approach that combines expert evaluation and unsupervised machine learning techniques. Specifically, we use latent dirichlet allocation (LDA) for topic modeling to assess semantic coverage. And, KMeans clustering was used to detect redundancy and assess semantic consistency. We created four questionnaires: two based on validated manual writing tools and two generated by the GPT-4. This covered the Unified Theory of Acceptance and Use of Technology (UTAUT) and its extended models. 310 The questionnaires assessed program quality on multiple dimensions. And, these judgments were objectively validated using machine learning outputs. Results show that AI-generated questionnaires are fluent and objective, but perform poorly in terms of accuracy, clarity, and comprehensiveness, especially under complex modeling conditions. Redundancy and semantic drift increased with theory integration. However, the AI performed well in areas requiring standardization and neutrality.

Similar Papers
  • PDF
  • Research Article
  • Citations65

Machine Learning Ranks ECG as an Optimal Wearable Biosignal for Assessing Driving Stress

  • Jan 01, 2020
  • IEEE Access
  • Mohamed Elgendi +1
  • PDF
  • Research Article
  • Citations2

Trends in COVID-19 Publications: Streamlining Research Using NLP and LDA

  • Jan 01, 2020
  • SSRN Electronic Journal
  • Akash Gupta +3
  • Research Article
  • Citations14

Latent Dirichlet allocation topic modeling of free-text responses exploring the negative impact of the early COVID-19 pandemic on research in nursing.

  • Nov 30, 2022
  • Japan Journal of Nursing Science
  • Madoka Inoue +4
  • Research Article

COVID-19 Research Trends in Social Work: LDA Topic Modeling Analysis in South Korea

  • May 13, 2024
  • Journal of Social Service Research
  • Taejeong Park
  • Conference Article

Research on Theme Mining of Corporate Resilience Capabilities Based on LDA Topic Model

  • Nov 14, 2025
  • Shumin Yin +3
  • PDF
  • Research Article
  • Citations11

Determination of Motivating Factors of Urban Forest Visitors through Latent Dirichlet Allocation Topic Modeling.

  • Sep 13, 2021
  • International Journal of Environmental Research and Public Health
  • Doo-San Kim +2
  • Research Article

Research on the Mining of Failure Causes of Intelligent Customer Service Based on LDA Topic Model

  • Jan 01, 2025
  • Journal of Management and Social Development
  • Shanshan Fang +3
  • Conference Article
  • Citations1

PLENARY LECTURE - Machine learning tools for the treatment of offshore site investigations

  • Jan 01, 2024
  • B Stuyts
  • Conference Article
  • Citations10

Augmented Latent Dirichlet Allocation (Lda) Topic Model with Gaussian Mixture Topics

  • Apr 01, 2018
  • Kedar S Prabhudesai +3
  • Research Article
  • Citations28

An inclusive survey on machine learning for CRM: a paradigm shift

  • Dec 01, 2020
  • DECISION
  • Narendra Singh +2
  • Research Article

ADOPTION OF LOW-CODE AI IN SUPPLY CHAIN RISK MANAGEMENT: A TECHNOLOGY MANAGEMENT PERSPECTIVE WITH BIBLIOMETRIC EVIDENCE

  • Mar 21, 2026
  • Yönetim ve Ekonomi Araştırmaları Dergisi
  • Fatih Çallı
  • Research Article

A Machine Learning-based Approach to Diabetes Prediction

  • May 01, 2023
  • International Journal of Scientific Research in Science and Technology
  • Amit Katoch +5
  • Research Article

Trends in Artificial Intelligence Research on Education: Topic Modeling Using Latent Dirichlet Allocation

  • May 16, 2025
  • Bulletin of Kharkov National Automobile and Highway University
  • Nataliia Fil +3
  • Conference Article
  • Citations1

The Use of Text Mining to Identify Trends and Relationships between Social Determinants of Health and Congestive Heart Failure.

  • Oct 01, 2024
  • Ifeoluwa Solaru +1
  • Conference Article

Feature Extraction and Semantic Analysis of University Party Building Texts Based on the LDA Topic Model

  • Sep 19, 2025
  • Junyu Zhang +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.