Qualitative Research Methods in Education and Educational Technology

In 2007, Borrego published her groundbreaking work “Conceptual difficulties experienced by trained engineers learning educational research methods”[1] that described five conceptual difficulties engineering faculty encountered as they began to learn about rigorous research in engineering education. Since Borrego's research was published the engineering education departments have been formed and a new generation of engineering education researchers has emerged. These researchers must tackle the same paradigm shift that faced the engineering faculty in Borrego's study. The research question that guided this study was: “What conceptual hurdles still exist for the new generation of engineering education researchers?” PhD students in Engineering Education were asked to reflect on their conceptual difficulties during the first semester of their PhD courses and write an end-of-semester reflective paper that summarizes their cognitive journey. The summative reflections were coded using Borrego's conceptual hurdles as a framework. The results of the present study can inform those entering the engineering education research community by highlighting areas that remain problematic and offer strategies that emerging scholars have used to make sense of educational theories and methods.

This paper reflects on a number of different ways by which comparative research in technology education has been undertaken across countries. The history of comparative research in technology education demonstrates that it is possible to identify three major periods in the process of its development. When technology education was established a comparison had been made at the level of curriculum documents, syllabi and State Orders. People involved in the development of technology education were looking around the world for ideas. During this first stage, approaches to the analysis were not very systematic. The second stage in the development of comparative research in technology education could be characterised by a great number of published or presented papers that describe the situation in a particular country. Even though comparison, as such, was not used in this research the underlying assumption was that it should inform the research community about different approaches towards technology education and that it would be beneficial for the field. The third stage involved comparison between two or more countries about one or more specific aspects of technology education. These include for example, the meaning of the major concepts, teaching methods, comparison of goals, comparison of the balance between the global trends and local specificities. In this paper it is argued that it is essential to move towards the next stage - a comparison on the basis of two ideological beliefs about the purposes of general education. These beliefs are: whether education is designed to broaden minds and develop all students in the creation of a better society or is it really about training students to live and work in a market-oriented state, to be 'productive' in seizing the opportunities of the market. These two beliefs and the approaches derived from them have been chosen because they summarize an important issue that divides different social theories in their views on the role of education in society. They can provide a useful framework for analysing historical influences, curriculum theories, documents, and school practices in technology education in different countries.

Contents: L.A. Shepard, Preface. J.L. Green, G. Camilli, P.B. Elmore, Introduction to the Handbook: What's Complementary About Complementary Methods. A. Skukauskaite, E. Grace, On Reading and Using the Volume: Notes to Students. Part I: Foundations. E. Bredo, Philosophies of Educational Research. G.J. Kelly, Epistemology and Educational Research. K.A. Strike, The Ethics of Educational Research. Part II: Introduction to Design & Analysis. C. Bazerman, Analyzing the Multidimensionality of Texts in Education. T. Barone, Arts-Based Educational Research. R.K. Yin, Case Study Methods. K.M. Borman, C. Clarke, B. Cotner, R. Lee, Cross-Case Analysis. A.C. Porter, Curriculum Assessment. K.H. Kim, P.M. Bentler, Data Modeling: Structural Equation Modeling. F. Erickson, Definition and Analysis of Data From Videotape: Some Research Procedures and Their Rationales. A.H. Schoenfeld, Design Experiments. H-Z. Ho, S.L. O'Farrell, S. Hong, S. You, Developmental Research: Theory, Method, Design, and Statistical Analysis. D. Bloome, C. Clark, Discourse-In-Use. H. Braun, Empirical Bayes. J.P. Shaffer, Estimation. K.M. Anderson-Levitt, Ethnography. J. Nespor, Finding Patters With Field Notes. R.J. Shavelson, N.M. Webb, Generalizability Theory. J.L. Rury, Historical Research in Education. A. Henry, Historical Studies: Groups/Institutions. M.E. Brenner, Interviewing in Educational Research. L. Crocker, Introduction to Measurement Theory. S. Embretson, X. Yang, Item Response Theory. D.M. Harrison, S.W. Raudenbush, Linear Regression and Hierarchical Linear Models. G.V. Glass, Meta-Analysis: The Quantitative Synthesis of Research Findings. C.A. Chinn, The Microgenetic Method: Current Work and Extensions to Classroom Research. M.L. Smith, Multiple Methodology in Education Research. F.M. Connelly, D.J. Clandinin, Narrative Inquiry. N.C. Burbules, B.R. Warnick, Philosophical Inquiry. M. Cochran-Smith, K. Donnell, Practitioner Inquiry: Blurring the Boundaries of Research and Practice. J.R. Levin, Probability and Hypothesis Testing. W.R. Shadish, J.K. Luellen, Quasi-Experimental Design. T.D. Cook, V. Sinha, Randomized Experiments in Educational Research. M. Eisenhart, Representing Qualitative Data. B. Thompson, Research Synthesis: Effect Sizes. H. Wainer, P.F. Velleman, Statistical Graphics: A Guidepost for Scientific Discovery. M. Berends, Survey Methods in Educational Research. J. Chromy, Survey Sampling. Part III: Programs of Research. C. Genishi, T. Glupczynski, Language and Literacy Research: Multiple Methods and Perspectives. L.M. McNeil, E.M. Coppola, Official and Unofficial Stories: Getting at the Impact of Policy on Educational Practice. M.E. Goertz, Policy Analysis: Studying Policy Implementation. S. Kushner, C. Adelman, Program Evaluation: A Democratic Practice. L.A. Rex, S.C. Steadman, M.K. Graciano, Researching the Complexity of Classroom Interaction. J.A. Banks, Researching Race, Culture, and Difference: Epistemological Challenges and Possibilities. J.G. Greeno, Theoretical and Practical Advances Through Research on Learning. S. Noffke, K. Zeichner, Programs of Research in Teacher Education.

ABSTRACTAustralasian Association for Engineering Education (AAEE) has been sponsoring a Winter School in Engineering Education Research Methods since 2011. This paper describes how in 2017 attendees at the School applied what they had learned about a little-used data-gathering technique: observation. Starting with a Program Logic analysis, which identifies what an intervention ought to be doing, and hence what kind of evidence needs to be collected to describe its effect, some participants who had attended prior Winter Schools, were given the chance to collect the evidence. They found observation to be much harder to do well than one would think. This paper describes their experience and argues for the use of observational techniques in order to triangulate our data-gathering methods and improve the quality of our educational research. However, we also learned that a great benefit of observation comes from sustained reflection on the process and the data collected. Without such reflection, we argue observation is likely to produce rather thin results.abbriviations: AAEE - Australasian Association of Engineering Education; JEE - Australasian Journal of Engineering Education; WS2017 - 2017 Winter School

Book reviewed in this article:Beard, R. (1969). An Outline of Piaget's Developmental Psychology.Elkind, D., and Flavell, J. H. (Ed.) (1969). Studies in Cognitive Development.Dale, R. R. (1969). Mixed or Single‐sex Schools?Freeman, J., Butcher, H. J., and Christie, T. (1968). Creativity: A Selective Review of Research.Hope, K. (1969). Methods of Multivariate Analysis.Kay, W. (1968). Moral Development.Lovell, K. (1968). An Introduction to Human Development.O'Connor, K. (1968). Learning: An Introduction.Musgrave, P. W. (1968). The School as an Organisation.Langford, G. (1968). Philosophy and Education.Apter, M. J. (1968). The New Technology of Education.Firth, B. (1968). Mass Media in the Classroom.McFarland, H. (1969). Human Learning: A Developmental Analysis.Mittler, P. J. (Ed.) (1968). Aspects of Autism: Some Approaches to Childhood Psychoses.Musgrove, F., and Taylor, P. H. (1969). Society and the Teachers' Role.Peters, R. S. (Ed.) (1969). Perspectives on Plowden.Pidgeon, D., and Yates, A. (1968). An Introduction to Educational Measurement.Rothney, J. W. M. (1968). Methods of Studying the Individual Child: The Psychological Case Study.Thomas, A., Chess, S., and Birch, H. G. (1968). Temperament and Behaviour Disorders in Children.Vernon, P. E. (1969). Intelligence and Cultural Environment.Wiersma, W. (1969). Research Methods in Education: An Introduction.

Research Methods in Education is the seventh edition of a classic in educational research. It encompasses the whole range of methods currently employed in educational research at all stages. Method...

ABSTRACTWe investigated research processes applied in recent publications in the European Journal of Engineering Education (EJEE), exploring how papers link to theoretical work and how research processes have been designed and reported. We analysed all 155 papers published in EJEE in 2009, 2010 and 2013, classifying the papers using a taxonomy of research processes in engineering education research (EER) (Malmi et al. 2012). The majority of the papers presented either empirical work (59%) or were case reports (27%). Our main findings are as follows: (1) EJEE papers build moderately on a wide selection of theoretical work; (2) a great majority of papers have a clear research strategy, but data analysis methods are mostly simple descriptive statistics or simple/undocumented qualitative research methods; and (3) there are significant shortcomings in reporting research questions, methodology and limitations of studies. Our findings are consistent with and extend analyses of EER papers in other publishing venues; they help to build a clearer picture of the research currently published in EJEE and allow us to make recommendations for consideration by the editorial team of the journal. Our employed procedure also provides a framework that can be applied to monitor future global evolution of this and other EER journals.

Recent concern about lack of scientific rigor in educational research has led to a reexamination of educational research methods. Methodology requires particular attention when researchers study learning in complex learning environments such as classrooms or computer-driven environments that simulate complex processes. This article reaffirms the importance of experimental research for answering some research questions. It argues that nonexperimental methods, such as design experiments and methods applied to the study of large-scale educational programs, such as randomized field trials, are useful for answering other kinds of research questions about learning in complex settings. This article discusses the validity of evidence gathered using these nonexperimental methods. Finally, it suggests that the study of the dynamics of learning can help integrate data from experiments that reveal details about learning and data from nonexperimental research that reveal how learning occurs in complex settings.

<div class="WordSection1"><p>I am honored for being an alumnus of KEMC, and am delighted to write this editorial for the Annals. I have, ever since done my research and taught qualitative methods, felt that clinical methods are quite akin to qualitative research. Then, why only few clinical researchers use qualitative methods? A witness to this assumption is the research published in the Annals. In this treatise, I try to identify some parallels between the two types of methods and argue for the clinicians to invest in learning qualitative research methods to practice clinical methods well.</p><p> Qualitative research is to identify and interpret issues from the perspective of participants, their experience of: illness or disability, using health service, and to appreciate the meanings they give to the behaviour, events or objects in the context of their social and cultural norms. In this type of research, the emphasis is on exploring the associations and understanding the phenomenon in its holism; and not, like in quantitative research, from an outsider’s perspective and for certain specific aspects.<sup>1</sup> It requires participants with specific characteristics, selected purposely that can best inform the research topic. More participants, identified inductively during data collection, are added to develop full and multiple perspectives about the cases.<sup>2</sup></p><p> No preset data collection tool is used, instead qualitative researcher guided by a research question acts as an instrument, since the line of enquiry he changes during data collection as new understanding is gained and/or the situation changes.<sup>3</sup> The data for research is derived from the observation, interviews or verbal interactions, focus group discussions, document reviews, life histories etc. and the researcher asks why, how and under what circumstances things occur; and not just what, where and when. It is recorded in words or pictures and log book is used to record notes arising from interviews, observations, extracts from documents etc.</p><p> In health care settings patients are the subjects for clinical methods. The clinician, even prior to any verbal communication, observes the patient, e.g. for his gait and appearance. If in a bed or examination couch, his posture could give some clue to the illness. Inspection, a clinical method, is like systematic observation, which is qualitative method, should be holistic. In my third year during bedside teaching, Professor (late) Rashid Ahmed Qureshi said, <em>“patient has come to you as a whole and not his stomach in a tray”</em>, when a student straight went to examine abdomen of a patient with acute abdomen. History taking, another clinical method, is like conducting semi structured in depth interview – a qualitative research method. In both disciplines, we are told, <em>“not to ask leading, but follow up and probing questions”; </em>and Professor (late) Alamgir Khan, while teaching clinical methods, would add, <em>“if a good history is taken, you will establish diagnosis in over 65% of cases”. </em>Likewise, as part of history taking, documents related to patient’s illness history and treatment are reviewed similar to document review in qualitative research.</p><p> The two approaches however differ in how the data is analysed. In clinical practice, diagnosis is established based on the pathophysiological knowledge or patient’s clinical condition is discussed in clinicopathological conference.<sup>4</sup> On the other hand, qualitative research employs meaning based data analysis, whereby the qualitative data is transformed into some form of explanation, understanding or interpretation of people and situation that is investigated.<sup>5,6</sup></p><p> In conclusion, the history, the observation and the review of document related to patient are since obtainned using qualitative methods, the clinician trained in these methods could not only conduct these methods well but also interpret the data to identify and detect obstacles to the change in clinical condition and the reasons why improvement does or does not occur.<sup>7</sup> Finally, while it is heartening that research forums are organised in the institutions affiliated with KEMU, in order the research is richer, the researchers’ skills in qualitative research methods should be built.</p></div>

This session is intended for engineering faculty and graduate students who are new to engineering education research. This workshop is based on the 2006 Frontiers in Education Special Session: “What IS Rigorous Research in Engineering Education?” winner of the Helen Plants award for best interactive session at FIE. A key objective of this workshop is to allow participants to compare and contrast educational research principles with the principles they are familiar with in their technical research area. Participants will work in small groups to co-construct and then compare criteria for quality technical and educational research in engineering. These criteria will be compared to those proposed in the NRC publication Scientific Research in Education. This activity allows participants to create their own list of criteria to judge “quality” educational research. This information can later be applied by participants to better understand engineering education research articles or proposals they read or are asked to review. This workshop is sponsored by the National Science foundation through Expanding and sustaining research capacity in engineering and technology education: Building on successful programs for faculty and graduate students (DUE — 0817461).

Global educational issues highlight the importance of preparing for the demands of the 21st century by developing core competencies. Geography, as a core subject and theme of the 21st century, necessitates the integration of technology into higher education to enable prospective teachers to effectively utilize technological tools. A crucial aspect of enhancing prospective teachers' technological pedagogical content knowledge is the use of instructional modules. Currently, geography modules in higher education are perceived to lack sufficient geographical content. This study proposes a solution to improve prospective geography teachers' Technological Pedagogical and Content Knowledge (TPACK) by employing a techno-geospatial based e-module. Researchers decided to apply the Educational Research and Development method because of its focus on developing effective products in the context of educational activities. The findings reveal a significant difference in the n-gain scores between classes using the techno-geospatial based e-module and those that do not. Classes utilizing the e-module demonstrated a moderate level on the self-report measure of TPACK competency, whereas classes without the e-module were categorized as low. The techno-geospatial based e-module has proven to be both effective and feasible for enhancing prospective geography teachers' TPACK. This research contributes to the development of e-modules aimed at preparation for 21st-century skills.

The purpose of this paper is to explore the possibility and significance of genealogy as an educational research method. This paper analyzed the concepts and characteristics of genealogy, and explored examples of research using genealogical methodology. Through this, the meaning and significance of genealogy as an educational research method are discussed and strategies for utilizing genealogy as an educational research tool are suggested. 
 Genealogy is the work of tracing the origins of knowledge about current concepts and of analyzing changes. It is a tool that can analyze that the knowledge we take for granted is not spontaneous but contains power relations. In particular, if genealogy is used as an educational research tool, it will be possible to be of help in conceptual reconstruction and practical development.
 The strategies for using genealogy as an educational research method are as follows. First, research starts by raising questions about the concept. Second, in order to collect data, a wide and vast amount of data is collected. Third, during the analysis, concepts are deconstructed and critically analyzed. Fourth, interpretation is done with the subjectivity of the researcher in mind. Fifth, in the writing stage, the researcher constantly reflects on themself.
 If we use genealogy when conducting educational research, we will be able to seriously reflect on the educational problems we are currently facing, and this will help us come up with new alternatives. This is expected to contribute to individual growth and to innovative changes in the education system.

Like many areas of curricula internationally, technology education has encountered difficulties in achieving continuity between the rhetoric of prevailing policy and the reality of enacted practices. In technology education, the conceptually oriented nature of curricular goals is theorised to play a significant part in influencing this relationship. One way in which investigations of this relationship have been approached is considering the application of pedagogical content knowledge (PCK) frameworks as a mechanism to understand the interaction of teachers’ knowledge and enacted practices. However, understanding from the philosophy of technology, and the technology education literature, suggests that technology education treats knowledge differently to many other disciplines. As a result of this, the interactions between teachers’ beliefs and knowledge are theorised to play a more significant role in influencing enacted practice in technology education. Building on this perspective, this article considers the need to investigate the roles of teachers’ knowledge and beliefs, and the interactions between these, in the investigation of enacted practice. Further to this, the article problematises the potential for a dominance of exploratory research, though acknowledging the need for research within different paradigms; a common frame of reference is advocated. In advocating a more holistic approach to investigating enacted practice, and the factors which may influence teachers’ enactment of teaching practice, it is envisioned that this article takes a step towards methodological coherence regarding the study of enacted practice in technology education.

This article describes a study undertaken in New Zealand, England and Sweden and is based on the use of a tool developed by the researcher as a professional development and teaching tool in technology education for teachers of students between four and six years of age. The aim of the research was to investigate teachers’ views of the effectiveness of the tool designed to deepening their understandings of technology content and pedagogical content knowledge. The tool, technology observations and conversation framework (TOCF) was designed to guide teachers’ interactions with and observations of young children when learning technology with the aim of developing teacher insight into their own understanding of technology and how students learn technology. The tool was developed using the building of learning power theory to facilitate the identification of key dispositions and attitudes within four aspects of learning and across five pre-determined behaviours relevant to technology education. Qualitative research methods were used to investigate teachers’ interaction with the TOCF by observing their use of it, and interviewing them about their perceived developed understanding of technology content knowledge and pedagogical content knowledge (PCK). The initial purpose of the framework, presented to the teachers prior to teaching, was to guide interactions with students to assist this development and subsequently assist their ability to teach technology effectively and give specific feedback to students in technology education. The study found that teachers felt that they gained a deeper understanding of technology education and their understanding of students’ learning in technology also developed. This article presents the final framework and teachers’ views on how they were assisted by the framework. The study offers an international perspective on ways to broaden and deepen students’ understanding in technological literacy through the development of teacher content knowledge and PCK and contributes significantly to the field of formative assessment in technology education.

Sharing research represents an essential part of the research process. The act of making one’s research public (or publication as we commonly refer to it) is so important, it can be argued, that research ceases to be research if it is not shared or published. A key reason for the sharing (or publication) of research is, and always has been, the betterment of society. This obtains particularly in education, which has as its central focus the advancement of the public good. Since it first started to see widespread use in education in the 1990s, the Internet has affected a transformative impact on research publication and sharing. Web technologies have made research available on levels never previously conceivable. Platforms such as Routledge Open Research can enable researchers to share their work at different stages in its development, and in innovative and imaginative, new ways. It can serve as an important stepping stone towards the appreciation and valorisation of a wider range of research outputs and resources, beyond the mono-media academic manuscript. It is our pleasure, as inaugural international academic advisors, to introduce this exciting new hub for research in education and educational technology, Routledge Open Research’s new collection, Digital Pedagogy. As a flagship collection, we hope it affords a new space for educators and researchers around the world to communicate, collaborate and share their concepts and ideas, methodologies, data, outputs and resources in dialogic ways, where the educational community at large can participate meaningfully and creatively in the constructive conversation that is the hallmark of all engaged research in education and digital pedagogy.