Time matters: the potential and pitfalls of using mixed methods approaches in longitudinal program evaluation

This report was prepared during the inception phase of the final evaluation of the Projet d’Appui au Développement Agricole pour la Nutrition et l’Entrepreneuriat (PADANE) coordinated by SNV and financially supported by the Embassy of the Kingdom of the Netherlands in Burundi. Once the outlines of the evaluation have been established, this report describes the phased and mixed methods approach - qualitative data collection and quantitative data analysis informing a 2nd phase of qualitative and quantitative data collection - developed to take into account the scope of the project and its objectives. The tools - simple and group interviews, observations, household surveys, sensemaking workshops and light contribution analysis - are described and linked to the 4 objectives, 6 evaluation questions and 25 evaluation sub-questions. Some operationalization aspects and the outline of the final report are also proposed in this document.

The main aim of this article is to discuss the factors a researcher should take into account when selecting the appropriate research design or method (i.e. qualitative, quantitative, and mixed-methods). The article also discusses sample size determination and sampling procedures in qualitative and quantitative research. Further, the paper has provided an examination of qualitative and quantitative data collection and analysis methods. The study used online desk research to collect data from online public access scholarly databases such as Google Scholar, ResearchGate, Academia, and many more freely accessible electronic books in research methods. Search terms that were used included, among many, the following: qualitative research, quantitative research, mixed-methods, qualitative and quantitative data collection, qualitative and quantitative data analysis; descriptive and inferential statistics. A wealth of relevant and timely scholarly literature was downloaded, read, and used to write this paper, and draw the conclusion provided. Keywords: Qualitative Research; Quantitative Research; Mixed-Methods Research; Qualitative Data Collection Techniques; Quantitative Data Collection Techniques; Qualitative Data Analysis Techniques; Quantitative Analysis Techniques; Descriptive Statistics; Inferential Statistics. DOI: 10.7176/JMCR/87-06 Publication date: November 30 th 2022

O86 - “I deserve to be in the loop”: A Mixed Methods Study of Young Women’s Experiences of Pregnancy Care in NSW

Chapter 28 - Best practices in mixed methods for pharmacy and health services research

Environmentally conscious construction has received a significant amount of research attention during the last decades. Even though construction literature is rich in studies that emphasize the importance of environmental impact during the construction phase, most of the previous studies failed to combine environmental analysis with other project performance criteria in construction. This is mainly because most of the studies have overlooked the multi-objective nature of construction projects. In order to achieve environmentally conscious construction, multi-objectives and their relationships need to be successfully analyzed in the complex construction environment. The complex construction system is composed of changing project conditions that have an impact on the relationship between time, cost and environmental impact (TCEI) of construction operations. Yet, this impact is still unknown by construction professionals. Studying this impact is vital to fulfill multiple project objectives and achieve environmentally conscious construction. This research proposes an analytical framework to analyze the impact of changing project conditions on the relationship of TCEI. This study includes green house gas (GHG) emissions as an environmental impact category. The methodology utilizes multi-agent systems, multi-objective optimization, analytical network process, and system dynamics tools to study the relationships of TCEI and support decision-making under the influence of project conditions. Life cycle assessment (LCA) is applied to the evaluation of environmental impact in terms of GHG. The mixed method approach allowed for the collection and analysis of qualitative and quantitative data. Structured interviews of professionals in the highway construction field were conducted to gain their perspectives in decision-making under the influence of certain project conditions, while the quantitative data were collected from the Florida Department of Transportation (FDOT) for highway resurfacing projects. The data collected were used to test the framework. The framework yielded statistically significant results in simulating project conditions and optimizing TCEI. The results showed that the change in project conditions had a significant impact on the TCEI optimal solutions. The correlation between TCEI suggested that they affected each other positively, but in different strengths. The findings of the study will assist contractors to visualize the impact of their decision on the relationship of TCEI.

BackgroundListening to “patient voices” in terms of symptoms, emotional status and experiences with care, is crucial for patient empowerment in clinical practice. Despite convincing evidence that routine patient reported outcomes and experience measurements (PRMs) with rapid feed-back to oncologists can improve symptom control, patient well-being and cost effectiveness, PRMs are not commonly used in cancer care, due to barriers at various level. Part of these barriers may be overcome through electronic PRMs collection (ePRMs) integrated with the electronic medical record (EMR). The PATIENT VOICES initiative is aimed at achieving a stepwise integration of ePRMs assessment into routine cancer care. The feasibility project presented here is aimed at assessing the knowledge, use and attitudes toward PRMs in a comprehensive cancer centre; developing and assessing feasibility of a flexible system for ePRM assessment; identifying barriers to and developing strategies for implementation and integration of ePRMs clinical practice.MethodsThe project has been organized into four phases: a) pre-development; b) software development and piloting; c) feasibility assessment; d) post-development. A convergent mixed method design, based on concurrent quantitative and qualitative data collection will be applied. A web-survey on health care providers (HCPs), qualitative studies on patients and HCPs (semi-structured interviews and focus groups) as well as longitudinal and cross-sectional quantitative studies will be carried out. The quantitative studies will enroll 600 patients: 200 attending out-patient clinics (physical symptom assessement), 200 attending inpatient wards (psychological distress assessment) and 200 patients followed by multidisciplinary teams (patient experience with care assessment). The Edmonton symptom assessment scale, the Distress Thermometer, and a tool adapted from existing patient reported experience with cancer care questionnaires, will be used in quantitative studies. A multi-disciplinary stakeholder team including researchers, clinicians, health informatics professionals, health system administrators and patients will be involved in the development of potentially effective implementation strategies in the post development phase.DiscussionThe documentation of potential advantages and implementation barriers achieved within this feasibility project, will serve as a starting point for future and more focused interventions aimed at achieving effective ePRMs routine assessment in cancer care.Trial registrationClinicalTrials.gov (NCT03968718) May 30th, 2019.

Mobile technology and its applications are among the technologies in education developing the fastest. They present an alluring chance to access numerous information sources at any time and location. In the study context, students who enrol on the Foundation program at Dubai Men’s College in the United Arab Emirates (the UAE) suffer from a cumulative lack of language proficiency in general and speaking proficiency in particular. The study’s primary purpose was to examine the impact of Mobile Assisted Language Learning (MALL) on students’ English language speaking skills. The study also examined how teachers perceived MALL’s effect on students’ speaking skills. The research question was: To what extent did students’ speaking skills in the experimental group develop through MALL compared to those in the control group? This subject was addressed using a mixed-method approach, including qualitative and quantitative data collection and analysis. A quasi-experiment of 66 students formed an experimental and a control group, and 20 teachers were involved. The results showed that the experimental group had statistically significant differences from the control group. The study emphasised the advantages of adopting MALL to develop students’ speaking skills, particularly those unmotivated.

Purpose The purpose of this paper is to study the design process innovations that enabled the successful delivery of a hybrid, mass-timber high-rise building in Canada, the Brock Commons Tallwood House at the University of British Columbia. It is one of a set of papers examining the project, including companion papers that describe innovations used during the mass timber construction process and the impact of these innovations on construction performance. Design/methodology/approach A mixed-method, longitudinal case study approach was used in this research project to investigate and document the Tallwood House project over a three-year period. Both quantitative and qualitative data collection and analysis techniques were used. Graduate student researchers were embedded within the project team to observe meetings and decision-making and to conduct periodic interviews. Findings The research highlights a case of a balanced triple-helix system that provided a context for the successful “clustering” of product and process innovation, which were developed and implemented to flow throughout the project’s lifecycle and across its supply chain to provide benefits at each stage. Four significant process-based innovations were implemented at the design phase of the building project to support radical product innovation: an integrated design process, virtual design and construction, designing for manufacturing and assembling and a rigorous quality control and quality assurance process. The product innovations developed through these process innovations were the structural system and the prefabricated envelope system. The context of innovation was seen to allow this “clustering,” which is believed to be a key condition of success and enabled the efficient and successful delivery of the project. Generally, the approach was successful; however, some factors including the number of stakeholders and good-faith collaboration may limit the replicability of these strategies. Originality/value This paper presents an in-depth investigation into the instantiation of an innovation system, identified as a balanced triple-helix system, which enabled and facilitated the design and decision-making process for a radical product innovation. Moreover, this paper describes the deployment of a “cluster” of process innovations that flowed throughout the project’s lifecycle and across the project supply chain. This was seen as a key factor in ensuring the successful delivery of the project.

This study aims to examine the effect of PAD, Balancing Funds in this case Revenue Sharing Fund (DBH), GeneralAllocation Grant (DAU), Special Allocation Fund (DAK), and Budget Financing Surplus (SILPA) on Capital Expenditures and the impact on the Economic Growth of the Regional Government of Kupang City, East Nusa Tenggara Province.This research used mixed methods, with sequential explanatory models, where data collection and quantitative analysis were carried out in the first stage by using secondary data. Population in this research are Statement of Budget Realization and Gross Domestic Regional Product Report by Kupang City Government with 13 years samples from 2005 to 2017. Then followed by collection and analysis of qualitative data in the second stage through interviews, in order to strengthen the results of quantitative research conducted in the first phase. To test and prove the research hypothesis, the analytical method used to test the effect of PAD, DBH, DAU, DAK, and SILPA on Capital Expenditure is Multiple Linear Regression and to examine the Impact of Capital Expenditures on Economic Growth, Simple Linear Regression is used.Based on the results of data analysis it can be concluded that through t test, partially PAD variable gives significant influence towards capital expenditure, it is showed by significant influence in amount of0.001 < 0,05 probability value. DAU gives significant influence towards capital expenditure, it is showed by significant influence in amount of 0.021 < 0.05 probability value. While DBH has no significant effect on Capital Expenditures with significant value of 0.381 > 0.05 probability value. DAK has no significant effect on Capital Expenditures with significant value of 0.219 > 0.05 probability value. SILPA has no significant effect on Capital Expenditures with sig value of 0.305> 0.05 probability value. While through the F Test PAD, DBH, DAU, DAK, and SILPA simultaneously gives significant influence towards capital expenditure with a value of sig 0,000 <0,05 probability value. Furthermore, through t Test Capital Expenditures gives significant influence towards Economic Growth of Kupang city with significant value of 0.010 <0.05 probability value. Key words: Local Own Sources Revenue, Revenue Sharing Fund, General Allocation Grant, Special Allocation Fund, Budget Financing Surplus, Capital Expenditure and Economic GrowthThis study aims to examine the effect of PAD, Balancing Funds in this case Revenue Sharing Fund (DBH), GeneralAllocation Grant (DAU), Special Allocation Fund (DAK), and Budget Financing Surplus (SILPA) on Capital Expenditures and the impact on the Economic Growth of the Regional Government of Kupang City, East Nusa Tenggara Province.This research used mixed methods, with sequential explanatory models, where data collection and quantitative analysis were carried out in the first stage by using secondary data. Population in this research are Statement of Budget Realization and Gross Domestic Regional Product Report by Kupang City Government with 13 years samples from 2005 to 2017. Then followed by collection and analysis of qualitative data in the second stage through interviews, in order to strengthen the results of quantitative research conducted in the first phase. To test and prove the research hypothesis, the analytical method used to test the effect of PAD, DBH, DAU, DAK, and SILPA on Capital Expenditure is Multiple Linear Regression and to examine the Impact of Capital Expenditures on Economic Growth, Simple Linear Regression is used.Based on the results of data analysis it can be concluded that through t test, partially PAD variable gives significant influence towards capital expenditure, it is showed by significant influence in amount of0.001 < 0,05 probability value. DAU gives significant influence towards capital expenditure, it is showed by significant influence in amount of 0.021 < 0.05 probability value. While DBH has no significant effect on Capital Expenditures with significant value of 0.381 > 0.05 probability value. DAK has no significant effect on Capital Expenditures with significant value of 0.219 > 0.05 probability value. SILPA has no significant effect on Capital Expenditures with sig value of 0.305> 0.05 probability value. While through the F Test PAD, DBH, DAU, DAK, and SILPA simultaneously gives significant influence towards capital expenditure with a value of sig 0,000 <0,05 probability value. Furthermore, through t Test Capital Expenditures gives significant influence towards Economic Growth of Kupang city with significant value of 0.010 <0.05 probability value. Key words: Local Own Sources Revenue, Revenue Sharing Fund, General Allocation Grant, Special Allocation Fund, Budget Financing Surplus, Capital Expenditure and Economic Growth <w:LsdException Loc

<h3>Background</h3> Ensuring effective access to vaccinations for people experiencing homelessness is crucial to protecting the health of a vulnerable, yet often overlooked population. Reaching this goal with COVID-19 vaccines takes more than a one size fits all approach. This study evaluates how a dedicated health team in Cork City collaborated with multiple agencies to register and deliver the COVID-19 vaccine to one of the most difficult to reach populations. <h3>Methods</h3> This mixed methods study includes the collection and analysis of quantitative and qualitative data and was co-designed with the Adult Homeless Integrated Health Team. Quantitative data on age, gender, nationality, language and accommodation status collected by the health team at the point of vaccine registration will be available for service users who participated in the vaccination programme. Semi-structured interviews will be conducted with staff and service users to explore their experiences with the pandemic and the delivery (staff) or receipt (service users) of the COVID-19 vaccine. The eleven agencies involved will be invited to participate in the interviews. A manager in each of the homeless agencies will ask service users within their agencies if they would like to participate in the study. <h3>Results</h3> The health team in partnership with all the agencies administered 728 vaccine doses to people experiencing homelessness in Cork City since 10th May 2021, 401 first doses were administered with a 95.2% uptake for the second dose. Study participants are mainly male (77%), middle aged (mean aged 41 years) and predominantly Irish (87%). Ten staff members who were involved in the vaccine rollout have agreed to participate in an interview and these interviews are ongoing. Initially, four interviews with staff have identified barriers for accessing mainstream vaccination services including addiction, mental health disorders and external stresses. Facilitators included organised transport, the help of key workers and trusting relationships between staff and clients. Face to face interviews with service users will commence in March. <h3>Conclusion</h3> This mixed method evaluation of an innovative approach to the delivery of vaccinations to very vulnerable and high-risk groups will provide insights into the design of services for the homeless population and will help inform future interventions for practitioners and policy makers, therefore improving health services for the homeless population.

The impact of online learning during the COVID-19 pandemic has significantly affected the activities, psychological conditions, and behavioral changes in adult learners, with technology-based learning becoming increasingly widespread in higher education institutions. Both professors and students have adapted to engage in distance learning interactions, including those at the Faculty of Tarbiyah, Islamic Institute of West Sumatra, who have become familiar with utilizing virtual classes through collaborative learning methods. This research aims to analyze the influence of implementing virtual classes with collaborative learning methods, student engagement, and the challenges faced by students in online learning on learning outcomes. The research method employed is a mixed-method convergent parallel design, involving the collection of both qualitative and quantitative data. Qualitative data analysis uses descriptive analysis, while quantitative data is analyzed using a cross-sectional study design. The research findings indicate that student learning through virtual classes at the Faculty of Tarbiyah, supported by platforms like Google Classroom, WhatsApp, and Zoom Meetings, has been effective and has positively impacted students' learning achievements. The challenges encountered by students and professors during online learning include technology mastery, cost, smooth communication, interactive success, internet signal, and time management. For the faculty members of Tarbiyah, it is recommended to conduct online learning based on virtual classes, involving various integrated platforms such as Google Classroom, WhatsApp, and Zoom Meetings. Additionally, preparing a comprehensive online learning guide with solutions to handle technical issues would be beneficial. This research contributes to the education field by providing insights into effective online learning practices and highlighting the challenges that need to be addressed to improve the overall learning experience.

Longitudinal research is well suited for investigating phenomena that change over time. With the growing acceptance of mixed methods, researchers are combining qualitative and quantitative approaches within longitudinal research. However, little attention has been paid to how researchers integrate longitudinal mixed methods databases. The purpose of this methodological review was to describe how researchers combine mixed methods and longitudinal approaches in practice and delineate dimensions and issues inherent within these complex designs. We examined published empirical studies from the health sciences that self-identified as longitudinal and mixed methods. Our results identify major dimensions, variations, and issues for designing longitudinal mixed methods research and suggest recommendations for researchers interested in using this complex approach.

The 1945 Constitution essentially guarantees that every citizen in national development in all aspects of life "promotes general welfare and social justice". Among the achievements of general welfare is the realization of better public health. The implication is that the life expectancy of the Indonesian population is increasing and the number of elderly or elderly people is increasing. However, the bad news regarding the Coronavirus Disease-19 (Covid-19) pandemic is that the elderly are vulnerable to being exposed to Covid-19. The data from the World Health Organization (WHO) shows that more than 95% of deaths occur at the age of 60 years or older (WHO, 2020). How is the elderly social protection provided by the government and society's elderly social institutions during the Covid-19 pandemi?, and what is the urgent need for these elderly social institutions in protecting the elderly during the Covid-19 pandemic? The methodology in this study uses a mixed method rapid study approach. Creswell interprets it as a research method characterized by the collection of quantitative data in the first stage, then followed by the collection and analysis of qualitative data in the second stage in order to support the results of the quantitative research conducted in the previous stage. The conclusion of this study states that the elderly have the right to obtain maximum protection. Ministry of Social Affairs R.I. as a government mandatory for the realization of the welfare of the elderly has an important and strategic role in this. For this reason, the Ministry of Social Affairs needs to continue to develop policies for the protection of the elderly according to contemporary dynamics. The study is an important instrument for developing policies, in this case the protection of the elderly. In this regard, the Center for Social Welfare Research and Development views the urgency of conducting a study on "Social Protection of the Elderly during the Covid-19 Pandemic in Government and Community Elderly Social Institutions" as an information and input for the development of the intended elderly protection policy.

This study aimed to explore the effect of teachers’ motivation on students’ academic performance in mathematics in public lower day secondary schools in Rwanda, specifically the case of Kirehe district. The study had three specific objectives which were to determine the influence of recognition and rewards of teachers on students’ academic performance in mathematics in public lower day secondary schools in Kirehe District, to determine the influence of in-service training of teachers in improving academic performance of students in mathematics in public lower day secondary schools in Kirehe District, to investigate the influence of teachers’ remuneration and incentives on students’ academic performance in mathematics in public lower day secondary schools in Kirehe District. The results of this study will support the Ministry of Education (MINEDUC), along with its bodies NESA and REB, which are responsible for monitoring and evaluating the academic performance of secondary schools, in making informed decisions on initiatives to address challenges faced by teachers and school management. Findings of this study will also enrich the existing knowledge and literature on teacher motivation, primarily benefiting academicians. One theoretical frame work, Self-Determination Theory underpinned and supported this research. In this research, descriptive survey design for quantitative and qualitative data collection and analysis were implied. The study gathered data through questionnaires and interview guides. Specifically, the questionnaires were made for Mathematics teachers while interview guides were designed for deputy head teachers and school head teachers. The study’s target population was 348 respondents made of 232 Mathematics teachers from Public lower day secondary schools, 58 deputy head teachers and 58 school head teachers from all public lower day secondary schools in Kirehe district, Rwanda. The selection of the participants for this study employed various methods, including both simple random and purposive sampling techniques. A sample size of 186 was chosen by using Yamane formula. The quantitative and qualitative data analysis in terms of frequency, percentage, mean, standard deviation, correlational analysis, Regression analysis, Analysis of Variance (ANOVA), and Regression Coefficient involved the use of Statistical Package for the Social Sciences (IBM SPSS 22.0) where each item from the questionnaires and interview guides was systematically categorized. Subsequently, the analysis revealed a strong positive correlation between students' academic performance in mathematics and teacher’s recognition and rewards (r = .901, p = .000). This suggests that when teachers are recognized and rewarded for their efforts, it positively impacts student achievement in mathematics. Additionally, a moderately strong positive correlation was found between students ‘academic performance in mathematics and in-service training of teachers (r = .701, p = .000). This implies that providing teachers with ongoing training and professional development opportunities contributes to improved student outcomes in mathematics. Finally, a moderately strong positive correlation was observed between students ‘academic performance in mathematics and teacher remuneration and incentives (r = .757, p = .000). This suggests that compensating teachers fairly and providing them with incentives can positively influence their effectiveness and, consequently, student achievement. The analysis demonstrates that all three teacher related factors: recognition and rewards, in-service training, and remuneration and incentives are significantly correlated with students' academic performance in mathematics in public lower day secondary schools in Kirehe district. To measure the results reliability, test and retest all methods were used in measurement of the reliability. The supervisor reviewed all data collection instruments items before the administration of data collection to ensure it validity. The research concluded by providing recommendations based on the findings of the study.

BackgroundChildren and adolescents with reproductive health issues may require immediate or emergency care, however in many countries availability of the pediatric and adolescent gynecology (PAG) service is low. That being said, teaching PAG examination to OBGYN, pediatrics and emergency medicine residents seems reasonable, and cannot be underestimated. In order to provide residents with opportunity to learn PAG examination, a high-fidelity hybrid simulation workshop was implemented in our institution.MethodsThe study aimed to investigate learners’ attitudes towards the high-fidelity simulation (HFS) hybrid model as compared with task trainer-SP (simulated patient)-voice model in the HFS environment and the factors connected to learners’ attitudes towards the hybrid model that could influence learning in high-fidelity simulation (HFS).The concept of attitude was used as the theoretical framework and the mixed method approach to study design was utilized with simultaneous collection of quantitative (original questionnaires) and qualitative data (semi-structured interviews).ResultsResidents valued the HFS hybrid model higher over task trainer-SP-voice model in regards to all three attitude components: cognitive (95%), affective (87.5%) and behavioral (83.7%). Analysis of qualitative data revealed six themes important to learners and informing learning of PAG examination in HFS. Further analysis of the themes allowed to develop a conceptual model, in which six factors connected to attitude components influenced learning. These factors were: task difficulty, attention, emotional realism of the simulation, patient’s emotions, physical realism of the simulation, and technical issues.ConclusionsParticipants of our study appreciated learning experience with the HFS hybrid model more, based on attitude questionnaire. Moreover, findings revealed that multiple, various factors connected to attitude may influence learning of PAG examination in HFS with hybrid model, and we propose a conceptual model illustrating relationships between those factors.