Designing Educational Project and Program Evaluations

Applicability of standards for evaluations of educational programs, projects and materials in an international setting

In 1980, a Joint Committee appointed by 12 organizations1 concerned with educational evaluation issued one of the most significant documents to date in the field of educational evaluation. It consisted of a set of 30 standards to be used both to guide the conduct of evaluation of educational programs, projects, and materials and also to judge the soundness of such evaluations. The document, entitled Standards for Evaluations of Educational Programs, Projects, and Materials, 2 was published in 1981 by the McGraw-Hill Company. The standards were the result of an extensive developmental process, which involved the work of about 200 people and required more than four years to complete. The committee’s work did not stop with the publication of the standards, and it presently works to promote sound use and conducts a process of ongoing review and development. This chapter briefly describes the development of the standards and the nature of the standards, and summarizes them for the reader.

Rubrics are well-established tools used in a variety of educational settings, such as student assessment, teacher performance, and curriculum review. This study investigates the extent to which and how rubrics are being used in program evaluation. After exploring the background, or etymology, of the word rubric, a review of literature is conducted. Results reveal that rubric use in program evaluation is relatively rare, although increasing. Rubrics are predominately used in education and health program evaluation to transform data from one form to another, to characterize organizational functioning, and to derive explicitly evaluative conclusions. Program evaluators use rubrics during data collection and data analysis study phases, and to synthesizing findings into conclusions. This paper is the first systematic study of the use of rubrics in program evaluation. It presents a picture of how program evaluation practitioners and scholars are using or discussing rubrics.

Not available

Program evaluations are routinely conducted in most educational institutions. This article reports the use of focus groups in the evaluation of a program of school library education at the University of Alberta, Edmonton, Canada. The program evaluati

Evaluations of Educational Practice, Programs, Projects, Products, and Policies

Stakeholders in early childhood (EC) education have a responsibility to construct and implement programs and evaluation methods that are developmentally appropriate and culturally responsive. Currently, widely used paradigms for preschool and other EC program evaluations are predominantly quantitative and unable to provide stakeholders with a holistic picture of an early education program’s impact within the crucial context of a child’s and family’s lived experiences. This chapter provides an overview of lessons learned from a past mixed-methods evaluation for Elevate PreK (EPK) in Northern Arizona and how these lessons informed the shift to a primarily ethnographic evaluation approach exploring program meaning and impact. The authors first provide a summary of their past approach to evaluation to introduce the authors’ shift to ethnography. The authors then outline the development of evaluation strategies, established through orienting themselves toward a constructivist and interpretivist way of thinking where the authors saw program meaning and impact as something that can be constructed through deeper, more intentional interactions. Data were primarily collected using classroom observations of teachers, observations taken during child drop-off and pick-up hours and family–teacher conferences, feedback solicited from teachers on curricular use, and interviews with families about their experiences in the program. Preliminary analyses have uncovered and reinforced valuable findings regarding the positive impact that the program has had on children, families, and teachers. The insights gleaned from this evaluation indicate the great benefit of implementing ethnographic and other qualitative methodological approaches in evaluating EC programs to understand these programs’ direct and indirect impacts on their beneficiaries.

Objectives The purpose of this study is to develop an evaluation plan by applying the CIPP model to the KAS2022 and then explore the possibilities that emerge when applied to the actual evaluation of education programs. Methods To solve the first research question, we collected evaluation criteria by providing an overview of the CIPP model, analyzing previous research related to the evaluation of the Korean medicine college educational program, and analyzing previous research related to the evaluation of medicine college education program applying the CIPP model. To solve the second research question, a draft evaluation plan applying the CIPP model to KAS2022 was developed and reviewed by two professors of Korean medicine and one professor of medicine. Afterward, limitations were examined by comparing the final draft of the evaluation plan with the evaluation criteria presented in eight previous studies. To solve the third research question, the computerized system for curriculum evaluation developed by S University were compared with the evaluation plan developed in this study. Results In this study, even though it is not presented in KAS2022, intermediate and final inspections criteria were developed on the activities planned for the educational program, and evaluation standards were developed to ensure the effectiveness of the educational program evaluation rather than just formality. Meanwhile, as a result of analyzing the improvements in the computer program dedicated to S University's curriculum evaluation through comparison of evaluation plans. Conclusions Based on the research results, this study is a follow-up study: first, a case study to apply this evaluation plan to the evaluation of an Korean medicine college education program and explore implications, second, an IPA research of the evaluation plan developed in this study, third, proposed a study on technical problems that may arise when adding menus to supplement the computer program dedicated to curriculum evaluation.

Review of Spector, J.M., & Yuen, A.H.K. (2016). Educational technology program and project evaluation. New York: Routledge

This chapter describes two cases that have applied the proposed strategy: an evaluation of the Eisenhower Professional Development Program and a series of evaluations of Title I of Elementary and Secondary Education Act. It discusses implications for the future of large-scale evaluations of funding stream programs in education. Evaluations of large-scale federal education programs began in the mid 1960s. For example, the federal government has funded a number of evaluations of Title I. Evaluating the impact of funding stream education programs has posed difficult intellectual and methodological challenges. Growing numbers of researchers and policymakers have been calling for more experimental studies in education, which should eventually lead to a larger pool of evidence about the impact of educational interventions. Evaluators of large education funding streams may have to supplement the existing research base with studies of impact that look at how proven interventions are implemented in a broad array of real-world settings.

Policy and program decisions involve choices among different ways of preparing teachers. These choices are shrouded in increasingly contentious debates as teacher shortages reach crisis proportions. Yet, research on special education teacher education is almost nonexistent. Findings from comparative research documenting the characteristics of effective teacher education programs can inform these choices, but these findings should be grounded in what we know from previous research in general teacher education. To assist educators, we have analyzed literature in general and special teacher education toward two ends. First, we present a framework, derived from work in general education, for analyzing teacher education programs. Second, we use this framework to analyze practice in teacher education in special education. Specifically, we conducted an exhaustive review of special education program descriptions and evaluations. We conclude by describing steps necessary to improve the special education teacher education research base.

This report is the fourth of four that stem from a study of the Impact of Approved Induction

Many financial education providers still do not have a basic level of evaluation capacity and are unable to identify program outcomes and design effective evaluation instruments. It is difficult to propose a national evaluation strategy without a basic understanding of current evaluation capacity and of the critical gaps in program evaluation. In addition, there has been little discussion about the challenges facing financial professionals and educators who are on the “front lines” delivering and evaluating programs. The purpose of this survey article is to address these critical gaps in the literature and to provide an overview of the current state of financial education and program evaluation. Using qualitative and quantitative data collected from financial professionals and educators nationwide, this study provides insight into what can be done to build national evaluation capacity and conduct more effective program evaluations.

Evaluation of educational programs is an activity that must be carried out in every educational institution, this is done so that every educational institution can improve and develop a predetermined planning process, especially in educational programs. In this study, we will discuss the evaluation of educational programs carried out at SMP IT Nurul Ilmi. The research methodology carried out in this study used descriptive qualitative research methods, namely describing the results of research findings in the form of observations and interviews. The results of the research process said that the evaluation of the education program carried out at SMP IT Nurul Ilmi went well because before carrying out the evaluation process for the education program at SMP IT Nurul Ilmi carried out a planning process with the principal and teachers so that in the process of implementing the evaluation of the education program running well. So in conclusion, the evaluation of educational programs conducted at SMP IT Nurul Ilmi runs according to predetermined procedures by carrying out the planning process and predetermined regulations.

Effective measures of knowledge about mental health in later life are valuable in both needs assessments and educational program evaluations. Described is the development of two short, true/false quizzes, one on alcohol problems and one on depression and suicide in later life. These quizzes can be used to plan and evaluate community education programs on mental health problems in later life. Procedures for developing other knowledge assessments are presented.