Operationalizing CAS standards for program evaluation and planning

The problem undertaken in this study was to determine the perceptions that chief student personnel administrators have regarding the structure, practices, procedures, and budget considerations of professional staff development programs in divisions of student affairs in selected public and private institutions;The basic problem involved in the investigation is the perceptions that chief student personnel administrators have of the factors which significantly impact staff development programs in public and private senior level institutions;The questionnaire was used to obtain the data to achieve the purposes of this study. The survey instrument focused upon 11 distinct areas of inquiry which indicate the manner in which chief student personnel officers perceive staff development programs in divisions of student affairs. Distinct areas of the questionnaire were demographic data, staff development defined, current active staff development programs, staff development policy, structure, focus of staff development, budget for staff development, frequency of staff development programs, factors motivating staff participation, program communication, and program and activity evaluation;The 2 x 3 factorial analysis of variance format was used to analyze continuous data and the crosstabs (Chi Square) format was used to analyze discrete data;A total of 402 questionnaires were disseminated to chief student personnel officers of comparable officials. There were 165 usable questionnaires returned which represented 41 percent of the sample; however, only 39.8 percent of the returned questionnaires were usable;Information obtained from the analysis of the data indicated that the highest degree offered in an institution significantly affected the nature of staff development programs in divisions of student affairs;Analysis of the data allowed the investigator to determine that staff development programs, when compared to other programs in the division of student affairs, was less than an average budget priority;Finally, three distinct models of staff development were developed as a result of this study. These models focused upon the staff development enterprise as it related to updating individuals based upon the institutional organizational structure, staff obsolescence, and the needs of a diverse student population.

Conveying the value of student affairs work doesn’t happen by accident or in isolation. And that's why Keith Rogers, D.B.A., PMP, Vice President for Student Affairs at the University of Nevada, Las Vegas, intentionally shares the stories and impact of his student affairs division and invests in key partnerships. He makes it a point to highlight how essential services outside the classroom directly support overall student success while boosting retention, persistence, and graduation rates. “By showcasing how student affairs contributes to academic outcomes and essential services, both campus and community partners gain a clearer understanding of the division's vital role,” Rogers said. “Articulating how programs and services foster a sense of belonging and community is critical in making the case for continued investment in holistic student support.”

SUMMARY This article intends to help school psychologists understand the nature of high stakes tests, methods for analyzing and reporting high stakes test data, standards for tests and program evaluation, and application of appropriate practices to program planning and evaluation. Although it is readily acknowledged that high stakes test data are not sufficient for effective program planning and evaluation, the availability of test results, and their salience for federally mandated accountability programs, argues in favor of using such data for program planning and evaluation. A decision-making model, which begins with high stakes test data, but also requires additional data from teachers and classrooms, is proposed to help practitioners evaluate program effectiveness, and make plans to improve student outcomes.

Effective program evaluation is a systematic way to improve and account for public health actions by involving procedures that are useful, feasible, ethical, and accurate. The framework guides public health professionals in their use of program evaluation. It is a practical, nonprescriptive tool, designed to summarize and organize essential elements of program evaluation. The framework comprises steps in program evaluation practice and standards for effective program evaluation. Adhering to the steps and standards of this framework will allow an understanding of each program's context and will improve how program evaluations are conceived and conducted. Furthermore, the framework encourages an approach to evaluation that is integrated with routine program operations. The emphasis is on practical, ongoing evaluation strategies that involve all program stakeholders, not just evaluation experts. Understanding and applying the elements of this framework can be a driving force for planning effective public health strategies, improving existing programs, and demonstrating the results of resource investments.

Background: The Program Evaluation Standards that were developed and approved by the Joint Committee on Standards for Educational Evaluation have served as a resource to the broader evaluation field for over four decades. However, little evidence has been collected regarding the extent to which the standards have influenced the field through scholarship or professional practice. Purpose: This study seeks to estimate the prevalence of the Program Evaluation Standards in evaluation scholarship and professional practice. Setting: Not applicable. Intervention: Not applicable. Research Design: The study combines a systematic review of evaluation literature with a survey of American Evaluation Association (AEA) and Canadian Evaluation Society (CES) members. Data Collection and Analysis: A systematic review of articles published in 14 evaluation-specific journals from 2010 to 2020 was conducted to identify and typify articles citing the standards. Additionally, AEA and CES members were surveyed, with a focus on knowledge and use of the standards. Descriptive analyses are presented to quantify the prevalence of the standards in evaluation scholarship and practice, respectively. Findings: The systematic review revealed that 4.48% of the 4,460 articles published in 14 evaluation-specific journals from 2010 to 2020 contained some use of the standards. Survey results show that 53.14% of AEA members and 67.12% of CES members are familiar with the standards and that, among those with knowledge of the standards, most AEA (67.67%) and CES (71.74%) members use them at least “occasionally” in their professional work, education, and scholarship activities.

This report is the fourth of four that stem from a study of the Impact of Approved Induction

Empowerment evaluation, objectivist evaluation, and evaluation standards: Where the future of evaluation should not go and where it needs to go

The Evaluation of Policies, Programs and Practices - Melvin M Mark, Jennifer C Greene and Ian Shaw An Introduction PART ONE: ROLE AND PURPOSE OF EVALUATION IN SOCIETY The Purposes of Evaluation in a Democratic Society - Eleanor Chelimsky Roles for Theory in Contemporary Evaluation Practice - Stewart I Donaldson and Mark W Lipsey Developing Practical Knowledge Evaluation for Practice Improvement and Organizational Learning - Patricia J Rogers and Bob Williams Evaluation and the Study of Lived Experience - Thomas A Schwandt and Holli Burgon Evaluation, Democracy and Social Change - Jennifer C Greene Evaluation after Disenchantment? Five Issues Shaping the Role of Evaluation in Society - Peter Dahler-Larsen PART TWO: EVALUATION AS A SOCIAL PRACTICE Government as Structural Context for Evaluation - Phil Davies, Kathryn Newcomer and Haluk Soydan The Social Relations of Evaluation - Tineke A Abma Intellectual Contexts - John Stevenson and David Thomas The Relationship between Evaluation and Politics - Ove Karlsson Vestman and Ross F Conner Ethics in Evaluation - Helen Simons A Comparative Analysis of Evaluation Utilization and Its Cognate Fields of Inquiry - J Bradley Cousins and Lyn M Shulha Current Issues and Trends Contextual Challenges for Evaluation Practice - Elliot Stern PART THREE: THE PRACTICE OF EVALUATION Methods for Policymaking and Knowledge Development Evaluations - Melvin M Mark and Gary T Henry Embedding Improvements, Lived Experience and Social Justice in Evaluation Practice - Elizabeth Whitmore et al Managing Evaluations - Robert Walker and Michael Wiseman Communicating Evaluation - Marvin C Alkin, Christina A Christie and Mike Rose On Discerning Quality in Evaluation - Robert E Stake and Thomas A Schwandt The Practice of Evaluation: Challenges and New Directions - Lois-ellin Datta PART FOUR: DOMAINS OF EVALUATION PRACTICE Evaluation in Education - David Nevo Evaluation of Health Services: Reflections on Practice - Andrew Long Social Work and the Human Services - Ian Shaw with Carol T Mowbray and Hazel Qureshi Evaluation in Criminal Justice - Nick Tilley and Alan Clarke Evaluation of Development Interventions and Humanitarian Action - Osvaldo Feinstein and Tony Beck Evidence-Based Evaluation in Different Professional Domains - Alan Clarke Similarities, Differences and Challenges

Editorial: Introduction to Special Section on Evidence-Based Practices for Persons With Intellectual and Developmental Disabilities

Background: Community organizations are essential in addressing health and social needs for populations often underserved by traditional healthcare systems. Recognizing the importance of program evaluation, these organizations increasingly seek to improve their services, demonstrate impact, and secure funding. However, they frequently face barriers such as limited resources, time constraints, and lack of in-house expertise. Evaluation training programs have been developed to bridge this gap to support community organizations in building evaluation capacity. Despite the availability of such programs, few undergo rigorous evaluation themselves, resulting in limited evidence of their effectiveness in fostering sustainable evaluation capacity within community settings. Addressing this need, this study presents key practices and findings from LaboEval, a five-year initiative launched in 2019 to strengthen evaluation capacity in community organizations across Quebec, Canada. Approach: LaboEval was developed through an iterative, participatory curriculum design process led by a steering committee of community representatives, students, and evaluation experts. The program offered seven online modules, each accompanied by follow-up coaching sessions grounded in a practical participatory evaluation framework. Modules covered essential topics such as stakeholder engagement, theory of change, data collection methods, knowledge transfer, and strategies for embedding evaluation within organizational practices. Implemented across 16 diverse organizations within the health and social services sectors, LaboEval emphasized flexibility to accommodate the schedules and needs of participants. Mayne's six-step contribution analysis was employed in the fifth year to assess LaboEval's impact on both individual and organizational evaluation capacities. A mixed-methods approach was applied, combining secondary data analysis, semi-structured interviews, document reviews, surveys, and literature reviews to ensure thorough data collection and analysis. Results: Findings indicate that LaboEval significantly enhanced individual and organizational evaluation skills and practices among participating organizations. Data reported increased confidence in conducting evaluations and improved integration of evaluation into their organizations’ processes. These improvements include more consistent use of logic models, incorporating evaluation activities into routine operations, and adopting appropriate data collection and analysis methods. The program's participatory design, which incorporated direct input from community stakeholders, contributed to its relevance, practicality, and success. Additionally, the modular structure and flexible implementation enhanced participant engagement and retention by addressing organizational constraints. However, challenges remained, particularly regarding the dissemination of evaluation findings by the organizations, which limited the broader impact of their evaluations on policy discussions. Implications: LaboEval offers important insights for both researchers and practitioners engaged in capacity-building initiatives. Its participatory curriculum design highlights the importance of involving organizations in the training program development process, enabling the creation of contextually relevant training that aligns with their specific needs and objectives. Contribution analysis provided a structured evaluation approach, yielding concrete, evidence-based insights into the effectiveness of training strategies and the pathways through which they build evaluation capacity. These insights illuminated specific outcomes, such as increased confidence and skills in evaluation practices, and highlighted how these pathways can refine future capacity-building initiatives and enhance their practical impact. The next phase of LaboEval will focus on expanding the program to additional organizations and enhancing mechanisms to support sustained evaluation practices.

This paper analyses the identity of auditing by comparing performance auditing to financial auditing and programme evaluation. Based on an analysis of textbooks, it is concluded that these evaluative practices are situated on a continuum. This implies that studies that rely on ‘audit’ as a label to attribute identity to a distinct evaluative practice become insensitive to issues concerning the relevance of their results to evaluative practices in general and their relation to specific characteristic of certain evaluative practices.

Higher education is evolving rapidly in today’s environment of global connectivity, technological advances and student diversity. The education industry must respond to these changes in order to remain viable. As a key division in the university, Student Affairs plays a vital role in helping universities respond to changes by ensuring that students’ needs are addressed effectively. To accomplish this, Student Affairs must continuously be able to improve by becoming skilled at creating, acquiring and transferring knowledge, in short, it must become a learning organization. The purpose of this study is to enhance the Student Affairs Division of Assumption University to become a learning organization by using organizational development methods including diagnostic procedures, organizational interventions and training programs. Qualitative and quantitative methods were utilized for this study. Data was collected through interviews and questionnaires. Pre-ODI analysis revealed that members of Student Affairs, although cognizant of the importance of some aspects of the learning organization, were not consistent in their behavior and actions. OD interventions were employed to increase both knowledge and behavior in regards to learning organization culture. They included team building, coaching, Appreciative Inquiry and SOAR workshops. Post-ODI analysis of the experimental and control groups using a paired samples t-test, supported by a calculation of Cohen’s effect size, revealed a significant difference in the total gain scores for Pre-ODI and Post-ODI results. The interventions were found to have an effective result in enhancing the learning organization culture in the Student Affairs Division of Assumption University.

A Development of Measurement and Evaluation Standards and Item Bank Approach Model for Teachers in Thai Secondary Schools

This work aims to assess standard evaluation practices used by the research community for evaluating medical imaging classifiers, with a specific focus on the implications of class imbalance. The analysis is performed on chest X-rays as a case study and encompasses a comprehensive model performance definition, considering both discriminative capabilities and model calibration. We conduct a concise literature review to examine prevailing scientific practices used when evaluating X-ray classifiers. Then, we perform a systematic experiment on two major chest X-ray datasets to showcase a didactic example of the behavior of several performance metrics under different class ratios and highlight how widely adopted metrics can conceal performance in the minority class. Our literature study confirms that: (1) even when dealing with highly imbalanced datasets, the community tends to use metrics that are dominated by the majority class; and (2) it is still uncommon to include calibration studies for chest X-ray classifiers, albeit its importance in the context of healthcare. Moreover, our systematic experiments confirm that current evaluation practices may not reflect model performance in real clinical scenarios and suggest complementary metrics to better reflect the performance of the system in such scenarios. Our analysis underscores the need for enhanced evaluation practices, particularly in the context of class-imbalanced chest X-ray classifiers. We recommend the inclusion of complementary metrics such as the area under the precision-recall curve (AUC-PR), adjusted AUC-PR, and balanced Brier score, to offer a more accurate depiction of system performance in real clinical scenarios, considering metrics that reflect both, discrimination and calibration performance. This study underscores the critical need for refined evaluation metrics in medical imaging classifiers, emphasizing that prevalent metrics may mask poor performance in minority classes, potentially impacting clinical diagnoses and healthcare outcomes. Common scientific practices in papers dealing with X-ray computer-assisted diagnosis (CAD) systems may be misleading. We highlight limitations in reporting of evaluation metrics for X-ray CAD systems in highly imbalanced scenarios. We propose adopting alternative metrics based on experimental evaluation on large-scale datasets.

Ethics is an important thread within higher education student affairs. Creating avenues for intentional, sustainable dialogues and consistent education for employees, one Ethics Officer at a four-year public university in the Northwest embarked on understanding the ethical climate to improve how student affairs professionals interact with students. The goal of this mixed-method design study examined the knowledge, skills, and attitudes of its division administrators about ethics, the ethical climate of a university student affairs division, and the design of an ethical training curriculum for division employees. This chapter reviews the current literature about ethics in higher education, organizational ethical climates, industry standards, and gaps in performance, while providing insights and ideas about best practices.