Mitigating gender bias in student evaluations of teaching.

Women are underrepresented as instructors in engineering, computing, and technology classes. One factor that disadvantages women in the classroom are student evaluations of teaching (SETs), as research finds they contain significant gender bias. This may contribute to the dearth of women in computing education, as SETs are used in decisions about contract renewals, hiring, tenure, and promotion. The double-bind is one cause of gender bias in SETs, meaning that it is more difficult for women than for men in leadership positions (such as being a professor) to be perceived as both competent and likable. We examine a lightweight intervention's impact on gender bias caused by the double-bind. Specifically, we conducted a field experiment in which the woman professor of a CS1 class for non-majors gave students in the intervention condition additional, positive exam feedback via email. We hypothesized this would increase students' perceptions of the professor's likability, which would then increase her SETs. We find that the intervention increased top-performing students' ratings of the professors' likability. We also find that the professor received significantly higher SETs the semester she sent the intervention emails. While women should not have to alter their behavior to accommodate students' gender biases, this intervention may be a useful survival strategy for women impacted by gender bias in SETs.

Much research has been undertaken on gender bias in student evaluations of teaching (SETs) in universities, with inconsistent findings. We undertake a qualitative analysis of the comments in four years of SETs in a school of political science and international relations in a highly regarded Australian university. We ask, can the same evaluations produce different results when analysed qualitatively rather than quantitatively? And do students evaluate male-identified and female-identified teachers differently, and if so what are the differences? We show that qualitative analysis can reveal gender bias that is invisible in quantitative analysis. We find that female-identified staff are evaluated more positively than their male counterparts for undertaking time-intensive, stereotypically feminine, emotional labour. Male-identified staff are evaluated more positively for their technical expertise and teaching style. This suggests SETs evaluate gender-stereotypical behaviour rather than only teaching quality, and has significant implications for their use in universities.

In a widely cited and widely talked about study, MacNell et al. (2015) [1] examined SET ratings of one female and one male instructor, each teaching two sections of the same online course, one section under their true gender and the other section under false/opposite gender. MacNell et al. concluded that students rated perceived female instructors more harshly than perceived male instructors, demonstrating gender bias against perceived female instructors. Boring, Ottoboni, and Stark (2016) [2] re-analyzed MacNell et al.’s data and confirmed their conclusions. However, the design of MacNell et al. study is fundamentally flawed. First, MacNell et al.’ section sample sizes were extremely small, ranging from 8 to 12 students. Second, MacNell et al. included only one female and one male instructor. Third, MacNell et al.’s findings depend on three outliers – three unhappy students (all in perceived female conditions) who gave their instructors the lowest possible ratings on all or nearly all SET items. We re-analyzed MacNell et al.’s data with and without the three outliers. Our analyses showed that the gender bias against perceived female instructors disappeared. Instead, students rated the actual female vs. male instructor higher, regardless of perceived gender. MacNell et al.’s study is a real-life demonstration that conclusions based on extremely small sample-sized studies are unwarranted and uninterpretable.

This paper tests how gender stereotypes may result in biased student evaluations of teaching (SET). We thereby contribute to an ongoing discussion about the validity and use of SET in academia. According to social psychological theory, gender biases in SET may occur because of a lack of fit between gender stereotypes, and the professional roles individuals engage in. A lack of fit often leads to more negative evaluations. Given that the role as a lecturer is associated with masculinity, women might suffer from biased SET because gender stereotypes indicate that they do not fit with this role. In two 2 × 2 between groups online experiments (N's = 400 and 452), participants read about a fictitious woman or man lecturer, described in terms of stereotypically feminine or masculine behavior, and evaluated the lecturer on different SET outcomes. Results showed that women lecturers were not disfavored in general, but that described feminine or masculine behaviors led to gendered evaluations of the lecturer. The results were especially pronounced in Experiment 2 where a lecturer described as displaying feminine behaviors was expected to also be more approachable, was better liked and the students rather attended their course. However, a lecturer displaying masculine behaviors were instead perceived as being more competent, a better pedagogue and leader. Gender incongruent behavior was therefore not sanctioned by lower SET. The results still support that SET should not be used as sole indicators of pedagogic ability of a lecturer for promotion and hiring decisions because they may be gender-biased.

ObjectiveAsking students to answer 30 + questions in an optional student evaluation of teaching (SET) increases the likelihood that instructors may not receive critical feedback due to survey fatigue. A considerable body of educational research on how to incentivize student completion of SETs offers a variety of suggestions to combat fatigue, perceptions of value, and other issues.MethodsThis retrospective cohort study from January 2021 to March 2024 investigates the impact of a shortened, 4-question mandatory SET on response rates and qualitative feedback within a Clinical and Translational Science (CTS) program. Unadjusted linear mixed-effects regression models assessed the relationships of SET type (optional SET vs experimental SET) and SET response outcomes.ResultsOver the course of the study, 844 students completed optional (N = 473) and experimental (N = 371) SETs. Requiring SET completion to access assessments increased both participation (Estimate 62.43, 95% CI: 52.41-72.45; P < .001) and qualitative responses (Est. 43.65, 95% CI: 32.24-55.05; P < .001). Relative to the optional SET, the experimental SET was associated with more blank qualitative responses (Est. 7.28, 95% CI: 4.70-9.86; P < .001) and a higher total word count (Est. 989.59, 95% CI: 513.78-1465.40; P < .001). Course satisfaction scores did not differ significantly between survey types (Est. 0.05, 95% CI: −0.13-0.22; P = .60).ConclusionThis study's findings demonstrate that short, mandatory, learning management system-integrated SETs effectively increase qualitative feedback and can offer a feasible model for institutions aiming to gather actionable items to inform course design and delivery.

Student evaluations of teaching (SETs) can provide valuable assessment information. However, online administration of SETs has corresponded with lower SET completion rates. We examined perceptions of strategies for increasing SET completion rates and the effect of offering psychological prompts (completion goals, reminders, progress feedback) and micro-incentives (-extra credit that has a negligible effect on student grades) on SET completion rates, grades, and student evaluations. Study one (N = 469 instructors) revealed that instructors who used incentive-based strategies (compared with instructors who did not) rated such strategies as more ethical and more effective, and reported greater likelihood of using incentive-based strategies in the future. Examination of student records revealed higher SET completion rates among instructors who offered micro-incentives than among instructors who did not. In Study two (N = 36 instructors), we randomly assigned instructors to one of three conditions: psychological prompts, micro-incentives plus prompts, or a no-information standard-practice control condition. SET completion rates were lowest in the control condition (53.9%), followed by the prompts (64.5%) and micro-incentives plus prompts (79.7%) conditions. The two experimental conditions did not differ in completion rates, but only micro-incentives differed significantly from the control. These results demonstrate that prompts and incentives are promising methods for increasing SET completion rates.

The literature on student evaluations of teaching (SETs) generally presents two opposing camps: those who believe in the validity and usefulness of SETs, and those who do not. Some researchers have suggested that ‘SET deniers’ resist SETs because of their own poor SET results. To test this hypothesis, I analysed essays by 230 SET researchers (170 lead authors) and classified the researchers as having negative, neutral or positive attitudes towards SETs. I retrieved their RateMyProfessors.com (RMP) scores and, using logistic regression, found that lead authors with negative attitudes towards SETs were 14 times more likely to score below an estimated RMP average than lead authors with positive attitudes towards SETs. Co-authors and researchers with neutral attitudes, on the other hand, did not significantly differ from the RMP average. These results suggest that personal attitudes towards SETs may drive research findings.

The validity of traditional opinion-based student evaluations of teaching (SETs) may be compromised by inattentive responses and low response rates due to evaluation fatigue, and/or by personal response bias. To reduce the impact of evaluation fatigue and personal response bias on SETs, this study explores peer prediction-based SETs as an alternative to opinion-based SETs in a multicultural environment. The results suggest that statistically significant fewer respondents are needed to reach stable average outcomes when peer prediction-based SETs are used than when opinion-based SETs are used. This implies that peer prediction-based SETs could reduce evaluation fatigue, as not all students would need to do each evaluation. The results also report that the peer prediction-based method significantly reduces the bias evident in the opinion-based method, in respect of gender and prior academic performance. However, in respect of the cultural variables, race and home language, bias was identified in the peer prediction-based method, where none was evident in the opinion-based method. These observations, interpreted through the psychology literature on the formulation of perceptions of others, imply that although peer prediction-based SETs may in some instances reduce some personal response bias, it may introduce the perceived bias of others.

Student evaluation of teaching (SET) is widely used to assess teaching effectiveness in higher education and can significantly influence professors' career outcomes. Although earlier evidence suggests SET may suffer from biases due to the gender of professors, there is a lack of large-scale examination to understand how and why gender disparities occur in SET. This study aims to address this gap in SET by analyzing approximately 9 million SET reviews from RateMyProfessors.com under the theoretical frameworks of role congruity theory and shifting standards theory. Our multiple linear regression analysis of the SET numerical ratings confirms that women professors are generally rated lower than men in many fields. Using the Dunning log-likelihood test, we show that words used in student comments vary by the gender of professors. We then use BERTopic to extract the most frequent topics from one- and five-star reviews. Our regression analysis based on the topics reveals that the probabilities of specific topics appearing in SET comments are significantly associated with professors' genders, which aligns with gender role expectations. Furtherly, sentiment analysis indicates that women professors' comments are more positively or negatively polarized than men's across most extracted topics, suggesting students' evaluative standards are subject to professors' gender. These findings contextualize the gender gap in SET ratings and caution the usage of SET in related decision-making to avoid potential systematic biases towards women professors.

Student evaluation of teaching (SET) is widely used to assess teaching effectiveness in higher education and can significantly influence professors’ career outcomes. Although earlier evidence suggests SET may suffer from biases due to the gender of professors, there is a lack of large-scale examination to understand how and why gender disparities occur in SET. This study aims to address this gap in SET by analyzing approximately 9 million SET reviews from RateMyProfessors.com under the theoretical frameworks of role congruity theory and shifting standards theory. Our multiple linear regression analysis of the SET numerical ratings confirms that women professors are generally rated lower than men in many fields. Using the Dunning log-likelihood test, we show that words used in student comments vary by the gender of professors. We then use BERTopic to extract the most frequent topics from one- and five-star reviews. Our regression analysis based on the topics reveals that the probabilities of specific topics appearing in SET comments are significantly associated with professors’ genders, which aligns with gender role expectations. Furtherly, sentiment analysis indicates that women professors’ comments are more positively or negatively polarized than men’s across most extracted topics, suggesting students’ evaluative standards are subject to professors’ gender. These findings contextualize the gender gap in SET ratings and caution the usage of SET in related decision-making to avoid potential systematic biases towards women professors.

Student evaluation of teaching (SET), one of the most widely used mechanisms for evaluating teaching worldwide, is arguably gender-biased, potentially obstructing women’s access to and advancement in academia. To date, a significant number of studies have focused on interpreting a difference in SET scores by gender as evidence of gender bias. The validity of inferences from many of these studies is in jeopardy if they lack a strong research design that effectively isolates various potential confounding factors. We conducted a systematic literature review of experimental and quasi-experimental studies testing the claim that SET scores are biased against women. Our search identified 21 relevant studies published between 2000 and 2021. We qualitatively analyzed the supporting theories that predict gender-biased SET scores, the study design features used to estimate the instructor’s gender effect, and the findings across studies. The two most dominant explanations across studies are gender stereotypes and expectancy violations theory. We found ingenious methods for simulating a lecture and manipulating the instructor’s gender. SET scales across studies feature up to five types of distinctive questions about teaching performance. Findings across studies suggest that women are more likely to receive lower SET scores than men. However, this review reveals that examining gender bias is far more complex than individual studies portray. We note that the lack of strong supporting theory and tensions in research designs affect the construct and external validity of findings across studies. We discuss implications for fair evaluation of teaching that protect the development of women in academia.

Student evaluation of teaching (SET) has become a ubiquitous feature of higher education. The attainment and maintenance of positive SET is essential for most teaching staff to obtain and maintain tenure. It is not uncommon for teachers to receive offensive and non-constructive commentary unrelated to teaching quality. Regular exposure to SET contributes to stress and adversely impacts mental health and well-being. We surveyed Australian teaching academics in 2021, and in this paper, we explore the perceived impacts of SET on the teaching and learning experience, academic standards and quality. Many respondents perceived that SET contributes to an erosion of standards and inflation of grades. A thematic analysis of open-ended questions revealed potential mechanisms for these impacts. These include enabling a culture of incivility, elevating stress and anxiety in teaching staff, and pressure to change approaches to teaching and assessment to achieve the highest scores. Playing the SET game involves balancing a commitment to quality and standards with concessions to ensure optimal student satisfaction. Anonymous SET is overvalued, erodes standards and contributes to incivility. The process of SET needs urgent reform.

Within the health professions education system, a significant proportion of teaching and learning occurs in the clinical setting. As such, the need to measure effective teaching for accreditation standards, faculty development, merit pay, academic promotion, and for monitoring the safety of the learning environment has led to numerous universities developing instruments to evaluate teaching effectiveness in this context. To date; however, these instruments typically focus on the student perspective, despite evidence demonstrating that student evaluations of teaching (SETs) lack correlation with learning outcomes and are not a true measure of teaching effectiveness. This issue is further exacerbated in small health professional training programs, such as genetic counseling, where clinical teachers may only supervise 1-3 students per year. As a result, not only are SETs more confounded due to small sample sizes, but a direct conflict exists between respecting learner anonymity and providing timely and relevant feedback to faculty. In such contexts, even using SETs to evaluate the nature of the learning environment may be unreliable due to student concerns about identifiability and fear of retaliation for unfavorable evaluation. This paper will review the literature regarding SETs, barriers to this process within the clinical setting, and the unintended downstream consequences. Options for addressing issues related to the use of SETs will be considered, with particular focus on the process of reflection and the use of teaching consultations or peer support groups as a means to improve teaching effectiveness in this learning environment.

The AACSB, AECC, AAHE and other institutions have recently suggested that colleges and universities reassess their commitment to teaching. Reliance on student evaluations (SEs) of faculty teaching has been criticized in the literature because many SEs include items that students cannot properly assess and exclude demographic and contextual questions that are known sources of response bias. This study surveyed administrators of accounting programs from a cross-section of schools and programs to determine whether there is an association between the weight given teaching and the weight assigned to SEs. The results show a statistically significant inverse relationship between the weight given SEs and the emphasis placed on teaching.

Students assessment of the prosthodontics theoretical teaching - APRD- Print ISSN No: - 2581-4796 Online ISSN No:- 2581-480X Article DOI No:- 10.18231/j.aprd.2019.002, IP Annals of Prosthodontics and Restorative Dentistry-IP Ann Prosthodont Restor Dent