Portrait of a Teacher

Bias of Student Evaluation of Teaching

18. April 2024
Logo: Podcast Lehrpersoenlichkeiten
Lehrblick - Lehrportraits
Bias of Student Evaluation of Teaching

In today’s podcast episode, we have a conversation with Bob Uttl, diving into the world of student teaching evaluations. We’ll explore what’s possible and, most importantly, discuss the boundaries of what these evaluations can really tell us.

Bob Uttl is a professor of psychology at Mount Royal University (MRU) in Calgary, Canada. He is a cognitive psychologist with main research interests in memory, intelligence, aging, assessment and psychometrics. As a (co-)author of several high impact papers, he  examined the validity and biases of SET ratings. Professor Uttl’s SET-related research has been featured in a number of news outlets including Inside Higher Ed, University Herald, The Guardian and elsewhere.

Time Stamps

01:26Bob, in your work on student evaluation of teaching, you’ve been criticizing the use of standardized instruments for measuring teaching effectiveness. Could you expand what arguments and findings led you to this conclusion?
03:58So intelligence is the most important factor that influences the learning outcomes. Would you agree?
04:33However, would it be possible that lecturers might influence the students’ interest in the subject they’re teaching and so they would indirectly be kind of responsible or able to influence the learning outcomes?
05:05Are there are some other bias that could influence or distort the results of student evaluation?
07:10That would apply to mandatory courses, I think. They should be poorer rated than courses the students can choose by themselves and by their interest, right?
07:43In one of your texts, you mentioned a study suggesting that incentives like giving chocolate can influence student evaluation. How would you explain such strange findings? And what psychological factors could be swaying students’ opinions in this particular instance.
10:39If I summarize for the moment, it can be said that there is no objective method for measuring teaching quality. However, SETs then perhaps measure how well teachers manage to meet students’ expectations in good quality teaching?
11:15Do students know their grades in Canada or at your University before they are doing the SETs?
11:48The American as well as the German Sociological Association, suggest using SETs not for summative evaluation but for formative evaluation. They recommend it as an opportunity to gather student feedback, their point of view on different aspects of a course. Would you agree to such scenarios?
13:17Can we identify and measure teaching behaviors that clearly signal ineffectiveness?
13:38In your latest text you summarize some of them as an indicator for ineffective teaching. That might be the basis for the deans to talk about what happens in this class, or to identify ineffective lecturers, right?
15:32Okay, so instead of trying to measure teaching effectiveness, you would suppose to identify more ineffective teaching. So to flip it around.


American Sociological Association (2019). Statement on Student Evaluations of Teaching. www.asanet.org/sites/default/files/asa_statement_on_student_evaluations_of_teaching_feb132020.pdf

Deutsche Gesellschaft für Soziologie (2020, September 3). Stellungnahme der Deutschen Gesellschaft für Soziologie zum Umgang mit studentischen Lehrveranstaltungsevaluationen. https://soziologie.de/aktuell/stellungnahmen/news/stellungnahme-der-deutschen-gesellschaft-fuer-soziologie-zum-umgang-mit-studentischen-lehrveranstaltungsevaluationen 

Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007 

Uttl, B. (2023). Student evaluation of teaching (SET): Why the emperor has no clothes and what we should do about it. Human Arenas. https://doi.org/10.1007/s42087-023-00361-7

Creative Commons Lizenzvertrag
Bias of Student Evaluation of Teaching ist lizenziert unter einer Creative Commons Namensnennung 4.0 International Lizenz

You Might Also Like

No Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.