Annotated Bibliography of Resources and Research on Evaluations

Best practices on evaluations: Keeley, J. W. (2012) Choosing an Instrument for Student Evaluation. In M. E. Kite (Ed.), Effective Evaluation of Teaching: A Guide for Faculty and Administrators. Society for the Teaching of Psychology. pp. 13-21. 

An overview of general principles and best practices in the purpose, design, and implementation of student evaluations. Proposes both formative and summative methods in evaluations.

Basow, S. A., & Martin, J. L. (2012) Bias in Student Evaluations. In M. E. Kite (Ed.), Effective Evaluation of Teaching: A Guide for Faculty and Administrators. Society for the Teaching of Psychology. pp. 40-49. 

A summary of several research studies and insights on various sources of bias in student evaluations. Includes an overview of key cultural stereotypes about teaching and teachers that are coded along gender, race, age, sexual orientation, and other markers of identity.

Franklin, J. (2001) Interpreting the Numbers: Using a Narrative to Help Others Read Student Evaluations of Your Teaching Accurately. New Directions for Teaching and Learning, No. 87, Fall. pp. 85-100. 

Strategies for interpreting and summarizing what evaluations actually say about teaching. Includes summaries of common problems with evaluation design and traps to avoid in interpreting flawed evaluation data.

Lewis, K. (2001) Making Sense of Student Written Comments. New Directions for Teaching and Learning, No. 87, Fall. pp. 25-32. 

Provides a rudimentary qualitative analysis framework to summarize and interpret the content of comments. Some of the criteria used to characterize good teaching are a little out-of-date, however (from a handbook published in 1971), but the basic methodology can be adapted with better criteria.

Addendum: Kite, M. E. (2012) Effective Evaluation of Teaching: A Guide for Faculty and Administrators. Society for the Teaching of Psychology. 

This is the book containing the above chapters on designing evaluations (Keeley) and bias in evaluations (Basow & Martin). Other chapters in the book provide best practices for formative methods (mid-semester feedback, small-group feedback, pre-course surveys), supplements to evaluation forms (peer reviews by colleagues and/or instructional advisors), and more holistic approaches (teaching portfolios, course portfolios, samples of student work).

Research on bias: Linse, A. (2003) Student Ratings of Women Faculty: Data and Strategies. White paper, Center for Engineering Learning & Teaching, University of Washington. 

Outline of multiple research studies and findings on gender bias in student evaluations, including explanations for why studying gender bias poses a difficult methodological challenge.

Reid, L. (2010) The Role of Perceived Race and Gender in the Evaluation of College Teaching on RateMyProfessors.com.Journal of Diversity in Higher Education Vol. 3(3), pp. 137-152. 

A broad statistical analysis of student ratings on a popular website (ratemyprofessors.com), focusing specifically on small, selective, liberal arts colleges (Amherst College is included in this study's dataset). The study is limited by the nature of ratemyprofessor.com ratings (which are problematic and strongly linked to consumerist attitudes about teaching), but the overall findings point to negative implications for minority faculty in small majority-white colleges.

MacNell, L., et. al. (2015) What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching. Innovative Higher Education Vol. 40(4), pp. 291-303 

A small-scale study of evaluations gathered from four online sections of a fully-online summer course, showing differences in ratings across sections correlated with what students perceived/assumed to be the gender of the instructor. The methodology is limited by the very small sample sizes for each section. However the strength of the paper is the more complex analysis of gender in the setup of the study, especially in the literature review of how gender bias operates in academia more broadly.

Merritt, D. (2008) Bias, the Brain, and Student Evaluations of Teaching. St. John’s Law Review Vol. 82, pp. 235-288. 

Comprehensive review of the psychology connecting student evaluations and professors’ nonverbal behavior. Includes a critique of how evaluations are administered in ways that exacerbate cognitive biases. Proposes an alternate system for gathering feedback that uses facilitated small-group discussions structured so as to mitigate the effects of cognitive bias.

Literature reviews and annotated bibliographies: Bias in Student Evaluations of Minority Faculty: A Selected Bibliography of Recent Publications, 2005 to Present Auraria Library, Denver, Colorado Teaching Evaluations and Bias Carolina Women’s Center, UNC Chapel Hill Gender and Student Evaluations: An Annotated Bibliography (2001) Center for Research on Teaching and Learning, University of Michigan

Recent media overviews of research on gender bias: Gender Bias in Student Evaluations Jeanne Zaino, Inside Higher Ed, Feb. 23, 2015 Is the Professor Bossy or Brilliant? Much Depends on Gender Claire Cain Miller, New York Times, Feb. 6, 2015 Best Way for Professors to Get Good Student Evaluations? Be Male. Amanda Marcotte, Slate, Dec. 9, 2014