Special Education, Counseling, and Student Affairs

Findings from research on student ratings are summarized from the 1970s to 2013. There were 1,874 references, including 564 since 1994, using the ERIC descriptors “student evaluation of teacher performance” and “higher education.” The authors address the validity of self-report data, misconceptions about student ratings, essentials of credible research, and elements of reliability and validity. Evidence of reliability includes consistency, stability, and generalizability of ratings. Validity evidence consists of relations to other variables, including achievement; instructor self-ratings; ratings by administrators, colleagues, alumni, trained observers; and student written comments; as well as survey multi-dimensionality. Possible sources of bias are extraneous student, instructor, and course characteristics either unrelated or related to ratings. Few meaningful differences occur between ratings administered online versus on paper, and ratings in online versus face-to-face courses. Recommendations are made for the appropriate use of student ratings and for future research.

We summarize research on several aspects of student ratings of teaching that provide evidence of their reliability and validity. We also examine extraneous student, instructor, and course characteristics that are either unrelated or related to student ratings. Finally, we address ratings administered online versus on paper, ratings in online versus face-to-face courses, and the usefulness of student ratings.

Student ratings were compared in courses offered either exclusively online (n = 13,416) or face to face (n = 5,272). Data from 105 institutions were accessed from archived files of the IDEA Student Ratings of Instruction system. If a course was taught online, the instructor was less likely to lecture and more likely to use discussion, especially in hard disciplines. A course was less likely to have been taught online if the instructor was rated high in establishing rapport and if the course was within hard and pure disciplines. A high rating on structuring classroom experiences and expecting students to share in responsibility for learning increased the odds the course was offered online. However, high ratings on stimulating student interest and student effort in the course made it less likely. Results are discussed with respect to transactional distance elements of dialogue, structure, and learner autonomy.

Data from student ratings of instruction (SRI) are used ubiquitously as a key element in providing instructors with valuable feedback and evaluators with critical student input. Nonetheless, calls for the elimination of SRI continue to appear in academic journals and higher education periodicals. This paper brings to bear the huge body of research on SRI to which so many academics and institutions have contributed. Some of the most egregiously erroneous statements about SRI are rebutted with brief reviews of the readily available compelling evidence. Although some faculty frustrations about misuse of SRI are valid, we argue that inclusion of student voice is critical. Students can provide useful feedback because they have firsthand experience over multiple occasions of what actually occurred in the classroom. Recommendations are made for best practices in using SRI as one of many sources for improving and evaluating teaching. 

This study examined which teaching methods are most highly correlated with student progress on relevant course objectives in first- and second-year (lower-level) general education courses. We specifically sought to identify teaching methods that distinguish progress made by students taking a general education course from that made by students taking a course in their major. Our main focus was on IDEA learning objectives aligned with the Association of American Colleges and Universities (AAC&U) Liberal Education and America’s Promise (LEAP) Essential Learning Outcomes.

In this study, we asked what effect survey length has on student non-response rates to individual items on IDEA’s Diagnostic Feedback (DF) and Learning Essentials (LE) forms. Because the DF has more items than the LE (41 vs. 19), users sometimes ask whether the non-response rates to individual items are higher on the DF. We found that about 99% of students respond to all forced-choice questions regardless of whether instructors use the DF or LE. Non-response bias most likely, then, imposes very little threat to student responses on individual items, and should not affect overall teaching evaluations. We can most likely conclude the missing responses occur when students elect not to advance to the next page in the survey.

We compared scores in classes administering IDEA student ratings of instruction on paper (N = 651,587) versus online (N = 53,000). Although response rates were higher wth paper delivery, class response rate was not meaningfully correlated with student ratings in either format. Overall, we found no meaningful differences in ratings between delivery methods.

The purpose of this report is to summarize results from statistical analyses comparing differences on student and faculty responses between paper and online administration of the IDEA Student Ratings of Instruction System (IDEA) from 2002 to 2008.