|
|
"Viewing student ratings as data rather than as evaluations may also help to put them in proper perspective. Writers about faculty evaluation are almost universal in recommending the use of multiple sources of data. No single source of data, including student rating data, provides sufficient information to make a valid judgment about teaching effectiveness."
William E. Cashin, Kansas State University. Management Newsletter, Vol.4, No. 1, September 1990.
---------------------------
see also http://userwww.sfsu.edu/%7Eperttula/refer.html
---------------------------
manipulating the ratings http://insidehighered.com/news/2007/10/18/sweets
---------------------------
.......on administering the questionnaire to the students:
Evaluation should be administered by a RESPONSIBLE STUDENT, GRADUATE ASSISTANT, or COLLEAGUE; NOT BY THE INSTRUCTOR. (excerpt from memo of December 5, 1998]
The College of Science and Engineering Committee to Study Student Evaluation of Teaching
(Paul R. Barnes, Cliff Berkman, Susann Novalis, and Wolf Stadler)
recommendation to the college dean, James C. Kelley.-------------------------
In 1987 Feldman reviewed more than 40 studies of the relation between faculty productivity, or scholarly accomplishment and instructional effectiveness (as perceived by students).
The average correlation between scholarly productivity or accomplishment and instructional effectiveness was +.12.
Feldman concluded that "in general, the likelihood that research productivity actually benefits teaching is extremely small or [alternatively] that the two, for all practical purposes, are essentially unrelated."
K.A. Feldman "Research Productivity and Scholarly Accomplishment of College Teachers as Related to Their Instructional Effectiveness: A Review and Exploration" Research in Higher Education, Vol. 27, 1987, pp. 227-298.
--------------------------
"The Validity of Use of Ratings in Personnel Decisions
The authors of the articles in this Current Issues section agree that student ratings are the most valid and practical source of data on teaching effectiveness. But, as I noted earlier, these data must then be interpreted by faculty or administrators who must make decisions about promotions and merit pay increases.
I contend that the specific questions used, the use of global versus factor scores, the possible biasing variables, and so forth are relatively minor problems. The major validity problem is in the use of the ratings by personnel committees and administrators (Franklin & Theall, 1989)"
from Student Ratings, The Validity of Use by Wilbert J. McKeachie, University of Michigan. American Psychologist, Vol. 52, No.11, 1218-1225. November 1997.
The Franklin & Theall reference is from a paper at the annual meeting of the American Educational Research Association, April 1989. J. Franklin and M. Theall. Who read ratings: Knowledge, attitude and practice of users of student ratings of instruction.-----------------------------
Do Good Looks Equal Good Evaluations? Chronicle of Higher Education, October 15, 2003
"In their study, Mr. Hamermesh and Ms. Parker asked students to look at photographs of 94 professors and rate their beauty."
see article
-----------------------------
Overview of Relationships Found Between Students' Ratings and Background Characteristics
Prior subject interest
Classes with higher interest rate classes more favorable, although it is not always clear if interest existed before the start of the course or was generated by the course or the instructor
Expected grade---actual grade
Class-average grades are correlated with class-average students' evaluations of teaching, but the interpretation depends on whether higher grades represent grading leniency, superior learning, or preexisting differences.
Reason for taking a course
Elective courses and those with a higher percentage of students taking the course for general interest tend to be rated higher.Workload---difficulty
Harder, more difficult courses requiring more effort and time are rated somewhat more favorably.
Class size
Mixed findings but most studies show smaller classes are rated somewhat more favorably, although some find curvilinear relationships where large classes also are rated favorably.
Level of course or year in school
Graduate-level courses are rated somewhat more favorably; weak, inconsistent findings suggest upper division courses are rated higher than lower division courses.
Instructor's rank
Mixed findings but little or no effect.
Sex of instructor or student
Mixed findings but little or no effect.Academic discipline
Weak tendency for higher ratings in humanities and lower ratings in sciences, but too few studies to be clear.
Purpose of ratings
Somewhat higher ratings if ratings are known to be used for tenure-promotion decisions.Administrative conditions
Somewhat higher if ratings are not anonymous and the instructor is present when ratings are being completed.
Students' personality
Mixed findings but apparently little effect, particularly because different "personality types" may appear in somewhat similar numbers in different classes.
This overview taken from Making Students' Evaluations of Teaching Effectiveness Effective, The Critical Issues of Validity, Bias, and Utility, by Herber W. Marsh and Lawrence A. Roche, University of Western Sydney, Macarthur. American Psychologist, Vol. 52, No. 11, 1187-1197. November, 1997.
------------------------------"A number of years ago, Sherman and Blackburn (1975) found that personality and teaching effectiveness were higher correlated. They suggested at the time that a professor "wishing to improve his perceived effectiveness may best begin on personal attributes rather than focus his energy on course functions and activities which, on the surface, seem more readily open to alteration" (p.130). The findings of this study reinforce that conclusion."
A quote from the authors of a study that examined the ratings in 17 sections of marketing core courses taught over a period of two years by five different instructors. And ..."The personality of the instructor was related to all other variables and was almost twice as strong of a predictor of the final evaluation as any other factor."
Student Evaluations in Marketing: What is Actually Being Measured? By Dennis E. Clayson and Debra A Haley. Journal of Marketing Education, Fall 1990. 9-17.
[Barbara R. Sherman and Robert T. Blackburn (1975). Personal Characteristics and Teaching Effectiveness of College Faculty. Journal of Educational Psychology, 67(1):124-31]
----------------------------"Measures of teacher effectiveness based on test scores of pupils are neither reliable nor valid enough to tell us anything about how competent a teacher is."
Measurement-Based Evaluation of Teacher Performance, An Empirical Approach. Donald M. Medley, Homer Coker, Robert S. Soar. 1984, Longman: New York. p.49.
---------------------------
University of Washington references to student ratings: http://depts.washington.edu/cidrweb/RatingsTools.htm
----------------------------
Center for Research on Learning and Teaching, University of Michigan
http://www.crlt.umich.edu/ [specific reference: http://www.umich.edu/%7Eeande/tq/index.htm ]
----------------------------
for a long excerpt from an article summarizing findings related to college teachers:
College Faculty As Teachers by Martin J. Finkelstein
[these notes were assembled by Bill Perttula for the use of his faculty colleagues at San Francisco State University. Please send corrections or suggestions to perttula@sfsu.edu]