« Stupid Box Tricks for Intellectuals | Main | Questions that are rarely asked, a continuing series »
Rating RateMyProfessors.com
You’ve heard the reasons why professors don’t trust RateMyProfessors.com, the Web site to which students flock. Students who don’t do the work have equal say with those who do. The best way to get good ratings is to be relatively easy on grades, good looking or both, and so forth. But what if the much derided Web site’s rankings have a high correlation with markers that are more widely accepted as measures of faculty performance? Last year, a scholarly study found a high correlation between RateMyProfessors.com and a university’s own system of student evaluations. Now, a new study is finding a high correlation between RateMyProfessors and a student evaluation system used nationally.
Strike another victory for Web 2.0. Here is more.
Posted by Tyler Cowen on April 25, 2008 at 07:18 AM in Education | Permalink
Comments
...and another defeat for student evaluation of teaching?
Posted by: Ed Lopez at Apr 25, 2008 7:56:01 AM
Quick look at my alma mater suggests that the observed hot/easy/good correlation is much tighter in undergraduate programs than in graduate programs.
Posted by: Andrew Edwards at Apr 25, 2008 8:00:55 AM
"For his part, Sonntag acknowledged that some RateMyProfessors.com reviews are “so mean-spirited” that they aren’t worth anyone’s time. "
I think that this is the crux of the issue; professors don’t enjoy being insulted anonymously with no chance of correcting the statement. The main thing this study seems to uncover is that the same biases that ruin RateMyProfessors.com credibility ruin university-sponsored student evaluations as well. All RMP adds in are nasty and unmoderated statements. RateMyStudents.com is already taken, unfortunately, because I'd love to have a site where professors could offer the same evaluations of their students. Employers could probably make great use of the site, and the value of academic signaling would be increased, benefiting students who make an effort. Of course, some students would be outraged, because how could anyone dare say something bad about THEM!
Posted by: Aaron at Apr 25, 2008 8:30:43 AM
Um, don't professors already have an outlet for rating students?
Posted by: Josh at Apr 25, 2008 8:53:23 AM
I take great joy in the discovery there’s a high correlation between results of student evaluations and RateMyProfessor.com. Wow, Voice of the Customer comes to higher education, now if only someone will listen and respond accordingly.
I write the above as a 53 year old who last year returned to graduate school in the School of Public Policy at GMU; I attended a large public university as an undergraduate and a small private college for an MBA. I also have a daughter finishing her junior year at a small public liberal arts institution and so also have a parent’s perspective
My experience as an undergraduate student in the 1970s, graduate student in the 1980s and currently, alumni and current parent is that all higher education really cares about is accessing my wallet. Customer satisfaction (with customers being defined as students and parents) is an unknown philosophy, with perhaps the exception of dorm space, food service offerings and extracurricular offerings. How teachers actually teach is never addressed, except as a question to the tour guide on your typical undergraduate tour.
I love the feedback on RateMyProfessor; maybe the classifications could use some work and refinement but it’s a great start and there’s great performance information in there.
I can understand academicians don’t like it but it’s time for the academy to move out of the middle ages and compete with performance in the classroom with customer satisfaction one of the metrics.
Posted by: Dave Richardson at Apr 25, 2008 9:51:35 AM
The best way to get good ratings is to be relatively easy on grades, good looking or both, and so forth.
Last year, a scholarly study found a high correlation between RateMyProfessors.com and a university’s own system of student evaluations.
There is absolutely no conflict between these two statements.
Posted by: Anthony at Apr 25, 2008 10:08:41 AM
Don't student evaluations suffer from the same problem as the web site? Students who don't do the work can just as easily fill out the evaluation as log onto the site.
I'm finishing up my undergraduate studies, and have always tried to leave well-reasoned feedback on RateMyProfessor.com. However, there is a lot of "I got a bad grade and will therefore complain".
Posted by: Bo at Apr 25, 2008 10:14:45 AM
Bo wrote: "Don't student evaluations suffer from the same problem as the web site? Students who don't do the work can just as easily fill out the evaluation as log onto the site."
Exactly correct, however the good thing about RateMyProfessor.com is that it is not actually used to evaluate faculty performances by university administrations that are setting faculty salaries.
Grade inflation is the inevitable consequence of the use of student evaluations of faculty by university administrators. Grade inflation has gotten so bad that some universities have instituted grade "caps" of various kinds. At the university where I teach, student evaluations are used as the primary indicator of faculty performance. While I have no problem with them being used after adjustments are made for obvious control variables (such as students' SAT scores, to proxy ability, student attendance, to proxy their due diligence, and student grade in the class, to proxy their performance in the class), I think that using student evaluations as an unadjusted measure is simply contrary to what any student passing undergraduate econometrics understands: leaving out all control variables is not good practice.
Posted by: indiana jim at Apr 25, 2008 10:30:00 AM
Good news for Cowen and bad for Tabarrok.
I'd trust ratemyprofessors and especially pickaprof (which includes grading history for some schools) a lot more than the officially sanctioned feedback. Students don't trust the anonymity of the latter and don't want to rattle the cages of profs they might see again even if they do trust it. Rate and pick are intended for other students and not meant to be fed back into some bureaucracy where they'll do nothing.
Sure, they're gonna skew low: malcontents are more likely to post than champions. But malcontents often have good points.
Posted by: burger flipper at Apr 25, 2008 10:43:22 AM
I read my student evaluations through the typical means, but I refuse to look myself up on ratemyprofessor. After reading comments for other faculty I know to be good or great, students slam profs for fashion or say nasty things unrelated to the class.
Why subject my self to this.
Posted by: seth at Apr 25, 2008 10:48:54 AM
Student evaluations are nonsense.
At my university, we have group final exams. There are a few (hard) professors who get terrible evaluations. But on the group final (all sections of calculus X take identical finals, except for changed numbers/orders), their students tend to score 5-10% higher than the average.
One of them explained it to me: "my students hate me because they only understand 2/3 of what I say. But I say 2x as much as anyone else, so they come out ahead. They just don't know or like that."
Posted by: mathgeek at Apr 25, 2008 10:49:28 AM
What's up with the .4 hot/easy correlation?
Posted by: Douglas Knight at Apr 25, 2008 10:53:22 AM
I've read my own ratemyprofessor rankings and even the bad ones are pretty accurate. I've read those of my friends and enemies; also pretty accurate. The problem with ratemyprofessor is that the scale is inverted; easy professors get smiley faces hard ones not. This certainly drives off the lazier students, and depresses course enrolments in a potentially troubling way but no more so than gossip in the dorms. I also think this does not reflect the values of all students and even can act as a signal to those students who want to learn.
The problem is not with student evaluations it is with the way they are used by faculty. I have seen more than one colleague on a tenure committee pick a very witty turn of phrase and use it as the sole basis for a devastating attack on junior faculty member's teaching ability. In my own experience I would certainly like to throttle the single student who wrote that my philosophy of science class was dismissive of sociology. Not because sociology should not be dismissed but because two of the five people on my tenure review committee are from the sociology department.
Posted by: Kevin G at Apr 25, 2008 11:16:38 AM
Here are bits of five of the comments that are posted for me:
1) "AVOID AT ALL COSTS!!"
2) "The best grade on our first test was a 61%"
3) "His expectations are unreasonable because his philosophy is you need to get beat up in his class in order to not get beat up in the world. This guy is ****ed up. Do not take a class with him as your prof. I'm forewarning you. I'm an "A" student and got a C- in one of his classes."
4)". . . he's a good prof. Stick with it, read the book, DO THE HOMEWORK IF YOU WANT A GOOD GRADE ON THE TEST, and go to class every day, and you'll be fine."
5) "This class is cake as long as you go every day."
From the above I conclude what? Well, it seems that students who attend and do homework consider me a "good prof." whose class is "cake as long as you go every day." On the other hand, for those who don't want preparation for the world, I am "****ed up." So be it.
Posted by: indiana jim at Apr 25, 2008 11:31:07 AM
As an undergraduate student, I don't trust a damn thing of what anyone else says. Mostly because what everyone else says is highly variable, and young students tend to be highly emotional while criticizing/complimenting their professors.
I'd say a Rate My Professor rating isn't really useful until the student has had some time to digest what really happened. I absolutely hated my marketing teacher at the time I took the class, but I learned later on that he really helped us learn about the world
Posted by: Robert Olson at Apr 25, 2008 12:45:37 PM
right out of high school in my first undergrad program, I had no clue who were the good profs. in my late 20s in a 2nd undergrad program and then a grad program, I took professors not classes, and picked them using a combination of recommendations from 1) other students (the ones who asked intelligent questions in class and who were usually prepared) and 2) professors I had taken previously who I liked and who I believed knew their stuff.
I had a tax prof who few of my classmates liked, but he came highly recommended by a professor I respected. That tax professor was OUTSTANDING - he not only knew the code, he could talk about the leg history/policy behind it. He was one of the toughest profs I ever had, but he really brought the tax code alive for me.
Want to know who is is good in the classroom? Ask the students who you know are hard working and previous professors you respect.
Posted by: chug at Apr 25, 2008 12:45:42 PM
ratemyprofessor.com is a good website because professors aren't interested in real feedback from students, so don't listen or blow off criticism with the thought "he's MY student, what does HE know about teaching this stuff?" Well students may not have first hand knowledge of teaching, but they have first hand knowledge of learning. Most professors make their subjects unnecessarily hard because they don't want to see themselves as soft professors. This site isn't for the bad professors, but for students interested in knowing who the bad professors are.
I am in graduate school right now. I'm not sure how much of a difference there is between bad professors and hard professors. You'll get comments like this, from above:
"One of them explained it to me: 'my students hate me because they only understand 2/3 of what I say. But I say 2x as much as anyone else, so they come out ahead. They just don't know or like that.'"
This is pathetic and clearly shows that many professors DO NOT take pedagogic concerns into account, then rationalize their own foolishness. Teachers that do not know how to properly balance their own courses to make sure the amount being taught in the class isn't so overwhelming that the highest grade isn't 61% are just as bad as teachers that cannot explain things clearly. Making students unable to understand the material because you've decided to present WAY to much material, or my favorite, the same material "from alternative perspectives" (this one is especially idiotic since the ability to appreciate alternative perspectives DEPENDS on understanding at least one perspective all ready) is just as bad as not being able to clearly explain the material to your class.
Any student truly interested about a professor will use any means available to find out about a professor. Without fail, the worst rated professors really do turn out to be the worst professors. And if I have any doubt, I ask my fellow mathematicians, who will tell you straight up if the guy is bad, hard, easy, good to learn from, etc.
Again, websites like this seem to upset faculty and the administration so much because these sites are uncontrollable and the truth of the idiocy of higher education gets exposed. This is something they do not find amusing. Welcome to the information age, fellas.
Posted by: Ken at Apr 25, 2008 12:58:04 PM
I for one use ratemyprofessor all the time in picking classes. If a professor receives a lot of bad ratings from lazy students, there's a good chance he or she is a fine teacher.
Posted by: Michael at Apr 25, 2008 1:02:22 PM
Indiana Jim:
Are all of those comments from the same course? There is one professor at the university I attend with RMP reviews that average "Horrible course, horrible professor" for a 100 level gen-ed course and "Hard as hell, but you'll learn a lot" for a 300 level course.
Posted by: Thomas at Apr 25, 2008 2:07:30 PM
Thomas,
Comments 1, 2, 3, and 4 are for Intermediate Price Theory, and comment 5 is for Econ 116 (a beginning survey class).
Posted by: indiana jim at Apr 25, 2008 5:10:53 PM
Ken wrote: "websites like this seem to upset faculty and the administration so much because these sites are uncontrollable and the truth of the idiocy of higher education gets exposed. This is something they do not find amusing. Welcome to the information age, fellas."
Well Ken, I'm not upset in the slightest. In fact, I am happy for RateMyProfessor.com. When students post comments like this guy is "****ed up" they just make it clear that ad homenim is not below them. When the same student argues that I'm an "A" student and got a C- in one of his classes" this indicates that he/she either: 1) does not understand that by getting a C- he/she is no longer an "A" student (if he/she means a straight A student); or 2) he/she believes that because he/she has gotten A's in the past that he/she deserves them henceforth regardless of performance, field of study, input of effort, etc. Being hectored in such a way is a clear signal to thoughtful readers of the dubious intellectual integrity of the post.
No, I think that the posts can be highly revealing in a way that was in no way the intention of the person posting. This has the ring of the invisible hand.
Posted by: indiana jim at Apr 25, 2008 5:31:21 PM
I am sure that there is such a correlation. However, for professors who do not fit into a normal distribution,
small sample bias can have them way off. I will simply note that my overall ranking number is below the equivalent
number (we use the same scale) that I have received in any of the well over 100 classes I have taught, in which
all the students in the class were sampled. I actually called for the official evaluations to be made public
because of this problem, but needless to say that has not happened. So, as it is the 1 out of 20 students who
really, seriously hate my guts have gotten to have their overrerpresented voices drive the public image of my
teaching, and there is not a damned thing I can do about it. And, no, I am not going to tame down my wild man/
very tough grading approach that gets me those reactions from a small, but very vocal, minority of students.
BTW, another way to look at these is to do a ratio of the overall number to the "toughness" number, keeping in
mind that "1" is actually zero. That is a way of correcting for how toughness tends to drag down the overall number.
Posted by: Barkley Rosser at Apr 25, 2008 5:33:07 PM
Indiana Jim,
Are you a good teacher or not? This would help assess whether the comments are predictive or not.
Posted by: D at Apr 25, 2008 5:37:33 PM
I'm a seasoned student--several undergraduate institutions and graduate school--and in my experience the single best indicator of professor quality is simply whether or not they make their own lecture notes. The ones who do are nearly invariably above average, and the more detailed the notes the better the professor, the more sincerely and explicitly they wish to convey their body of knowledge.
Most students, though, evaluate professors based on about a 50/50 weight of whether or not they: 1) Enjoyed the class and 2) Got good grades. To me, this argues powerfully for the signaling model of education because post-enrollment, these are the two key variables that determine your utility gain from the class (1 determines cost and 2 determines the change in the strength of the ability signal). Neither has much to do with much you learned.
I should say--as someone whose gotten pretty good grades even in advanced, graduate level math classes--that most professors who are notably harder than other profs in their institution/department are just arrogant jerks. They think that there is some real level of difficulty that students ought to experience, but without standardized testing this is just sheer nonsense. You need a large comparison population in order to be able to say that a test really wasn't as difficult and that some students deserve low grades. All profs grade on a curve, the jerks just enjoy making the curves so that most students suffer.
As well, the profs who don't make lecture notes are also much more likely to be the hard-grading jerks. This is both because they are not nearly as good at their job as they think and because the lecture notes make information more available for students.
Most profs, though, don't make lecture notes or prepare for class much and delude themselves that it is somehow their students' fault when they're not understood.
Posted by: kadkool at Apr 25, 2008 6:41:39 PM
I'm a seasoned student--several undergraduate institutions and graduate school--and in my experience the single best indicator of professor quality is simply whether or not they make their own lecture notes. The ones who do are nearly invariably above average, and the more detailed the notes the better the professor, the more sincerely and explicitly they wish to convey their body of knowledge.
Most students, though, evaluate professors based on about a 50/50 weight of whether or not they: 1) Enjoyed the class and 2) Got good grades. To me, this argues powerfully for the signaling model of education because post-enrollment, these are the two key variables that determine your utility gain from the class (1 determines cost and 2 determines the change in the strength of the ability signal). Neither has much to do with much you learned.
I should say--as someone whose gotten pretty good grades even in advanced, graduate level math classes--that most professors who are notably harder than other profs in their institution/department are just arrogant jerks. They think that there is some real level of difficulty that students ought to experience, but without standardized testing this is just sheer nonsense. You need a large comparison population in order to be able to say that a test really wasn't as difficult and that some students deserve low grades. All profs grade on a curve, the jerks just enjoy making the curves so that most students suffer.
As well, the profs who don't make lecture notes are also much more likely to be the hard-grading jerks. This is both because they are not nearly as good at their job as they think and because the lecture notes make information more available for students.
Most profs, though, don't make lecture notes or prepare for class much and delude themselves that it is somehow their students' fault when they're not understood.
Posted by: kadkool at Apr 25, 2008 6:43:09 PM
In my experience, many of the ways that a professor can make a class more entertaining do not actually allow students to learn better or get more out of the course.
Thus you can't really be a hard-ass if you're going to do things like read interesting but unrelated journal articles to the class or tell stories about other professors because these things tend to be counterproductive to the goal of being a hard-ass.
It's a little better in advanced classes because the students have enough of a background in the subject to make the tangents relevant, but since the average intro class has many times the students of the average advanced class, it's the 100-level courses that form the backbone of a professor's reviews, not the 300-level courses. I found my electives to be much more rewarding than people who got their social science elective credits by taking stuff like intro to mass communication as a senior.
Posted by: Sean at Apr 25, 2008 7:33:32 PM
D and I.J.,
So, let me be specific, and you can judge for yourself. My overall rating on ratempdc
is 2.7. My easiness number is 1.9, giving me about a 2 to 1 ratio on that. My actual
overall general rating from my over 100 class averages in the official evaluations,
being kept secret for my "benefit," is over 4.0 definitely, although I have not calculated
it (we are talking about several thousand students, whereas the ratempdc score is based on
somewhat over 40). Many say I am "tough but fair," and each semester several say I am
"the best" they have ever had, while about 1 in 20 regularly excoriate me as much as they
can. The ones who like me a lot also show up somewhat on rate..., making my ratings a
rather sharply mixed bag. So there; go figure.
Posted by: Barkley Rosser at Apr 25, 2008 8:40:28 PM
Indiana Jim: "No, I think that the posts can be highly revealing in a way that was in no way the intention of the person posting. This has the ring of the invisible hand."
Of course they are. That doesn't change the aggregate of the information given on these sites. If a professor mostly gets a bad rating, then he is probably a bad professor and if a professor mostly gets good ratings, then he is probably a good professor.
A few angry comments doesn't detract from a sea of comments pointing out a bad professor. As I mentioned in my comment, I use ratemyprofessor to gauge a professor. If that professor is teaching a subject I am really interested in, I ask around to see if that professor is as bad as the site says, or if the ratings are the rants of a few angry students. I have yet to find that ratemyprofessor is wrong.
Teachers grade students all the time thinking they are grading only the student's performance, yet seem to be oblivious that they are concurrently testing themselves to see if what they think they've been teaching is actually what they've been teaching.
I think that your A student getting a C in a class is indicative of a bad teacher rather than a student feeling entitled to an A being pissed he didn't get an A. Consistently getting A's is comes from a strong work ethic and mind that doesn't go away upon entering a new class. Chances are the teacher taught poorly or tested on topics related to the material taught in class rather than the material itself or asked questions at a level much higher than was presented.
Kadkool is correct is saying "I should say--as someone whose gotten pretty good grades even in advanced, graduate level math classes--that most professors who are notably harder than other profs in their institution/department are just arrogant jerks."
Posted by: Ken at Apr 25, 2008 8:53:46 PM
Ken,
"I think that your A student getting a C in a class is indicative of a bad teacher rather than a student feeling entitled to an A being pissed he didn't get an A. Consistently getting A's is comes from a strong work ethic and mind that doesn't go away upon entering a new class."
Forgetaboutit; the inflation of grade that I have collected data on at indicates that getting A's need not come from either a strong work ethic or a "mind that doesn't go away."
Expectations are generally set by people relative to what they observe nearby. My department, economics, in entry level classes assigns grade that do NOT differ significantly from a 2.0. On the other hand, in beginning classes in marketing and management the average grade is closer to 3.0 than 2.0. This difference creates false expectations, obviously, and means that while your comment that A's come from hard work DOES apply to course in economics it DOES NOT elsewhere.
You may blog what you like ("I think that your A student getting a C in a class is indicative of a bad teacher"), but blogging something does not make it so; and it certainly does not in this case.
Posted by: indiana jim at Apr 25, 2008 9:30:52 PM
indiana jim: "blogging something does not make it so"
I couldn't have said it better myself. Of course this applies to you too. Whatever you have to tell yourself to make you think your a good professor.
Posted by: Ken at Apr 26, 2008 12:10:44 AM
I went to college before there were rating systems, but I remember most of my professors pretty well. I would have rated about half of them correctly (that is, I would still regard them as good or bad). The other half would have received highly inaccurate ratings: I would have overrated some professors that entertained me even if they didn't teach that much and I would have underrated professors that were hardasses and challenged everybody to achieve more than they thought were possible. After the fact, I gained much more from the demanding teachers I disliked at the time I took the course. Now I'm thankful for those people, but at the time I feared and disliked them.
I have been a professor for 30 years now, and have received good to excellent ratings the whole time. But I know I could make them higher by manipulating the system and pandering to my students. Some of my colleagues have cynically done so to their benefit. I cannot see how that is good for education generally.
Posted by: RJ at Apr 26, 2008 2:12:40 AM
Ken,
Of course I meant what I said and by no means want to be exempted; the way I blog is informed by this. That is why I allude to evidence and connecting ideas in a logical fashion (see the information and discussion of grade inflation in my above posts). In general, I do not find blog entries that merely express self-referential opinions to be either very interesting or informative.
Posted by: indiana jim at Apr 26, 2008 9:36:50 AM
Ken and others who think that ratemyprof is just fine,
Would you support open publication of official evaluations of professors that are full sample, all the students in the classes they teach? I do, and because I happen to know that what shows up on ratemyprof is in some cases a classic example of small sample bias. It is not always accurate of broader student opinion. It represents a self-selected ssmple, and anybody who knows anything about statistics knows that this can be very misleading.
Regarding whether being a "hard-ass" is a good thing or not, well, I am one, but I shall not get into this further other than to say to those who appear to be arguing for dumbing down material in courses apparently so that students can get good grades, are you really sure that this is what you think should be done? It is certainly my view of what has been done to a lot of textbooks in economics in recent years, especially at the Principles level.
Posted by: Barkley Rosser at Apr 26, 2008 10:15:20 AM
Barkley Rosser: Would you support open publication of official evaluations of professors that are full sample, all the students in the classes they teach?
Yes.
As for your hardass comment, I am not arguing for dumbing down material so students can get good grades. This post just happens to come out during a semester in which I have two particularly bad professors.
Currently, I am a grad student in applied math. I am taking an engineering course from a professor who is teaching from two sources, one of which seems inappropriate for an introductory class. Even at the graduate level there are books that are far more accessible and easier to learn from than other books, but teach the same material.
Math and engineering are hard subjects and can only be made so easy. I am not arguing in favor of dumbing down the material. I am arguing for the clearest and easiest presentation of the material. Many professors, mean spiritedly, intentionally present material in a more difficult manner than is necessary, with what seems to be the idea that 'if you can't handle this you shouldn't be here.'
My other teacher plays gotcha games, instead of testing knowledge of the material taught. He asks VERY open ended questions. For example on about half of the graded assignments, we've written code that generates numerical output. He then asks "What can you tell from this data?" Well there's usually 3 or 4 things you can say. If you happen not to write down the ONE that he is thinking of you get points marked off. Another thing he likes to do is take an excessive amount of points off. Failing to read his mind and write down what he's looking for on these open ended questions, I have more than 10% of the assignment grade deducted.
One of the professors was not on ratemyprofessor, but I was warned about him. The other is on the site with a poor rating. I took the classes anyway, like an idiot. Now I'm paying for it.
Posted by: Ken at Apr 26, 2008 12:01:59 PM
Barkley,
I have no problem with hard material, and I don't really have a firm opinion on what grade distributions should be in the first-best sense. That's a very complex question that requires balancing the signaling quality of grades, fairness to students, the role and purpose of the professor, etc. For instance, is it ethical for a single professor to give a grade distribution far below that of similar institutions just because he thinks that those are the grades that are "deserved?" I just plain dunno.
I just wish to emphasize a few things:
1) The level of difficulty of material and the grade distribution are separable. You can teach a really tough course but curve to all hell.
2) The prof who gives very low relative grades and blames it on the students seems to assume an unjustifiably privileged position for himself. He/she is administering a non-standardized test that he himself made up and assuming that he knows what the "right" grade should be. ALL PROFESSORS CURVE; it's just that the hard-asses like to curve downwards. If you want to say to a student that they deserve a low grade, you ought to be able to back it up by demonstrating that their score was low relative to some representative population test score. Otherwise, you’re basically just saying that they did low relative to the subjective standard that you yourself hold, and that cannot be questioned.
3) Almost all the hard-asses I’ve had (and I’m a pretty good student who has survived even the hard-asses), seemed to me to more or less be lazy (almost none of them made their own lecture notes and distributed them to the class (see my comment above)). How much students learn isn’t just a function of student effort, but teacher effort as well. Hard-asses more often than not assume that almost all the work should be done by the student. I will bet dollars to donuts that if the average hard-ass’s test were administered to a large, representative population that had time to study for it, grades would go up because other, good professors would find innovative ways to teach the material.
4) Hard-asses almost always see themselves as smarter than everyone else. Being profs, they’re certainly smarter than most people, but usually not as smart as they think. They think that students do badly because their stupid; indeed, most students are stupid, but in other areas of life even stupid people can learn a lot IF properly instructed. Stupid people routinely do better on the bar exam and myriad financial exams than they do on the average hard-asses test because even though the former test much more difficult and complicated material (on average), there are standardized, tried and tested ways of learning these things. Often, the hard-asses would just rather not be bothered with putting in the effort to figure out what the better methods are of teaching their material.
Posted by: kadkool at Apr 26, 2008 5:01:22 PM
Ken,
Sorry to hear that you are dealing with bad professors. With respect to math and economics, if you read "A Beautiful Mind" you will read about how Nash, of the Nash Equilibruim, gave unsolvable problems on exams that he wrote for students.
Kadkool,
I think that there are many "hard asses" who are the best professors you will ever have. The best fictioal movie example is Professor Kingsfield in The Paper Chase. Challenging students is part of a professor's job; it doesn't generally make me their buddy (as my brother-in-law always says "learning is an emotion experience." It depends what you mean by "hard ass"; I'll bet that your definition and Barkley's are quite different.
Posted by: indiana jim at Apr 26, 2008 9:18:29 PM
indiana jim: Sorry to hear that you are dealing with bad professors.
I appreciate the sentiment. It does come with the territory of being a student, though. I knew that upon returning to school, I just find it more frustrating than ever.
What I am so surprised at is just how bad some professors are. As an undergrad, I seemingly had unusual luck with professors, or was just too young to realize how bad professors were. I am in my 30's now and my patience with the arrogance of academia runs incredibly thin.
As a grad student, I have much for intimate contact with the professorship and am amazed at just how much professors look down on their students. Of course, this rubs off on the TAs, who do nothing but bash their students as well. They have the attitude that if students do poorly it is clearly the students fault. The fact that the grad students and professors have far more mathematical talent than their average student has no correlation whatsoever on their effectiveness at teaching. Yet they think this must be so.
Teachers generally speaking like to give themselves passes on the poor performance of their students, which I find to be disgusting. I am aware of the laziness of many students and that many bad grades or the result of this. I am not an A student because I prefer to hang out with my friends more than study, and many of my poor grades are my fault. But I am very indignant about the poor grades I receive from poor professors and grown more indignant as I get older. As with students, the following is true of teachers: half of them are below average. And as a lifelong student, I am aware of just how low the bar is for average for both students and teachers.
As with all people, teachers are incredibly immune to any sort of criticism, blaming all others (it clearly cannot be them) for the woes of their students performance. Sorry to rant and rave so much, but I truly find it irksome at the idea that poor performance lies mainly with the student. Any sort of thought given on the subject will lead to the conclusion that poor instruction makes up for half of the poor performance of students. After all, formal learning in a classroom setting requires good instruction and good study skills. Both are equally important in this setting.
Poor professors have their place because they force students to acquire skills needed for independent learning, but this is no excuse and some teachers are so bad that even this fails.
Posted by: Ken at Apr 26, 2008 11:55:04 PM
Market market market.
If a kid get the same A for hard class 101 as for easy class 101, why take the hard class?
No! kids who want a harder class choose a harder major, because it gets paid for in the real world. There is a sort of market for majors, but no market for classes within a major. So, shouldn't the professors attempt to equalize and adapt? If the department wants a particular class to be hard, shouldn't they have all the students pass through that class by their hardest professor, and then reward the professor inverse to his rating? Shouldn't ratemyprofessor.com disaggregate the professor from the course? In other words, It's not the student's problem. Will universities use the ratemyprofessor information, or will they continue to curse the darkness?
"I am in my 30's now and my patience with the arrogance of academia runs incredibly thin."
Ken, I feel your pain. When you are young, you assume it is you and all these eminent people know what they are doing. But after spending time in a professional environment, and then coming back to school, it becomes clear that most people in academia never did learn what it is to be a professional. High school dropouts at my old job were more mature than many professors at the university. Worse, you get treated just like all the other kids. Now, I wouldn't treat my toddler with as little respect as the adminstration and professors treat these young adults. And I certainly wouldn't treat someone like me that way unless I was completely clueless, emotionally stulted, and running a virtual monopoly system that was ripe for a shakeup.
By the way, I gave my hardest professor so far the most glowing recommendation. Why? Because he laid out his expectations, they were high, he was consistent, and he treated students respectfully. He had also worked in industry before returning to academia.
Posted by: Andrew at Apr 27, 2008 5:46:46 AM
Here's a rule of thumb for people blaming the students. Once you get more than 30 data points and you still think they are below average, you might want to doublecheck your assumed average.
Ken, check out phdcomics.com for a little therapy.
Posted by: Andrew at Apr 27, 2008 5:52:36 AM
After reading the other comments, I'd say profs need to just be thankful there is no "ratemygradschoolprof.com"
Or, god forbid, ratemyadvisor.com
Posted by: Andrew at Apr 27, 2008 6:03:35 AM
Okay, last thought. Universities could disaggregate the learning process from the accreditation function (grades). Will they do this? Group finals sounds like a start, but I'd bet they do this more for convenience and cost reduction rather objectivity.
A 300 level class may be too small for group finals, but what if you pooled the whole world and profs everywhere submitted questions? Presumably, if something is worth learning, other people are interested in it. This would also work to standardize the curriculum over time as well as approach better evaluation of the value of a department's degree.
Posted by: Andrew at Apr 27, 2008 6:14:13 AM
Andrew,
Why should any university use ratemyprof over official evaluations they make
based on the full sample of students taking each course? Should the latter not
be preferred for any professional teaching evaluation over the former, with its
potential small sample bias problems, which, I can assure you, do exist. And,
no, I am not miscalculating when I tell you my full sample numbers come way the
hell out higher than my ratemyprof ones, indeed, again, for the record, my supposed
overall average on ratemyprof, based on some 40+ students, is lower than any equivalent
number I have ever received in well over 100 courses that I have taught for over 30
years. Which are more reliable?
Now, I fully agree that there is a problem of information for students. That is
why I support public publication of official evaluations. Unfortunately, that is
not going to happen, at least not at most institutions.
Regarding this business of how hard ass to be and what does it all mean, I can
only say that I have in fact caved to some extent and relaxed my standards somewhat
over the years. This includes shortening the length of my reading lists, even though
the amount of information/readings/potential knowledge has actually been increasing.
But, even so, I see increasing complaints about how long my reading lists are, even
as they are getting shorter. I do try to present enough material so that even the
best student is still finding something "more" there to be learned.
So, as Indiana Jim has noted, there is a problem of grade inflation out there.
I happen to teach in economics, which tends to grade harder, partly because indeed
students in the subject are able to do better in the job market than for many majors.
But, I keep track, and make sure that I am not too far out of line with the distribution
of grades in my generally tough department, although I have always been near the lower
end of the distribution, hence, indeed a hard ass, for better or for worse.
Oh yes, on "gotcha" questions. I have been sometimes accused of asking those.
However, some of what appear to be those are actually designed to ferret out important
ideas, and they are also often the questions that separate out the A from the B students.
Of course, I agree that there are all sorts of questions of this sort that are
pointlessly difficult even to the point of being silly. And a lot has to do with the
attitude of the prof. Does s/he engage in a "gotcha" mentality or approach? I hope
that I do not do so, although I am sometimes called an "arrogant jerk," more often on
ratemyprof than in the actual evaluations, and also often here on this blog, where
generally it is usually deserved.
Posted by: Barkley Rosser at Apr 27, 2008 1:18:48 PM
Barkley: "Why should any university use ratemyprof over official evaluations they make
based on the full sample of students taking each course?"
Because everyone knows what hasn't been corrected on ratemyprof. That's one reason. It's been my opinion that student/employee surveys sponsored by the institution are more about making the underling feel like the institution cares. They don't, of course, until it gets so bad as to impinge on their survival instinct.
I also always felt like the surveys were done kind of helter skelter. My last couple semesters I made a point to write comments on a sheet of paper so I'd have them ready for survey day.
Also, everyone is talking about RMP as it is. Granted, it's kind of a joke right now, with the hotness or whatever. What about how good it is likely to get in a few years?
Everyone dealing with grading (on both sides) might benefit from reading a little Deming and quality control methodology. Why do you consider it "caving"? I think it should be more like changing the tolerances to match the process capability.
"Which are more reliable?" To who? The absolute results don't matter. What matters is where you are relative to your competition. People are smart enough to discount outliers. And when more data populates the database, people won't even need the comments.
Posted by: Andrew at Apr 27, 2008 2:40:59 PM
Andrew,
I do not know where you go to school, but where I teach, James Madison University, there is no
"correcting" of the evaluations by anybody. Does that go on where you are? How do you know if
it does? It is precisely that there is no "correcting" that they are not made public. A lot of
stuff gets said in them, some of which is verey valid and some of which is not true, just like on
ratemyprof, where there is zero correction for outright false statements, and I have seen such there.
Why is there any reason to believe that ratemyprof will improve? It will never be a full sample. It
will remain a sample of the self-selected, and so open to small sample bias. And, no, it is not the
case that the "relative" comparisons are the same there as in the actual full sample evaluations. That
is just wishful thinking on your part.
Regarding grading, I consider it "caving." Grade inflation is a bad social trend, but it is one that is
hard to resist, and one is put under pressure to cave. Why is there a trend? Part of it is student evaluations.
Just as on ratemyprof there is a strong correlation between that overall number and how easily one grades. There
are all-too-many institutions who rate teachers strictly and only on that overall score. Do I need to spell this
out further for you?
Personally, I did not cave because I wanted somebody to pat me on the back. I did so for the sorts of arguemnts
being made here, that one should not be too far out of line from reasonably comparable grade distributions being
given. But, for those of who have been around for some time and trying to resist the trend to grade inflation,
this is depressing and in the end amounts to feeling like one is caving, even if folks like you think it is just
great. Are you one of those who thinks everybody should get an A? That is the endpoint of the grade inflation process.
Posted by: Barkley Rosser at Apr 27, 2008 4:38:46 PM
The proper measure of a teacher is how much the students learn. Measuring this is not always easy, but no other measure makes any sense, because what the teacher is hired to do is to cause learning in the students.
Posted by: Bruce K. Britton at Apr 27, 2008 5:35:23 PM
Barkley,
I like the idea of a cap on the average grade in "core" classes; I think that about a 2.6 (where 2.0 is a C) would solve most of the grade inflation problem. I think that "honors" sections should be exempted, and also small classes (of size less than about 30) should be exempted from such a cap to avoid small sample bias problems. The reason I would put the cap at 2.6 rather than some number closer to 2.0 is to make it more politically palatable to the administrators at my institution who I have been trying to persuade to adopt such a cap for "core" classes. My argument is that the grades are signals that students use to assess their comparative advantages. It seems to me to be completely unprofessional to be providing signals that are as noisy as those at my institution (In some core classes the GPA is above 3.0).
Posted by: indiana jim at Apr 27, 2008 5:36:40 PM
I.J.
Something around 2.6 sounds pretty reasonable. I know programs and departments around
here (ones whose majors do not get paid too well), where the GPAs are now well in
excess of 3.5. My wisecrack about a convergence on 4.0 was not a joke.
Posted by: Barkley Rosser at Apr 27, 2008 8:27:46 PM
Barkley,
We are getting a new "core" that will be populated by many more competitors. My concern is that sans cap the convergence you are not joking about will, as they say in Star Wars, "make the jump into hyperspace".
Posted by: indiana jim at Apr 27, 2008 8:54:49 PM
Perhaps a good modification to the sight would be to require professors to provide an aggregate grade score for the class. Then, when each student signed in, they had to post their grade. With a minimum of say 15 or 20 students,an AVERAGE grade of those posting could be calculated. Now, in order to protect students and professors, the site could post a score showing the DIFFERENCE in values between the overall class grade average verses the average of those on the site. That way, complaints could be taken with a grain of salt, if there was a wide variation in overall scores the prof gave verses those complaining, winnowing out the 'sore looser' problem.
Thoughts?
Posted by: Tony Cohen at Apr 28, 2008 2:24:37 AM
Barkley,
I like you man, but I don't think you are trying to hear what I'm saying. Put your ear real close to the keyboard.
The correction I'm talking about is not data massaging. I'm talking about correcting problems uncovered by the surveys. I see little incentive to fix things based on the official survey. I figure the administration probably tries to use the survey results as an excuse to get rid of someone they don't like in the first place, post hoc justification. The administration, I'd suspect doesn't put teaching at the top of the priority list, at least in my field.
Being public and transparent, ratemyprofessor has grand incentive to improve itself, as well as to make the department improve problems that will become increasingly obvious or be remedied as more data rolls in. It will never have the same n as surveys that you take class time to force students to fill out, which will filter out the dispassionate, but I'm not sure that's a huge drawback.
Amazon's comment section is populated with self-selected entries, but they are more helpful to me than whatever internal process Amazon uses. And, as a customer, I get more books that I end up liking more. Good for Amazon, good for me, everybody happy.
As for grade inflation, what about standardized testing? I am not one who thinks everyone should get an A. But, if a school wanted to do that, I'd be fine to let them try it. Then, employers could factor that into their evaluation of the school (as they already do). Heck, maybe eliminating grade competition might improve learning. I don't know. Besides, we already have grade inflation. The only "should" I think I'm promoting is that people should recognize reality.
RMPdotcom will become an indispensible tool. Ignore it at your peril.
Posted by: Andrew at Apr 28, 2008 9:06:40 AM
"no other measure makes any sense, because what the teacher is hired to do is to cause learning in the students."
In my field, professors are hired to bring in research grant money. Teaching is something you do to justify the university's existence and supplement your income. This problem is a separate but related issue.
Posted by: Andrew at Apr 28, 2008 9:23:25 AM
Andrew,
You keep talking about "the administration" and "the deparment" along with references to what you have seen or do see. There is no "the administration" or "the department." Things are done differently in different places. Are you actually a teacher or have you ever been one, or are your remarks based on memories as a student from a particular department somewhere once upon a time? Your generalizations are remarkably empty.
So, frankly, more change does come from actual evaluations than from ratemyprof in most places, certainly where I am. But, this again depends on the case. For starters, most department head/chairs know that ratemyprof is an unreliable source. When there is a good department head/chair, they will/should discuss clear ongoing and chronic problems identified in student evaluations, the full sample ones. I know of cases where profs have in fact modified their teaching in response to some things, including even sometimes from reading the evaluations themselves without prompting from higher up. I have myself over time, without getting into details, although some comments I ignore (e.g., "his tests are too hard!"), and I imagine many others have as well. Just because you do not read news stories about or read of professors announcing how they are changing things because of student evaluations, real ones, not the yapping on ratemyprof, does not mean it does not happen. Do you know what you are talking about? Frankly, I do not think so. You make empty generalizations.
I do recognize, of course, that sometimes student evaluations get misused to go after somebody that the higherups do not like for other reasons. But that is not a reason to dispense with them, much less to substitute what shows up on ratemyprof for those full sample evaluations for the purpose of actual evaluation of teaching.
As for ratemyprof improving, why should they? As yours and others' comments have shown, all too many people think they are just a wonderful and perfect source, so why should they bother? I see zero pressure on them to do so. What are you suggesting that they do to "improve"?
Posted by: Barkley Rosser at Apr 28, 2008 12:25:54 PM
Do I have to be a teacher to understand the product? In fact, aren't non-teachers more objective about the service they provide? How long has it been since you were a student? I think between the two of us, I would have the authority in this argument.
And yes, things are different from place to place. But here's a generalization for you. If something works pretty well, chances are it has been copied, or needs to be copied, more or less.
Your over-specifics are remarkably empty. Without generalization, what do you have? Details that don't mean anything to anyone else. I'm overgeneralizing? Okay, fair enough.
"frankly, more change does come from actual evaluations than from ratemyprof in most places, certainly where I am"
Well, is that really saying much considering RMP has probably been around for 5 years, and only popular for about 2, while universities have held court for centuries?
People actually adjust to their reviews? Why? The goodness of their heart?
The last half of your response is a lot of strawman arguments against things I never said. Generalizations one might say. It would be really dumb for people to replace their current system with RMP, if the system is good. I never said RMP is great, just that it provides information and will provide more and more in the future. Its potential is great. It will change academia. That's a prediction you are free to disagree with.
Department heads who know it is unreliable better reassess every few years.
Can you see the official reviews of other professors in your department? Do you know where you stand in relation?
I never said they are a perfect source, but for some people for some information, they are the only source.
Posted by: Andrew at Apr 29, 2008 7:11:22 AM
"What are you suggesting that they do to "improve"?"
Well, since you asked...
I shouldn't have to be an expert on everything that sucks. But I try. And as you have pointed out, I'm not an expert in the inner workings of academic departments.
So, you discount my opinion because I'm not an expert and because you don't think I have specific improvement ideas. Okay, I'm used to it.
But here's a short list off the top of my head.
More data - this will happen over time and will ultimately rival your official reviews for quantity as well as quality.
The data could actually be better because it will not be done during class time but when students have time to concentrate and put thought into it.
It could also include students who dropped the course prior to the evaluation day which is usually the last day before exams.
Disaggregate professor performance from course difficulty - There's no point in slamming a professor who teaches a hard course that is required for a degree
A better system for quantifying the comments, maybe evaluating on more dimensions than a linear star system
A method for capturing teaching style so students can match their preferred learning style to a teacher's preferred style.
Less emphasis on the hotness thing, although if you don't like it, ignore it.
A method for evaluating the reliability of the student creating the evaluation.
Make people pay to post in some way, a nominal fee of some sort, which doesn't have to be monetary, would weed out a lot of the riff-raff.
Reward people for posts that other users find particularly accurate, useful, or insightful.
If one wanted to generalize, they could generalize all the same improvements to a lot of similar websites. But, that would require generalization.
Posted by: Andrew at Apr 29, 2008 7:26:10 AM
BTW, for me, RMP sucks because only a small fraction of my available profs are in they system.
But, considering how good it is already, contrasted with how bad it is now, virtually ensures dramatic improvement.
And RMP isn't just a site, it's a concept. If RMP doesn't improve, why wouldn't someone else overtake it. You said you taught economics, right? ;)
Posted by: Andrew at Apr 29, 2008 7:34:35 AM
Andrew,
I know what the rankings are in my department because I am a very senior member of my department, and
I have seen pretty much everybody's evaluations at one time or another when they went up for tenure or
promotion.
You should not be so cynical. Most of us actually do want to be better teachers, and thus are inclined
to make changes when there are consistent and reasonable suggestions being made. In some cases, especially
where there is a serious problem, the department chair or head may put stronger pressure for changes to be
made, although these are more likely to be successful with junior faculty or someone wanting a promotion, or
if pay is tied to the evaluations, which is the case in many institutions, and are more likely to be successful
if the professor in question is open to making changes rather than being forced to do so. The latter is rarely
very successful. I will say that for my own case, I have adjusted my teaching over the years in response to
consistent and reasonable, and sometimes not so reasonable, suggestions in evaluations. Heck, my cutting back
of my reading list, which I do not consider necessarily to be something good, has been largely done in response
to an increasing level of whining by students who do not wish to go to libraries.
I do not dispute that in the absence of the real evaluations being made public, RMP provides a useful
service for students. I do not disagree that there is a correlation between what one sees there and what
one sees in the fuller samples. It has been the repeated statements that I see from students who do not
know better asserting that RMP is fully accurate or possibly even better than official evaluations to which
I objected. You made many of these statements without any basis for doing so, and were wrong in doing so.
BTW, I know a lot of department chairs/heads, and all agree that there are some serious disjunctures between
what is on RMP and what one sees in the full sample evaluations. For a student without access to the full
sample evaluations, the former may serve a useful purpose. For a chair or head, they are a joke.
Posted by: Barkley Rosser at Apr 29, 2008 12:00:01 PM
If you don't like RateMyProfessors.com just use SyllabusCentral.com.
Posted by: SimplyStated at Aug 19, 2008 12:44:44 AM
If you don't like RateMyProfessors.com just use SyllabusCentral.com.
Posted by: SimplyStated at Aug 19, 2008 12:45:13 AM
My opinion is that reviewing professors is not nice or appropriate. If it is about education student should review classes, like at www.5caps.com. After all they need to know that the class is useful, not that “prof is hot”.
Posted by: at Nov 17, 2008 5:14:06 AM
