Arthur Poropat, who lectures in applied psychology at Griffith University, recently wrote an interesting post about how “students don’t know what’s best for their own learning”. This matters. As Poropat explains:
Universities and governments around the world rely on student evaluations to assess university teachers and degrees. Likewise, potential students check online ratings when deciding where to study.
These evaluations are based on the logic that students must know best what helps them learn.
So it’s surprising to discover that students may be the worst people to ask about the quality of education. Two recent studies of student evaluations clearly demonstrated this point. Both studies looked at student evaluations and learning. Both came to the same conclusion: university students evaluate their teachers more positively when they learn less.
Relevant details about the studies (again by Poropat):
The first study involved students at the United States Air Force Academy, among the top few American engineering colleges, while the second was conducted at Bocconi University in Italy, one of Europe’s top ten business schools. Both are highly esteemed institutions.
Both studies looked at later student performance because if students have learnt well they should do better in later courses. It is reassuring that students of the best teachers did consistently better in later courses, while students of the worst teachers did consistently worse later in their degree. This is consistent with substantial research on the effect of teacher quality on students’ later performance.
Both used randomised assignment of students to teachers. This means we can be confident that students really did improve (or reduce) their grades because of the quality of their teachers, and not because some teachers got all of the good or bad students.
The empirically demonstrated effect of teacher quality on students’ later performance was really nice to know.
On how evaluations are linked to grades, not learning:
Many educators worry that students are more positive about teachers who give better marks regardless of what the students learn, and are more negative about teachers who make students work hard in order to learn. If this is true, it means the simplest way for a teacher to get a good evaluation is to make it easy for students to get good marks.
As it happens, students who rated their current teacher most highly got better marks in their current course but did much worse in later courses. This confirms the fears of educators: students’ evaluations are linked with current grades, but also with students’ failure to learn things they need for the future. So, a student who is happy with their grade and teacher should worry — they may not have learnt that much.
How could students be so wrong about which teachers actually help them the most? Poropat again:
Educational psychology provides a clear, evidence-based answer — students do not understand what helps them to learn.
Students often assume learning depends on how smart they are and downplay the value of hard work. Despite this, my own research has shown that when compared with intelligence, effort and curiosity have as big or bigger an effect on learning outcomes.
Sounds familiar: the abstract for Mueller and Dweck’s now-famous paper on the effects of praise for effort vs. intelligence / ability goes like so:
Praise for ability is commonly considered to have beneficial effects on motivation. Contrary to this popular belief, six studies demonstrated that praise for intelligence had more negative consequences for students’ achievement motivation than praise for effort. Fifth graders praised for intelligence were found to care more about performance goals relative to learning goals than children praised for effort. After failure, they also displayed less task persistence, less task enjoyment, more low-ability attributions, and worse task performance than children praised for effort. Finally, children praised for intelligence described it as a fixed trait more than children praised for hard work, who believed it to be subject to improvement.
Back to Poropat. Here’s the old “teacher’s fault” excuse you hear trotted out time and again:
If students do not accept that they are stupid and do not believe their efforts are at fault, they are left with another explanation: it’s the teacher’s fault. And there is independent evidence that students who think highly of themselves blame the teacher if they get bad grades.
What sucks is that students don’t recognize what helps them to learn; in fact, students commonly assess whether they have learnt something based on the ease with which they complete a related task.
Mea culpa, dammit.
That is why many students assume that reading or highlighting passages in their text-book, or merely listening to a lecture, is enough to produce learning. They mistake the ease of the task with greater knowledge. Time-consuming and effortful tasks, like self-testing their knowledge, are consequently seen by students as less efficient for their learning, despite the fact that the more difficult tasks produce the most learning.
Hammer that into your head, Mo Beans! Hammer dat thang!
I should add that Poropat writes with the kind of clarity that calls to mind Robin Hanson.
Back to the issue. Why does this matter?
For students, it means it is important to discover what actually helps your learning and focus on that, and live with the fact that real learning takes effort. Poor marks probably do not mean you are stupid or the teacher is bad. It is more likely to mean you need to raise and/or redirect your effort.
Students should also pay less attention to student evaluations when choosing a university course — happy students may not be learning.
There are also consequences for universities and the governments that regulate and fund them. Governments that use student evaluations to assess universities are likely to be misled, potentially resulting in reduced funding for precisely the courses that contribute most to student ability and their later contributions to the economy.
Likewise, universities that rely on student evaluations are likely to punish good teachers and encourage those who simply make it easy for students. Most universities have codes of conduct that require decisions to be made on valid evidence. Any manager discussing student evaluations when reviewing lecturers’ performance is probably breaching that part of their own job requirements. Given the evidence, student evaluations are a distraction from the responsibility to provide the best possible education for the nation.
For university teachers, the challenge remains the same it has always been: keeping students motivated, while ensuring that they learn. Part of achieving that requires teaching students how to learn, not just what to learn, and to ask them what they are working on and how hard they are trying. But it is probably best to avoid asking if they are happy with the course.