Only 16% of success is showing up

As a teacher of a humanities discipline, I’m not inclined to analyze things statistically. Wild claims, thinly supported? Now that’s more my style. But at the end of last semester, I realized that I had a convenient cache of data for helping figure out whether doing the reading really does contribute strongly to getting better grades in my classes – something I often tell my students, though I doubt many give these words much heed.

In both of my courses (each of which had two sections), I administered ten in-class quizzes. Most were unannounced. The quizzes generally tested basic knowledge of the reading assignments due on the day of the quiz. My ideal quiz is one that is extremely easy for a student who did the reading and extremely difficult for a student who didn’t. If the quizzes are at least reasonably well-designed, then I should be able to tell who read and who didn’t just by looking at the students’ grades on the quizzes.

The quizzes also serve as a way to spot-check attendance. Even though my classes are not huge (20-25 per section), I think that taking attendance every day wastes time and smacks a bit too much of high school. (It also requires better record-keeping than I’m really capable of.)

With the quiz data, then, I can tell how closely correlated attendance and reading are with overall performance in class. For each student, I calculated their grade on all written assignments other than quizzes, as I wanted to make quiz performance a truly independent variable. I also excluded class participation, which is partly determined by attendance. Correlating this grade with attendance was simple, as I could just tally up the number of quizzes a student took and compare it with their grade on all other assignments. Here’s what the graph of that looks like:

  

As you’d expect, students who are in the habit of coming to class regularly tend to do better in the class as a whole. Ones who aren’t there are missing something. But most students come to class most of the time, and there’s pretty significant variation in the overall performance of students who attended 90% of the time. The outliers are probably students who for whatever reason did not turn in a major assignment. The correlation between these variables is 0.4, for an R² of 16%. Your class attendance does predict how you do on your written assignments, though not enormously.

 

What about how often you completed the assigned reading? Here I had to look at students’ actual scores on the reading assignments. For each student, I tallied up the number of quizzes that got grades of seven or higher (out of ten). While you probably would not have to read too carefully to get seven points, it would be hard to score that high without reading at all. So, a little arbitrarily perhaps, I’m calling a grade of 7/10 or higher an indicator of having satisfactorily completed the reading assignment. I didn’t take into account average quiz grade, just whether or not the student did the reading. Here’s what I found:

To my statistically untrained eye, the trend is similar but more apparent here than in the previous graph, due to the wider distribution of frequencies of reading, compared with frequencies of attendance. Most students come to class nearly all the time, but not all of them read well enough to do reasonably well on a quiz of their reading. Those who do read more consistently tend to do better in the course. The correlation coefficient here is 0.56, giving us an R² of 31%. Doing the reading is, as I certainly hoped would be true, a better predictor of overall performance than is merely showing up. 

You might expect that doing the reading really carefully would contribute even more to overall performance. To find out, I repeated the exercise where I looked for satisfactory completion of reading assignments, except that I tallied instances of scoring 9 or higher on quizzes. And I found there was an even closer correlation between that tally and overall grades, right?

Wrong. The correlation was effectively the same as what I found with the frequency of satisfactory reading. Lots of factors go into doing well in my classes, and while it might be important to read carefully, it’s important as well to retain what you got out of that reading, and ultimately you need to do something with what you’ve read, whether carefully or cursorily. And in fact, a class full of students who’ve read “satisfactorily” would probably go pretty well. Students in that class would be ready to learn, from me and from each other. I’d take that every time over a class with mostly unsatisfactory readers and a handful of really careful readers. (Though I think I get the latter much more often.)

I’m not yet sure what these findings mean, exactly. I showed them to my students on the second day of class this semester (along with last semester’s grade distribution), in order to convince them that doing the reading really does contribute to better grades. Maybe that's all this little study needs to do. I didn’t tell them about the third finding; doing so wouldn’t suit my rhetorical purpose – getting them to read both more consistently and more carefully.