Bonanza of Correlations, Spring 2019 Edition, part 2

Lower Intermediate Mixed Skills

This course is a bit of a chimera – ostensibly a pre-requisite for transfer-level writing, but in practice very similar to free adult education courses. Students are quite open about this, and the extraneousness of the course in light of the growing AESL program is part of the reason that it will no longer be offered in the fall (in addition to a law passed in CA mandating that community colleges move ESL students up to transfer level writing within 3 years). On the other hand, it’s the course that I’ve taught in at this school the longest, and I have a sentimental attachment to it. In light of that, it might not be all that useful to be combing over my curriculum for areas of potential improvement, but I still want to see what I did right and what I did wrong.

Source.

Group quizzes

I gave group quizzes in all of my classes this semester, but in my lower-intermediate class I had more traditional solo quizzes to compare them to. (For clarity’s sake, a group quiz has 1 quiz sheet shared by 3 to 5 people, all of whom must write the answer to at least one question. Mine were a mix of closed-ended and open-ended short answer questions, usually in response to a reading). It turns out, as with my advanced academic writing class, that group quiz grades predict final grades very little. I will go into some possible reasons later, but first, here are some numbers:

Average correlation of group quizzes with final grades: 0.003

Average correlation of solo quizzes with final grades: 0.51

Average correlation of group quizzes with solo quizzes: -0.29

Besides this class and the class from my last post, I had one more class this semester that used group quizzes – a content-based IEP class that read, to give one example, Merton’s original description of a self-fulfilling prophecy. By this, I mean to say it was a very difficult class, although they complained about the difficulty much less than any other class I had this semester. Anyway, this class’s group quizzes correlated with final grades at -0.09, compared to 0.42 for 3 randomly selected other assignments with the same weighting. In other words, group quizzes seem useless for evaluation across all my classes.

To me, this strongly suggests that if I want to continue to include group quizzes in my classes, I need some purpose for them besides to evaluate knowledge – because it seems that whatever construct group quiz scores measure, it’s not the same construct that my course grades do. And I really hope that my course grades measure some notion of English competence.

To speculate on why group quizzes don’t seem to measure anything worth measuring, it may be worth keeping group dynamics in mind (as this paper points out). A simplistic view of the group quiz may hold that as long as at least 1 group member knows the answer to each question, the group should get that question right. However, in practice, four patterns emerged frequently:

  1. One very advanced, respected, or simply loud member controlled which answers were written for each question
  2. Members deferred to whoever was writing the answer, even when they could see it was wrong
  3. Members with low confidence deferred to the group on many aspects of answering (including spelling and word choice), in effect slowing the group’s progress
  4. As the time available (projected on the screen for all to see) ticked down, members increasingly deferred to each other (as in cases 1 and 2 above) rather than checking answers

End-of-semester comments yielded frequent complaints about group quizzes being unfun and unfair, with some students specifically mentioning the group dynamics that got in the way of scoring well.

On the other hand, the unpopularity and apparent evaluative uselessness of group quizzes doesn’t preclude their other possible uses in the classroom. For example, in cases 1 and 2 above, some proactive teaching of agreeing/disagreeing language might turn the quiz into a task with a second pedagogical purpose. After all, students need to interact in order to boss each other around (or ideally, cooperate in a constructive way). In a writing class, where my quizzes often request students to find quotes that support or refute a specific point, the quizzes with students’ answers could be retained by them for use in future writing assignments; this may alleviate the problems that many students have in finding suitable quotes for their essays (although at the expense of the instructor’s likely reading many of the same quotes used in essays a few weeks later). In any case, group quizzes should probably be worth half the points they are now, be counted as participation rather than quizzes, and be rewarded visibly in class (in support of the social aspect of group quizzes) rather than handed back silently with a grade. In short, I should treat a group quiz like a long-form Kahoot!

SI Workshops

This semester, I had a Supplemental Instructor (SI), basically a TA, who assisted in class once a week (class met twice a week) in addition to holding workshops the mornings before my classes and meeting students individually in the language lab. Students could turn in a worksheet after each workshop or meeting for 1 point of extra credit, meaning it was very easy for me to look at my grade spreadsheets and determine how going to the SI helped students succeed in my class overall.

The quantitative picture is not as bullish as you might imagine, but the qualitative picture (and probably quantitative picture for people with more access to student data than I do) is probably very encouraging.

First, the average score in this category was an amazing 9.32, and the standard deviation 5.30. That means that the average student went to more than half of the workshops, and many (4 out of 21, to be exact) went to all of them. The average final grade for those who went to every workshop was 7% higher than for those who went to none, and the correlation between this one score and final grades was 0.45 (higher than that of 3 of the 6 solo grammar quizzes).

On the other hand, submitting extra credit worksheets after sessions with the SI correlated poorly with their Projects grades (0.14), Quizzes (0.02), and alarmingly, Final Exams (-0.12). What these students were practicing in workshops didn’t seem reflected in some other big components of their grades – even ones like quizzes that they were specifically reviewing for. This may seem to be a selection bias – of course, I can’t help but think, students who struggle with quizzes are more likely to go to quiz review workshops. But then, why do these same students do better in the class overall (particulary on homework, of which this score is a component, and language logs)? Is workshop attendance a better measure of general attentiveness and the ability to handle classwork than grammar skill per se, and if so, is it still worthwhile to reward students with extra credit for attendance at these workshops?

I tend to think that it is, but more as a gesture toward future semesters than to my own class. That is, even if workshop attendance correlated negatively with every component of final grades in my class, they would still be worth rewarding in that 1) the class in question is ostensibly for students preparing for transfer, and seeking help from tutors, teachers in the lab, and other campus services gets more and more important as students get closer to transfer-level courses, making this a good habit to build; 2) students build community and rapport in workshops with each other and with the SI, which may help with retention; and 3) without commenting on my SI specifically, SIs are frequently students themselves, and in the ESL program, often current or former ESL students. This gives current students time with a much more realistic and attainable “near peer” model of success than the instructor of record, usually an MA-holding native speaker. I can’t say for sure whether the factors above really do predict retention and success since I don’t track these students once they leave my classes. I do observe informally that students who meet frequently with the SI tend to have a more specific plan for following in the her footsteps.

Thanks for reading! Here’s something from a channel I like to have on in the background while I’m grading:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s