Teach a man to find correlations, he posts them for a lifetime

Aphorism showing its age aside, this post is designed for both men and women who use Canvas and are curious about statistics that may be hiding in their classes’ grades.

I have my own data to share about this semester’s classes, but first, here is a tool that you can use to do the same:

Stat sheet for grades 1.1

And an explanation of how to use it:

On to what I found.

I had 4 classes this semester – 2 Oral Language classes and 2 Written Language classes, both in the 2nd to last term of my university’s IEP. My university’s IEP works a bit unusually – my 4 classes were just 2 groups of people meeting for 4.5 hours a day 4 days a week, about half of which was “Oral Language” and half of which was “Written Language”. The first group of people were my students for the first “term” (=half of a semester), and the second group were mine for the second term. All told, I still had 4 gradebooks on Canvas to export and fiddle with. Between the 4 of them, I found these interesting statistical tidbits:

Scores of 0 are more predictive of final grades than full scores are

One would expect the number of 0s on assignments to negatively correlate with final grades, and the number of full scores to do the opposite. That is, thankfully, true. However, they correlate at different rates – across all my classes, on average, 0s are more (negatively) correlated with final grades than full scores are (positively) correlated. The reason for this is that full scores were more evenly distributed among all students than 0 scores, which were concentrated among a few students. The one class for which this was not true was the one that I changed my late work policy and started giving 1/2 credit for certain late assignments.

This would not be a cause for any particular change except for 2 reasons: 1) as shown by the last class, many of the 0s that students were getting were from late work rather than unsubmitted work, and 2) we have a fairly strict policy about grading by SLOs (student learning outcomes, one of the first abbreviations I had to learn upon my return to the USA after years in Japan), and nowhere in our SLOs does it say that students should learn the sometimes-merciless grading policies that one may encounter at university.

Therefore, I should really make the “late work gets partial credit” policy permanent. I should also probably give fewer full scores.

5% 0s is a line in the sand

I enjoy running t-tests to see what values in what grade categories produce statistically significant differences (p=0.01) in my students’ final grades. One t-test I ran (on the “other stats” sheet in the file linked above) was seeing if students who missed 5% of assignments were different in statistically significant ways from those who didn’t. It turns out that they are, in all 4 of my classes this semester. On the other hand, those who missed 2% of assignments weren’t. Perhaps I should give an opportunity to make up homework on about 2% of assignments (as I already do for classwork).

I’m hoping that my future classes have grades that reflect the average quality of their work, which in turn reflects their ability to do academic work in English, rather than their tendency to check due dates and read rubrics thoroughly on Canvas. These are important skills, but I won’t want to make them a bottleneck through which every grade must pass.

RDs need a bump, FDs need a nerf

Across 4 essays in both Written Language classes, the average correlation of rough draft scores with final grades was 0.70. The average correlation of final draft scores with final grades was 0.76. Since final drafts are worth at least twice as many points as rough drafts, this is rather surprising – even moreso because for 3 of the 4 essays, the rough drafts’ correlations are actually higher than the final drafts’ (the last had a very low correlation for the rough drafts).

I’ve been making changes to my writing process over the last few semesters, and it seems I need to make a few more. I think part of the comparitively low correlations that final drafts have is due to my grading practices – I think I take it easier on final drafts precisely because they’re so many points. My average scores for final drafts are higher than for rough drafts, and the standard deviations as lower – roughly 62%-95% with an average of 78% for rough drafts and 65%-95% with an average of 80% for final drafts. It’s not a huge difference, but looking back at the scores now they don’t seem to reflect the range in quality of the essays. Part of the high correlations for the rough drafts is also due to the skills that are involved in producing a first draft – planning, reading, responding to a prompt, and a bit of grammar – that are assessed in a lot of other assignments as well. Final drafts, meanwhile, assess (in addition to the same things that first drafts assess, but less directly) responding to criticism and editing, which don’t figure largely in many other assignments. Seeing how first drafts track more of the skills that I care about, and I seem to grade them with less of a high-stakes mentality, I should probably weight them more. On the other hand, since final drafts have a somewhat narrow range of skills that they assess, I should weight them less, or even separate my grades for final drafts into smaller sub-assignments like the COCA assignments I currently use, but also a written response to criticism and proof of visiting tutors instead of trying to indirectly read those things into the final draft.

I need to keep in mind too that I’m not necessarily serving my students well if I introduce them into a writing process that none of their psychology, history, or any other professors will use – I hear that most papers turned in for any class other than English are just the final drafts, already assumed to be revised and polished to a sheen. Maybe having one paper like this per term is also justifiable just in terms of preparing students for being taught by PhDs who know more than anyone else in the world about the behavior of certain species of field mice under certain conditions but have never studied pedagogy.

Look forward to more like this same time next semester, and let me know if you find the sheets useful for your own classes.

Goodbye to California, pt. 1

Shortly after my acquiescent post on the constant rejection one faces applying for full-time ESL jobs, I got an email curiously positive in nature and free of formulaic boilerplate. I had gotten so used to rejection that I almost didn’t comprehend it at first – but it was an invitation to interview, something I had gotten just a few times in the years since my MA. And after that first interview on Skype, I got another such email from the same place, inviting me for a campus visit. When the date came in late May, after I made sure my grading for the weekend was already done, I boarded a plane at John Wayne Airport at 4 AM and spent the whole day in a state besides the one that I have lived in since returning to the US in 2016.

Now, I was breathing such rarefied air at this point that I felt zero pressure to succeed, happy to plant my flag at the “second interview” stage before what I assumed would be a quick descent back down to solid adjunct ground. This was a Monday. I had classes again at my usual schools on Tuesday and plenty of proctoring and grading to do after that to help push the entire episode into the past tense – I was already imagining the conversations I would have in the break room at all the same schools next semester about the time I came this close to getting a full-time position.

But as a call a few days later informed me, I did get it, and very soon after this post goes up, I’ll be starting my first classes there.

By crazy when-it-rains-it-pours coincidence, this was the 2nd full-time job offer I took this year – although the first was a contract only for the summer. That job, which just ended, has given me a bit of a sneak preview of my life as a full-time teacher in a context other than Californian community colleges. I thought I would share a bit of my reflections here, both as a document of my thoughts for myself and as a guide for other adjuncts hoping to do something similar.

Adjunct Goodbyes and Full-time Goodbyes

I’m excited about my new job, but I do have a few regrets about leaving the colleges where I teach now. One of those regrets is that I did many things for the last time at my main schools without realizing they were the last times. I had my last norming meeting (and I enjoy those), my last walk with a student between the classroom and the lab to show them where it is, and my last unexpectedly long pause while the projector warms up, all without knowing that I would never do those things there again. I saw a bunch of people in passing in a hallway or copy room and said some simple words of greeting or an inside joke not realizing that those were the last times I’d be doing that with those people. Not to strike too melodramatic a tone, but for the most part these were the first workplace acquaintances I made in California, and they witnessed my whole process of getting my feet wet, asking silly or obvious questions really politely (“Sorry if this is obvious to everyone here but me, but what is an SLO?”). I will probably like my new coworkers – teachers are usually nice – but they won’t be my first coworkers in the US. I have a lot of words of thanks to go around, but I won’t be specific here. If we spent more than one microwave’s cooking time together, I appreciated it.

There are a few students who had let me know that they wanted to sign up for my fall classes with whom I’m not holding up my end of the bargain. This makes me feel a bit guilty, as does the fact that I won’t be able to wave or chat to former students that I see around campus, but both of these are a bit of an unnatural extension of the teacher-student relationship, which formally has a lifespan of one semester. The same goes for quite a few “single-serving friends” I made in break and copy rooms, for whom the loss isn’t of a deep friendship but just the potential for a longer one of whatever quality it was for 30 minutes a week while we both ate Amy’s frozen burritos. I got some kind words from my now-former coworkers, but of course the definition of an “adjunct” is something inessential to the major workings of whatever it’s part of. At any school with adjuncts, some portion of instructors and students will have the experience of suddenly not having a colleague or teacher on campus anymore every semester. I suppose part of my newbieness that never wore off was expecting to know when that time was coming for me.

(OK, I will single out for thanks 4 people whose initials are G.P., R.B., C.C., and B.W. who saw me at my most newbieish and imparted some very important and well-timed advice. Shucks, also my most frequent collaborators H.L. and D.P.. Also all my SIs.)

On the other hand, at my full-time summer job, we all knew pretty well from at least mid-June that I would be gone, and the program exists solely so that students matriculate out of it and into another program. The goodbyes here had pomp and ritual and lots of tears. People act differently when they know things are ending, and the entire last day of work was dedicated to ceremonial closing of the program, complete with thank-you cards being exchanged, speeches, skits, musical performances by every combination of students and teachers, and a lovely banquet to top it off. It was the best way to conclude a summer program and my time in California, with some really excellent people.

The lesson here, I guess, is to know as much as possible when you’re heading into a round of goodbyes.

More to come later.

Taking steps in class

I mean this literally. I got a Fitbit last year, and during the spring semester, I tracked how many steps I took during an average of 5 class sessions of each of the 3 courses that I taught.

My classes were a content-based IEP class with 13 students, a mixed-skills intermediate-level credit community college ESL class with 21 students, and an advanced ESL writing class with 25 students.

Across 5 class sessions, the average number of steps total for each class was:

  • Content-based IEP: 236
  • Intermediate CC: 626
  • Adv. writing CC: 440

Of course, since the class sessions were of different lengths, it makes sense to divide the number of steps by the number of minutes in which I had to take them.

Steps per minute of class time, including breaks:

  • Content-based IEP: 2.63 steps per minute
  • Intermediate CC: 2.78 steps per minute
  • Adv. writing CC: 1.96 steps per minute

Last, because higher numbers of students might feasibly require the teacher to move more and farther around the classroom, here are the steps per minute further divided by the numbers of enrolled students:

  • Content-based IEP: 0.20 steps per minute per enrolled student
  • Intermediate CC: 0.13 steps per minute per enrolled student
  • Adv. writing CC: 0.08 steps per minute per enrolled student

What does this tell me?

I tended to walk around more, all other things being equal, in the content-based class. I attribute this to the type of work they typically did – small group discussions in which I would move from group to group and either guide the discussion, participate as an equal, or just listen. The other two classes, at community college, usually involved at least some “lecturing”, standing relatively still or sitting at the computer and typing notes projected onto a screen.

I think my classes could benefit from structuring more lessons around small group work rather than lectures to begin with. As it turns out, a further benefit might be that it helps me reach my fitness goals.

Image result for fitbit blaze
Lecture disincentivization tool. (source)

The corpus of rejection

Every few weeks, depending on the season, I get a message like the following in my inbox:

Dear [name],

On behalf of the application review committee, we thank you for the submission of your application for the [position]. We recognize that the application process requires a great deal of time and effort on your part. Regrettably, you were not selected to move forward for an interview.

[more stuff that I never read]

Sincerely,

[Office of Somethingorother, name of college]

The slightest amount of experience with this type of letter lets you figure out the gist after the first line, or even from the existence of the email itself, coming as it does prefaced “DO NOT REPLY”, a subject line with the illocutionary force of a restraining order.

I’ve gotten enough of these over time (more than some, not as many as others – adjunct is a job with a depressing number of grizzled veterans sporting depressing amounts of grizzle) to start noticing patterns in the language that these messages use. A mini-corpus thereof can be found below.

Image result for gordon ramsay fuck off
Spoiler alert: This level of frankness would be refreshing.
Continue reading “The corpus of rejection”

ESL Students’ Feared Selves

Part 3 of a 3-part series on possible selves (scroll down for parts 1 and 2).

If I’m being honest, these were the most fun to read, although as I stated before I can’t share any of them with you.

It’s not some kind of sadism that prompts me to say that: The descriptions in students’ responses to this final question were much more affective in content than the first two. Rather than lists of future colleges and jobs, here we had responses more along the lines of “I have no friends and I have a SAD SAD life”. Again, you can’t see them, but you can see what types of complaints were the most common, which should be just as fun. As in my last 2 posts, I combed over each entry looking for mentions of specific subjects. Because emotions were much more commonly mentioned for the feared self than for the other 2 selves, I tried sub-categorizing types of negative affect as well.

Below was the prompt, answered by my 2 multi-skill intermediate classes and 2 advanced academic writing classes over the past 2 semesters.

Imagine the worst version of you in 5 years (the opposite of the first). What happened to your English, and why didn’t you succeed? Give details. What is different in your life because you can’t use English?

Continue reading “ESL Students’ Feared Selves”

ESL Students’ Ideal Selves

Part 1 of a 3-part series. As an end-of-semester assignment, I had my summer and fall classes (4 total; 2 intermediate multi-skill and 2 advanced academic writing) write about their ideal, ought-to, and feared selves. Besides being a recent buzzword in ELT, possible selves make an interesting writing assignment for both the teacher, who gets to find out his students’ motivations in a bit more detail, and the students, who get to describe their (hopeful) future lives. Now, in fairness to you, I should point out right at the start that I won’t be excerpting their writing here; I didn’t warn them that I’d be using this assignment for my blog and I am one of those teachers who doesn’t even share pictures with his students’ faces in them without asking each one of them individually. Instead of showing you what they actually wrote, I will be analyzing each of their answers for the prevalences of certain topics and concerns and then doing some basic statistics with these. As it turns out, this takes a lot longer.

This post will only deal with ideal selves, with ought-to selves and feared selves to come later. First, here is the prompt and example that they saw.

“For this discussion, please answer these questions in different posts:

  • Imagine it is 2023, and you have succeeded in English in the best way. What steps did you take to get here? How do you use English now (in 2023)?
  • What can you, now, do every day to bring yourself closer to that future best version of you? What kind of things should you do? How should you “study” or “practice”?
  • Imagine the worst version of you in 5 years (the opposite of the first). What happened to your English, and why didn’t you succeed? Give details. What is different in your life because you can’t use English?

Last, reply to a classmate in at least 3 sentences.

Example first post:

In 2023, I am a college graduate. I have transferred to UCI and graduated with a major in computer engineering. I used English in all of my classes to do homework, work on group projects, and give presentations. Computer engineering was still hard, but my English helped me a lot. It also helped me to make friends and find a job. Now, I work for Blizzard Software and I design graphics for upcoming games. I use English at work, of course, but I don’t think of it as ‘practice’ anymore. Now, it’s just life.”

Continue reading “ESL Students’ Ideal Selves”