The black box of take-home writing

This semester I’m trying something new in my writing classes: trying to eliminate the interference of “writing enhancement” software, along with all other potential sources of noise between the students’ brains and the page, from my take-home essays. This is because as an ESL teacher, I need to maintain the validity of the “grammar” scores on writing assignments that I give, assuming that I need grammar scores at all, and of course I need to know that whatever students are turning in is a product of their own thought processes. To that end, I’m changing the planning and drafting processes and part of my grading rubrics.

For comparison, the writing process that I used to use looked like this:

  1. Outline: 5 points in the “homework” grade category
  2. Draft 1: 1 point of the total essay score
    1. typed or handwritten at home
    2. Peer-reviewed only
  3. Draft 2: 9 points of the total essay score
    1. typed
    2. Gets detailed feedback on content, structure, and grammar from me
  4. Final draft: 90 points of the total essay score

And a typical rubric for the final draft looked like this (initially adapted from a few coworkers’ rubrics):

Content


10
The essay has a well-focused thesis.

15The writer supports this thesis in the body paragraphs.

15Sources are utilized well and integrated into the argument.

40Total for content
Organization

5The introduction paragraph(s) captures the reader’s attention and introduces the sources and background enough so that the thesis is understandable to a unknown reader.

15The body paragraphs show clear and effective organization, and have clear idea progression and relationship between paragraphs. The point of each body paragraph is always clear.

5The concluding paragraph readdresses the thesis nicely, does not exactly repeat it, and gives the reader a reason to care.

25Total for organization
Grammar/Language

20The essay has sophisticated, well-chosen sentence structures. The language errors do not interfere with communication. In particular, there should be no errors with noun clauses, comma splices/run-on sentences, hedging, or hypotheticals.

20Total for grammar/language
Format/Writing Process

5Effective use of MLA format including a Work Cited Page.

5Total for format & writing process.


90

Total points

The problems with this approach were 1) a lot of more feedback was given than was actually used for revision, 2) the first draft scores (out of 1) were consistently found to be very predictive of final course grades but were worth very little on their own and 3) I could not tell when the grammar scores I was giving were valid and when I was basically giving Google Translate an A.

Outside of our classrooms, an arms race is being waged between smarter and harder-to-detect ways of generating papers through AI on one side and software designed to detect plagiarism on the other. Copying and pasting still happens (and is the easiest to catch, even without Turnitin.com), but a minimally savvy plagiarist can direct a writing “assistant” generate an essay (Google “generate essays” for examples) or a summary, as I found on a recent podcast episode. At least for the moment, automatic plagiarism-checking software doesn’t catch AI-generated text, whether it comes from Google Translate or Ultron. An add-on to Chrome called Draftback can play back each keystroke in the creation of an essay (or any other Google Doc), potentially catching copying and pasting from AI sources (as copied and pasted text appears all at once as opposed to one letter at a time), but can’t tell who’s sitting in the chair typing text that is entered manually. When I see grammaticality, vocabulary and idiomaticity that is conspicuously improved, I have no way of knowing whether it comes from hard work and scrupulous proofreading or from the magic of smartphones:

I thought English/ESL departments might be some of the first to notice the black box of take-home writing, but others are even more on the cliff’s edge. The transition described in this post was also partly spurred by a conversation that I had in the adjunct work room at one of my community colleges in California, in which a Philosophy professor decried the amount of plagiarism going on in his and others’ classes and told me that he had on good word that UCLA’s Philosophy department no longer gave take-home writing at all. There is, after all, several hundred years’ worth of plagiarizable text on Plato’s Cave.

At my new job I’ve had the chance to talk to a few professors in different departments, and when it comes up, I’m often surprised at how large a portion of their writing assignments has also moved from students’ homes on the weekend to labs on campus during class hours. The reasons stated are usually a combination of wanting to help the students build good writing habits more actively and also simply having no ability to trust what you are getting when an assignment leaves your classroom doors. Some have also said that they dislike the for-profit model of services like Turnitin and Unicheck as well as the message of distrust that they send to students, preferring to keep writing to class hours where at least the pretense of benevolent watchfulness instead of red-pen-policing can be maintained.

I realized that there was a way to kill all of these birds with one stone as well as emphasize the “ideas” part of essays by radically changing my writing process.

The new process looks like this for a non-research essay based on a book or article:

  1. Outline: 10 points of the total essay score
    1. Peer review and instructor feedback on the outline
    2. Done in Google Classroom
    3. Many activities to build robust outlines before Draft 1
  2. Draft 1: 30 points of the total essay score
    1. In-class in a computer lab with only the outline and one page of notes (the outline has whatever quotes they’ve chosen to use)
    2. Typed into the same document as the outline with no other websites or software allowed
    3. Peer review and instructor feedback
      1. Grammar feedback is only on the first 2 paragraphs, and after that only in the form of the COCA tag
      2. All other feedback is on higher-order issues
  3. Draft 2: 60 points of the total essay score
    1. Revised at home and turned in
    2. Accompanied by separate grammar assignments based on Draft 1

What has changed is that:

  • weights for all 3 parts of the writing process are distributed more equally
  • only Draft 1 has a grammar score
  • Draft 2 has grammar assignments in place of a grammar score
  • There are only 2 drafts

Both Drafts 1 and 2 have most of their points given to Content, a bit less for Structure, and a tiny bit for Format/Mechanics. Overall, compared to my old writing process and rubric, more time and more points are given to Content.

(I should also point out that I’m working with a shorter time limit now than I used to – 7-week terms instead of 16-week semesters. Still, I think the important parts don’t suffer much from the eliding of one draft.)

The grammar assignments that I give now in place of a grammar score for Draft 2 are all COCA-derived, and students use my COCA tags in their Draft 1 to know what to look up. This was actually the topic of a talk I gave at ITESOL last month (titled “Using COCA to Simplify Your Correction Codes”), and even if I find reasons to change the 2-draft model outlined above, I will almost certainly be keeping COCA in place of grammar on my rubrics. The assignments are short but open-ended in both the problem (something from their Draft 1) and the solutions.

In addition to changing the process, I try to have prompts that discourage ghostwriting or copying – a combination of new or unusual source texts (Digital Minimalism being a recent example), personalization (the DM essay required screenshots from the students’ own smartphones), and just topics that students want to write about (again, smartphones).

A lot of my former and current colleagues have described moving to a “studio” model of teaching academic writing – lab co-reqs at my last community college, 5-unit plus-sized courses at my current one. Who knows how the proliferation of text-generating technology will affect the “academic essay” in future writing classes?

I joke about this in class, but it’s probably not too far off that we’ll be asking students to turn off their retinal implants before doing anything in class (or generating class content by AI ourselves).

Corpus Family Feud

Since I started teaching community college ESL, I’ve set aside at least one class period in all my writing classes to teach students how to use COCA and the other BYU corpora, but I struggled for a long time to incorporate it in an intuitive way into my intermediate multi-skill classes. I think its utility is clear, but the interface (computer literacy can be a problem) and baseline metalinguistic knowledge necessary just to use it have thus far stopped me from making it a regular feature. I do, however, have one activity that uses corpora (either COCA or iWeb) that is reliably entertaining and useful for classes of any level. I call it Corpus Family Feud.

Like the real Family Feud (a TV game show, for those of you outside the US and non-fans of SNL), the point is for participants to guess the most common answers to a survey question. Unlike the real Family Feud, the questions are specifically concerned with language use, and the “survey” is of corpus data rather than 100 people randomly by phone.

Also like the real Family Feud, it’s the studio’s (i.e., the teacher’s) job to prepare the questions and collate the survey answers beforehand, and then reveal them to the participants after they have made guesses.

The basic steps are:

  1. Before class, prepare sentences with one or more blanks, and then find the most common words that fill those blanks according to corpus data. 3-5 total sentences for one session seems to be a good rule of thumb to keep interest high throughout the activity.
  2. Also before class, prepare a slideshow (I use Google Slides) that features the sentence with blanks, directions for what kinds of words go in the blanks, and the answers in list form. The answers should be set to be invisible when the slide loads and appear on subsequent clicks.
  3. During class time, announce that you are playing a game, and display the slide with the first sentence. Tell explicitly what kinds of words can be used to fill in the blank, and tell in general terms that you found the top 5 words that people actually use to fill in that blank in their real communication in the real world.
  4. Have students write down the top 5 words that they think fill that blank in the real world. Announce that they will get 1 point for each of their answers that is actually in the top 5. Tell them also that it doesn’t matter which order they put them in; they get 1 point for any answer that was in the top 5.
  5. After a few minutes, announce that you will start displaying the answers. Drum roll and display the first answer. Students will probably applaud, shriek, or say, “ohhhhh”. Remind them to keep track of how many points they have as you continue drum rolling and displaying the answers in sequence.
  6. After you’ve displayed every answer, ask the students who has 3 points, 4 points, or 5 points until you figure out who the winner is. Give the winner a piece of candy or some other gold star-equivalent. Repeat with the next sentence.

As a variation, you can choose 5 words in advance, display them when you display the sentence, and ask the students to put them in order. This allows you to choose words other than the true top 5 according to corpora (which are often boring words that nobody ever thinks of, like “be” or “doing”), but requires you to give points only for correct order of words rather than giving points for any word that appears in the actual list.

For example, let’s say your intermediate multi-skill class is covering gerunds (I mean “covering” as in it came up for one reason or another, whether as a front-loaded chapter of a synthetic syllabus or as focus on form after a task). You might decide on a few chunks where gerunds are commonly used, like “I enjoy ___” or “____ is important”. These would be the questions for your game. Your slides might look like this:

I really ought to make this look more game show-like…

I display the frequencies, but this is probably unnecessary. In the variation where you supply the words, it might look like this:

Where only the ranking and frequency numbers appear on click and the words are displayed from the beginning.

Other variations I have used in the past look like this:

There is almost literally no end to the kinds of phrases or grammar you can use to play this game. Besides an excuse to use corpora in a mid-level class, this helps turn what could be an abstract grammar lesson into one that respects chunking and the conventions, rather than just the rules, of language. Have fun!

Nudgework

With my teacherly black robes in mind, I’ve been giving my students a particular type of assignment recently that maximizes use of the teacher’s ability to give orders. This type of homework, which I think is worth exploring as a new teacher- and student-friendly homework paradigm, has a few qualities in common:

  • It places students in situations where input is likely.
  • It does so with directions that on the surface have little to do with language learning.
  • It involves minimal paperwork.
  • It requires little or no reporting or reflecting.

This kind of homework is ideal for low-intermediate students, particularly in a place like Southern California where it is very easy to spend one’s entire life surrounded by L1 speakers (of Chinese, Korean, Spanish, Arabic, Farsi, or what have you), and a little nudge is all that might be needed to gain practically unlimited meaningful input or interaction. The goals are increasing input, building confidence, and setting up habits which will facilitate language learning throughout the students’ lives.

banking business checklist commerce

Photo by Pixabay on Pexels.com

As an example, one of my recent homework assignments requires students to get a tutor’s signature and then draw (not take) the tutor’s picture (our college offers a variety of tutoring services). There is nothing in the homework assignment that requires them to seek a specific lesson from the tutor or even to ask a question. The point of the homework is just to put the student in a situation (talking to a tutor face-to-face) where their instincts will lead them to inevitably have some kind of interaction, as well as give them the experience of having talked to a tutor and thus taking away some of their reticence to do so in the future.

This kind of homework tends to rely on human instincts to interact or to latch on to things that are interesting to them in any given situation to be effective for language learning. If a nudgework assignment is to “sit at a café for 30 minutes without your smartphone”, it’s very likely that their trip will include a conversation with a barista and incidental input from Auto Trader or Healthy Living magazines. It’s the kind of thing students could feasibly do anytime, but a directive from someone standing in front of the white board makes much more likely.

The downside is that input is simply likely with this type of assignment, not guaranteed. A much more straightforward language assignment, along the lines of “read this and then prove you read it with a detailed report”, makes input practically inescapable (and makes it much easier to talk about it as a class if everyone read the same thing). The downside of a traditional assignment is that the input will probably be of less interest to the students, and a large part of the time taken for the assignment will be devoted to proving to the teacher that the input happened rather than getting more input. Krashen isn’t the last word on these things anymore, but I still tend to think input is superior to reporting when it comes to moving the interlanguage ball forward.

In my teaching career, this nudgework idea evolved out of my Language Logs, which are a regular type of assignment I give that follow the format: “Find examples of grammar point X on the Internet or in real life. Copy and paste/post a photo on the discussion board and describe the grammatical form.” The Language Logs are still a regular part of all of my classes, but particularly in my lower intermediate classes, I wanted a kind of assignment that would facilitate more natural interaction/input and have less emphasis on metalinguistic analysis.

As a last perk, there isn’t much to grade.

input-reporting

Losing my mind

What follows is a long, student-unfriendly version of a 3-paragraph paper (not an essay) on a 30-day challenge that I did with an intermediate integrated skills class.  The paper has to have an academic paragraph on the time before, the time during, and the time after the challenge.  Originally, the paragraphs had to use the past tense, present tense, and future tense (with any aspect), but I haven’t followed that rule faithfully here.

Getting lost in hectic thought was the default mode of my mind before I started my 30-day challenge.  The challenge, which was to meditate 10 minutes a day for 30 days, came at a time when I my mind was almost constantly in a state of emergency.  Every thought of grading, making new assignments, or updating a class vocabulary list was a red alert in a long line of red alerts.  I would be exhausted at the end of a day of classes, but unable to take a nap without thoughts of all the papers I had to grade rushing in and beating back my attempts at rest.  As a result, I was often in a sour mood and was inclined to greet any attempts at contact from colleagues or students as yet another demand on the limited resources of my attention.  When I had a minute, or just a desperate need to pretend that I did, I spent it with value-free distractions (the App Store specializes in them), afraid to glance back at the wave of paperwork threatening to crash over me from behind.

Since I started meditating, I haven’t ceased being distracted, but I have been better able to incorporate distraction into my workflow, i.e. to be mindful of distraction.  In the interior of my mind, thoughts of work have begun to appear less like photobombing tourists in the lens of my attention, and more like part of the shot.  I have become better able to take a long view of my own time and attention and to refuse to devote my full mental resources to every problem, incomplete task, or request that jumped into frame.  What is called “mindfulness” is key to this.  While I meditate, thoughts still appear, and I still think them, but I am aware of the process, and that awareness prevents me from identifying with them completely.  I become something of an observer of my own mental life.  I see how this could be described as being “mindful”, as it does in a sense feel like an additional layer of abstraction has been placed between my stream of consciousness and the thoughts that usually occupy it, but in a sense more important to me, something is also taken away.  That thing is the formerly irresistable urge to load that thought into the chamber of my executive-function pistol and start manically squeezing the trigger.  It is also the need to build a spider’s web around each thought, connected to all my other thoughts, and claim it irrevocably as mine.  In these senses I believe “mindlessness” is just as good a term as “mindfulness” for what occurs in and as a result of meditation.  In any case, disassociation from my thoughts, most of which are proverbial red circles with white numbers in them, has helped me to control the way that I react (or not) to them.

This brief experiment with meditation has given me a good deal of perspective to take with me into future semesters.  I can now see the regular rhythm of the waves of classwork as something other than a renewed threat.  Now, they seem more like tides, dangerous if unplanned for but predictable in their rises and falls.  Importantly, I also see the high water mark and know that as long as I keep my mind somewhere dry, it will recede without doing much damage.  In the future, as long as I refrain from doing something crazy like teaching 20 units, I think I will be able to maintain calm with the help of this perspective.  Also, in a more specific sense, I will be better able to resist the call to distract myself from my work.  I can recognize the formerly irresistable need to latch onto an interesting task, and this recognition enables me to prevent YouTube or WordPress (except for right now) from hijacking monotonous tasks like grading or… well, mostly grading.  Next semester and into the future, I will feel less threatened and better able to deal with inbound masses of schoolwork.

Learning software in need of a theory of learning

I have a new project that I enjoy and I think my students will enjoy, but I have trouble fitting into any known theory of language learning.

I call it the Random English Grammar Generator, and it started off as a way to challenge myself to relearn Java before slowly metamorphosing over time into a semi-professional Javascript/JQuery/XML hobby.  I say semi-professional because I tell my students about it and encourage them to use it but never give points or even extra credit for it owing to the fact that I don’t know of any theory of language learning that could justify spending time on it.  See, it’s fun for me to tinker with as a former CS major, but as a teacher I have trouble explaining its utility, as it’s certainly not meaning-focused, interactive, or communicative.  It’s not even very useful as input because the software-generated examples are decontextualized and sometimes have very odd collocations (which will be improved in the next update, but will never be completely natural).  In short, it is to English students what one of those Rube Goldberg-looking cat gymnasiums (gymnasia?) is to a typical housecat.

A few things make me feel like this is something other than a pure hobby.  I know some kinds of students, mostly my former students in Japan who loved manipulation of abstract systems and perfunctory tokens, who will enjoy playing with it, and this provides me some comfort.  Many ESL departments at universities and community colleges in California also seem to spend money on software packages which are similarly grammar-McNugget-oriented and only slightly less contrived in their examples, and they may show an interest in something like this if I can make it a bit more tailored to the grammar books I know they use (for instance, by putting all the passives in one place and the hypotheticals right after the basic if clauses).  For the moment though, it is a showy jalopy that I spend a lot of time working on but can barely get me to the supermarket.

Chinese-to-English Whispers

There is a children’s game called Telephone in which a long line of people whisper a preselected message down the line, comparing how it came out in the end to what it was when it started, often with hilarious results. This game is called Chinese Whispers in the UK, possibly because the name Sneaky Orientals was already taken.  I have adapted this game into a few short written activites for English classes, one of which is what I consider a defensible use of grammar-translation in the classroom.

I presented the version of this activity with pictures at Shizuoka JALT this April in a My Share event as a form-focused activity with a clear communicative purpose. Both are for classes of 3 or more people.  The translation version is best for high school and above; the pictures version for elementary and up.

Continue reading

Guidelines for educationally responsible grammar-translation

Grammar-translation in the ELT community is a bit like Republicanism in the San Francisco Bay Area.  It’s practiced widely outside our community by what we imagine are people who generally don’t know any better or whose priorities are twisted against the interests of their socio-economic group, but espousing its values among we who (think we) are more enlightened is like opening up a bag of Doritos at the ballet.

For those in need of explanation of what exactly I’m condescending about,

Yakudoku [grammar-translation in Japan] is defined as a technique or a mental process for reading a foreign language in which the target language sentence is first translated word-by-word, and the resulting translation reordered to match Japanese word order as part of the process of reading comprehension.

(The article that this quote comes from is a good one to read if you wonder why people keep saying Japanese students are good at reading in spite of the entire culture seeming to regard printed English like a dog regards catnip.)

Grammar-translation, back and forth into and from the TL, seems to be the default method in state schooling worldwide.  It is unpopular among trained ELT professionals for a few good reasons. As a top-to-bottom lesson plan or curriculum, grammar-translation clearly fails, and there is no justifying its nonetheless overwhelmingly widespread practice (claims of its necessity for standardized tests, even when true, just increase the scale of the problem).  It surrenders all responsibility for language learning to the most mentally taxing tasks – memorizing connections between abstract tokens, applying rules of transformation in ways that place a heavy burden on short-term memory rather than automatization, knowing long lists of exceptions that invalidate most of the rules you just learned.  I have a hunch that part of the reason for its continued practice in public schools worldwide is precisely because it turns language learning from an activity that most of humanity has engaged in successfully for most of the history of our species into a bell-curve producing all-purpose test of general intelligence and dedication to the ritualized study process.  What it doesn’t do is produce functional language users, hence its unpopularity among ELT specialists and incredulity that it could be so common.

Grammar-translation also has a role in the ongoing cartelization of skills of native speaker and non-native speaker teachers in Japan.  Demanding as it does fluency in Japanese as well as long familiarity with the conventions that constitute “correct” translations, grammar-translation is seen as the exclusive domain of Japanese (NNS) teachers.  Whether NS teachers can ever become competent practitioners of it is beside the point; they are never asked to.

There is reason though to believe that grammar-translation can still have a place in a responsible and modern curriculum.  Since the 1990s, grammatical teaching, i.e. teaching the rules explicitly, has made something of a comeback among the ELT elite, albeit usually reactively and among lots of input and interaction.  Approaches seen as forward-thinking in recent years, including Task-based Language Teaching and Dogme, recommend explicit negative feedback and focus on form, i.e. some time to look at the abstract rules that govern grammaticality and what is correct or incorrect according to them.  The latest incarnation of the need for explicit as well as implicit knowledge and teaching seems to be the volume reviewed here, which I hope to read when the price drops. Grammar-translation can be a very helpful type of reactive grammar-focused activity and has strengths that other such activities don’t.  In this post I plan to introduce a few rules of thumb for grammar-translation as part of a modern language class.

11105_naiyou1

First step: Throw away this textbook.

Continue reading

Things (the game)

Here’s a class activity that like my last two is basically a frame for communicative use of selected grammar and vocabulary.  This one is based on a party game that I was introduced to on a trip back to the US called Things.  It has the advantage, in addition to its flexibility, of requiring very little in terms of materials or preparation.  For groups of 3-6, ages 9 and up.

The rules of the game are simple:

  1. Everyone writes a sentence or idea on a predetermined theme (e.g. “Things you put on a sandwich”) on their own small slip of paper.
  2. A player designated the reader collects the slips (in smaller classes, the teacher is usually the reader).
  3. The reader reads all of them anonymously, including his/her own.
  4. Players except the reader take turns guessing who wrote what.  Obviously, they can’t guess their own.  If they are right (other players have to be honest about whether theirs has bee guessed), that answer is eliminated and the player who wrote it is eliminated too.  The reader can read the remaining answers aloud again as requested.
  5. After all the answers are guessed or after a set number of go-rounds, the player who guesses the most correctly is the winner.

Read on for detailed steps, benefits for language learners, and 工夫 (customizations).

Continue reading

Memory, customized

I figure I’d better put my class activities up on this blog before I forget all of them.  This will probably be the last such entry to feature pictures since almost all our class materials are in garbage bags at this point.  Today’s activity is another take on an old favorite.

Memory, the game of trying to find matching cards which are lying face down, is a classroom staple in Japan (and amusingly called 神経衰弱 しんけいすいじゃく shinkeisuijaku “neurasthenia”), and along with karuta is one you can expect all your learners from preschool up to be able to play without needing to learn the rules, which with younger learners sometimes is an activity in and of itself.

Our version of this game, like our version of Apples 2 Apples, is flexible enough to be used with almost any type of card you have on hand – months, occupations, TOEIC vocabulary, past tense verbs, or whatever you and your students make.  The game is for small groups of 3-6, aged 7 and up.  I suppose you could use it with bigger groups too if you don’t mind copying and cutting a lot of cards for them.

Continue reading