The black box of take-home writing

This semester I’m trying something new in my writing classes: trying to eliminate the interference of “writing enhancement” software, along with all other potential sources of noise between the students’ brains and the page, from my take-home essays. This is because as an ESL teacher, I need to maintain the validity of the “grammar” scores on writing assignments that I give, assuming that I need grammar scores at all, and of course I need to know that whatever students are turning in is a product of their own thought processes. To that end, I’m changing the planning and drafting processes and part of my grading rubrics.

For comparison, the writing process that I used to use looked like this:

  1. Outline: 5 points in the “homework” grade category
  2. Draft 1: 1 point of the total essay score
    1. typed or handwritten at home
    2. Peer-reviewed only
  3. Draft 2: 9 points of the total essay score
    1. typed
    2. Gets detailed feedback on content, structure, and grammar from me
  4. Final draft: 90 points of the total essay score

And a typical rubric for the final draft looked like this (initially adapted from a few coworkers’ rubrics):

Content


10
The essay has a well-focused thesis.

15The writer supports this thesis in the body paragraphs.

15Sources are utilized well and integrated into the argument.

40Total for content
Organization

5The introduction paragraph(s) captures the reader’s attention and introduces the sources and background enough so that the thesis is understandable to a unknown reader.

15The body paragraphs show clear and effective organization, and have clear idea progression and relationship between paragraphs. The point of each body paragraph is always clear.

5The concluding paragraph readdresses the thesis nicely, does not exactly repeat it, and gives the reader a reason to care.

25Total for organization
Grammar/Language

20The essay has sophisticated, well-chosen sentence structures. The language errors do not interfere with communication. In particular, there should be no errors with noun clauses, comma splices/run-on sentences, hedging, or hypotheticals.

20Total for grammar/language
Format/Writing Process

5Effective use of MLA format including a Work Cited Page.

5Total for format & writing process.


90

Total points

The problems with this approach were 1) a lot of more feedback was given than was actually used for revision, 2) the first draft scores (out of 1) were consistently found to be very predictive of final course grades but were worth very little on their own and 3) I could not tell when the grammar scores I was giving were valid and when I was basically giving Google Translate an A.

Outside of our classrooms, an arms race is being waged between smarter and harder-to-detect ways of generating papers through AI on one side and software designed to detect plagiarism on the other. Copying and pasting still happens (and is the easiest to catch, even without Turnitin.com), but a minimally savvy plagiarist can direct a writing “assistant” generate an essay (Google “generate essays” for examples) or a summary, as I found on a recent podcast episode. At least for the moment, automatic plagiarism-checking software doesn’t catch AI-generated text, whether it comes from Google Translate or Ultron. An add-on to Chrome called Draftback can play back each keystroke in the creation of an essay (or any other Google Doc), potentially catching copying and pasting from AI sources (as copied and pasted text appears all at once as opposed to one letter at a time), but can’t tell who’s sitting in the chair typing text that is entered manually. When I see grammaticality, vocabulary and idiomaticity that is conspicuously improved, I have no way of knowing whether it comes from hard work and scrupulous proofreading or from the magic of smartphones:

I thought English/ESL departments might be some of the first to notice the black box of take-home writing, but others are even more on the cliff’s edge. The transition described in this post was also partly spurred by a conversation that I had in the adjunct work room at one of my community colleges in California, in which a Philosophy professor decried the amount of plagiarism going on in his and others’ classes and told me that he had on good word that UCLA’s Philosophy department no longer gave take-home writing at all. There is, after all, several hundred years’ worth of plagiarizable text on Plato’s Cave.

At my new job I’ve had the chance to talk to a few professors in different departments, and when it comes up, I’m often surprised at how large a portion of their writing assignments has also moved from students’ homes on the weekend to labs on campus during class hours. The reasons stated are usually a combination of wanting to help the students build good writing habits more actively and also simply having no ability to trust what you are getting when an assignment leaves your classroom doors. Some have also said that they dislike the for-profit model of services like Turnitin and Unicheck as well as the message of distrust that they send to students, preferring to keep writing to class hours where at least the pretense of benevolent watchfulness instead of red-pen-policing can be maintained.

I realized that there was a way to kill all of these birds with one stone as well as emphasize the “ideas” part of essays by radically changing my writing process.

The new process looks like this for a non-research essay based on a book or article:

  1. Outline: 10 points of the total essay score
    1. Peer review and instructor feedback on the outline
    2. Done in Google Classroom
    3. Many activities to build robust outlines before Draft 1
  2. Draft 1: 30 points of the total essay score
    1. In-class in a computer lab with only the outline and one page of notes (the outline has whatever quotes they’ve chosen to use)
    2. Typed into the same document as the outline with no other websites or software allowed
    3. Peer review and instructor feedback
      1. Grammar feedback is only on the first 2 paragraphs, and after that only in the form of the COCA tag
      2. All other feedback is on higher-order issues
  3. Draft 2: 60 points of the total essay score
    1. Revised at home and turned in
    2. Accompanied by separate grammar assignments based on Draft 1

What has changed is that:

  • weights for all 3 parts of the writing process are distributed more equally
  • only Draft 1 has a grammar score
  • Draft 2 has grammar assignments in place of a grammar score
  • There are only 2 drafts

Both Drafts 1 and 2 have most of their points given to Content, a bit less for Structure, and a tiny bit for Format/Mechanics. Overall, compared to my old writing process and rubric, more time and more points are given to Content.

(I should also point out that I’m working with a shorter time limit now than I used to – 7-week terms instead of 16-week semesters. Still, I think the important parts don’t suffer much from the eliding of one draft.)

The grammar assignments that I give now in place of a grammar score for Draft 2 are all COCA-derived, and students use my COCA tags in their Draft 1 to know what to look up. This was actually the topic of a talk I gave at ITESOL last month (titled “Using COCA to Simplify Your Correction Codes”), and even if I find reasons to change the 2-draft model outlined above, I will almost certainly be keeping COCA in place of grammar on my rubrics. The assignments are short but open-ended in both the problem (something from their Draft 1) and the solutions.

In addition to changing the process, I try to have prompts that discourage ghostwriting or copying – a combination of new or unusual source texts (Digital Minimalism being a recent example), personalization (the DM essay required screenshots from the students’ own smartphones), and just topics that students want to write about (again, smartphones).

A lot of my former and current colleagues have described moving to a “studio” model of teaching academic writing – lab co-reqs at my last community college, 5-unit plus-sized courses at my current one. Who knows how the proliferation of text-generating technology will affect the “academic essay” in future writing classes?

I joke about this in class, but it’s probably not too far off that we’ll be asking students to turn off their retinal implants before doing anything in class (or generating class content by AI ourselves).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s