Assessing Robot Writers

Reflections on the professional and pedagogical validity of AI writing

With all the recent commotion over ChatGPT I finally decided to give it a whirl myself. From what I’ve overheard, the writing that ChatGPT produces doesn’t overlap much with what I do now (although that could change). I generally write in two areas. The first is creating copy for professional clients based on specific project goals and feedback. The second is my own creative pursuits, like writing stories. Until experiencing the AI myself, I might have considered these vulnerable areas. But after some practice with the tool and a few focused questions, I’ve gained a better understanding of what AI writing is actually good for, versus where it falls flat. At least for now.

Overview

Here’s an overview before I break things down: I initially decided to test out the effectiveness and overall originality of ChatGPT by having it write me a cover letter. It impressed me in a couple of ways, but failed in others. Right now the biggest drawback of ChatGPT (and this is a well-known issue) is that it writes authoritatively on topics where it has no experience (experience being the territory of carbon-based organisms). Another fault was the AI’s inability (or unwillingness) to ask questions rather than just guess about its topic. This led me to some questions about how it generates content in the first place. I gave it a simpler, more archetypal assignment, which it passed in one sense, but failed in another.

The Task

As I said, my initial test of ChatGPT was to have it write me a cover letter. (Strictly speaking, this was its second assignment, after I asked it what I should bring to an Oscars Party. Have you heard of appetizers and popcorn? They sound great!) This was not a purely theoretical assignment. The cover letter pertained to a job requiring an advanced degree and skill set. I had already written and submitted my own version, which would serve as a point of comparison to what the AI wrote. In my ignorance of how the tool works, I had intended to plug in some key information about an existing job and compare the AI’s cover letter to one I had written myself. But it doesn’t work like that.

In its current form, ChatGPT doesn’t allow you to input specifics per se. You can give it a detailed prompt, such as “Write me a cover letter for an assistant editor position.” It gets tricky adding depth or specificity beyond that point, however. As I’ve noted, one of the issues with this kind of AI writing is that it speaks with an authority it doesn’t possess. It uses machine learning and algorithms to model its content on the well of existing text available online or seeded by its developers. In a sense, this is what human writers do (more on that later). The problem is that the AI can’t tell when it’s out of its depth. And rather than request more information, it starts bluffing.

The Conundrum

That AI bluffs its way through writing tasks is no secret. The tool comes with caveats about plagiarism and the need to review the AI’s work (which calls into question its usefulness as a time/effort saving tool, but that’s another discussion). The biggest concern on this point is that AI writing will proliferate misinformation, often unbeknownst to the people who generate AI written content (or the people who pay them to).

At this stage we reach a conundrum: If you want to replace expertly-written human content with AI writing, you will require an expert to verify that content. If you aren’t interested in expert content or validity of information, you can obviously use ChatGPT to write whatever you want. But this is beside the point. Bad or inaccurate writing is par for the course in the digital world and has been for decades. So what’s the big deal?

Like so many inexpert human writers, ChatGPT is woefully self-assured. Almost comically so. When I asked it to write my cover letter, it forged ahead by inventing past experience and job titles I never possessed. When I challenged its claims or tried to prompt it with more details about my work experience, it responded by making the prior claims more generic. At my prodding, it literally replaced erroneous job titles with vague allusions to “past experience” and my “background.” This makes a certain sense, from an accuracy standpoint. It is safer to speak generally than commit to details. But overall the exercise proved a bit absurd. It was like someone trying to assume my identity, unaware that they were speaking to the original owner. I imagined I was in a cheesy sci-fi movie, confronting my alien doppelganger, but my double hadn’t done its homework.

As you may have surmised, the letter was a failure, both as an experiment, and as a functional document. This type of over-confident flop appears to be typical of ChatGPT. Ian Bogost of The Atlantic shared a similar experience:

I asked ChatGPT to “find me a story in The Atlantic about tacos,” and it obliged, offering a story by my colleague Amanda Mull, “The Enduring Appeal of Tacos,” along with a link and a summary (it began: “In this article, writer Amanda Mull explores the cultural significance of tacos and why they continue to be a beloved food.”). The only problem: That story doesn’t exist. The URL looked plausible but went nowhere, because Mull had never written the story. When I called the AI on its error, ChatGPT apologized and offered a substitute story, “Why Are American Kids So Obsessed With Tacos?”—which is also completely made up. Yikes.

https://www.theatlantic.com/technology/archive/2023/03/chatgpt-api-software-integration/673340/

The Strength of AI Writing

It would be unfair to ignore what the AI did well. Its opening and the general structure of the cover letter were spot on. Based on the way it generates content, this is to be expected. The more examples an AI like ChatGPT has to work from, the more accurately it can simulate that kind of writing. There are a lot of cover letters out there, not to mention scads of templates and articles explaining the process of writing them. As someone who has written a lot of cover letters — for myself, friends, and clients — I’ve seen a lot of what’s out there. A common frustration is that people navigating the job application process can’t determine whether it’s better to be original in their letters or stick to something traditional. My read of the general consensus from hiring professionals is that a traditional approach is less likely to get your letter dismissed outright. The letter that ChatGPT wrote for me, beginning with the typical line “I am writing to express my interest…” would seem to back this perspective. In fact, this was the one line where ChatGPT’s letter and my original one overlapped.

Does that mean it’s great at writing cover letters? Well, not necessarily. Because as I mentioned, a cover letter is an extremely formulaic document by design. It’s why so many templates exist in the first place. And while the AI did a remarkably good job creating a letter based on my fictionalized experience, my personal efforts were a far more effective path to something useful.

Other Options

It’s worth pointing out here that what I really wanted was a tool where I could input specific information and receive targeted feedback or suggestions on my writing, as opposed to developing something from scratch. The thing is, such tools already exist. Applications like Grammarly and Hemingway can already assist you in revising or proofreading your content. Jobscan allows you to plug in a job description and optimize an existing resume or cover letter for the position based on metrics in use by Applicant Tracking Systems.

In my experience, this kind of direct interface is far preferable to the coaxing I had to do to get a serviceable result from ChatGPT. That’s not to say AI writing is without use, particularly if you don’t know where to start. But for discerning content creators, it’s unlikely to be the most efficient method.

Is AI Writing Plagiaristic?

In a certain sense, yes, AI writing essentially copies from what’s out there to generate content. But humans do this too. In writing pedagogy, we call it modeling, or teaching by demonstration. In his benchmark guide On Writing, Stephen King summarizes his advice to new authors in two imperatives: Write a lot and read a lot. A litany of other authors share similar directives.

But is ChatGPT truly “learning” from its online reading (as it claims), or is it borrowing? Part of writing pedagogy involves teaching student writers to cite their sources. We do this for two reasons. The first is academic honesty, a vast subject in its own right, but one that we can accept out of hand for the moment. The second reason is less mired in questions of right or wrong, and closer to the heart of writing. Essentially, when we share information in good faith, it is for the purpose of educating others in some way. The silly assumption is that we want to be truthful. So it makes sense to cite our sources, so that those seeking to understand us can pursue every resource available.

Summary, Paraphrasing, & Quotation

There are three main ways to reference source material within a text: summary, paraphrasing, and direct quotation. Direct quotation is simple – it’s copying someone else’s words, while acknowledging the fact with quotation marks and a citation. Summary is also widely understood as a generalized overview of something someone else said or created, while giving them credit. Paraphrasing lies somewhere in between, a kind of indirect quotation where you explain the specifics of what someone else wrote or said without copying it word for word. What remains is the originator’s idea or the way in which they expressed it. Paraphrasing may sound the most wishy-washy in theory, but is probably the most commonly used form of citation in practice. Look up some of the original quotations that people invoke everyday and you’ll see what I’m talking about.

All of the above are considered legitimate forms of citation. Of course reliance on any of these without giving credit to the original author is considered plagiarism. Tools like Turnitin scan for such instances of plagiarism by comparing one author’s text to the well of existing text at its disposal. When the user of such a tool (i.e. a college professor or editor) runs the plagiarism checker over a submitted text, they see what specific sections and overall percentage of the text overlap with something already in existence. Some overlap is normal, as the result of common rhetorical patterns or phrases. And of course direct quotes reveal themselves, at which point the user decides whether or not the author has correctly credited their source.

Patchwriting

There is a way around plagiarism checkers that works similarly to paraphrasing. It’s called patchwriting, and it involves a very localized approach to rewriting existing material. In patchwriting, one follows the ideas and structure of someone else’s writing, while making changes at the sentence level. This can be as simple as employing synonyms, or rearranging the order of a sentence. It differs from the synergy of writing new content because the patchwriter is exploiting the original author’s efforts to develop ideas and organize thoughts. Patchwriting is essentially a line-by-line paraphrasing of another author’s work without crediting them. It’s sort of the textual version of someone in a business meeting restating your own suggestion as their own. Consequently, it is considered a form of plagiarism. It is also difficult to detect with plagiarism checkers because it isn’t directly copied.

You might expect patchwriting to be a major concern in academia, but this generally isn’t the case. Despite being easier to catch, direct copying of source material remains far more common in lower-level writing courses (at least in my experience). The simple cause for this is that any form of academic dishonesty or general cheating is intended to save time and effort. Careful, undetectable patchwriting might be easier than original composition, but it still demands consistent effort on a level that most cheaters wouldn't consider a worthwhile shortcut.

But what if you had an AI to do it for you? I asked ChatGPT directly if it knew what patchwriting is. Naturally it’s familiar with the term, having the internet at its disposal. What I found interesting was that it immediately referred to patchwriting as a form of plagiarism. I had begun to think that the AI was employing its own sophisticated form of patchwriting to create content (actually, I still think this, but on a more philosophical level). If ChatGPT did resort to patchwriting, while acknowledging the fact, I could leave it at that. But when pressed, it insisted that it does not plagiarize. It claims to use machine learning and algorithms informed by existing content to generate new text. If true, is that so different from what humans do?

It’s clear why so much of the controversy surrounding ChatGPT focuses on education. AI has potential both to enable academic honesty and augment learning. It’s more than an academic specter, as AI could have real impact on teaching jobs, for better or worse. A Princeton study placed English language and literature teachers at the postsecondary level as ranking in the top 20 occupations most exposed to AI language modeling. In fact, the top ten occupations most likely to experience some form of disruption were educational roles (with the exception of Telemarketers, who topped the list). This list was compiled by linking 10 AI applications to 52 human abilities. More on the methodology of that study here:

How Will Language Modelers like ChatGPT Affect Occupations and Industries?

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4375268.

Is All Writing Plagiaristic?

Learning to write is a nebulous process for humans. On the one hand, we have direct rules of grammar and spelling. But even these rules are contextual, interpretive, and in constant flux. And on the other hand we have expression…and that’s another universe of complexity.

People learn by practice, by doing. It’s why a dictionary and a copy of Strunk and White isn’t enough to equip a writer. It takes a lifetime of immersion in the works of other authors to master writing. And in this way, every new piece of writing is like an interpretive dance. It relies on motions that countless other humans have made to generate something completely new.

Taking ChatGPT at its word, it does the same thing, except that the lifetime of immersion has been reduced into a relative instant. I considered if there was any objective difference between the human process and that of the AI. The clear delineator is time. Time isn’t really a factor for an AI, but it matters to humans. In fact, it’s more than a constraint, it’s a vital part of most of our process-based tasks. While a pan simmers on the stove or the paint dries on a canvas, we reflect on our progress and plan our next move. We may even adapt our end goal if the process moves in another direction, embracing a more savory approach rather than sweet, or adding cloud cover to a landscape to make the light more appropriate.

So it goes with writing. Even in the context of a relative short-term project, we perform research on existing materials, then synthesize something new. We may have just looked up the information we’re relying on, or have it backlogged in our brains. But in the interim between seeking information and expressing something new, our brains are still working. It can happen over the course of years, while walking the dog, taking a shower, or even in the span of time it takes to open a fresh Word document. But something happens in that gap: we forget. Some of the information we’ve collected slips away, and consciously or otherwise, we compensate.

Invention

This is where invention occurs. It’s the part where we make up something new to fill in the gaps. It isn’t a lie or a fabrication, it’s a fresh way of stating what’s known, our unique spin.

Earlier I shared how ChatGPT tried to write me a cover letter, using its best guess to flesh out its knowledge of my experience. While this might be seen as its attempt at invention, it isn’t what I’m describing. A resourceful human, in the absence of relevant information, would find a creative way to write around the holes in their knowledge. How do I know? I was a writing professor. But you don’t need me to tell you this. Ask any teacher. Ask yourself. Even for those students who struggle with writing, the saving grace of an essay question is its openness. The interpretive nature of writing gives us room to circumvent our weaker areas at the very least. Really inventive writers play to their strengths.

I gave ChatGPT a simpler assignment, and asked it to write a personal anecdote about falling off a bicycle. It delivered a flawless, relatable, and breathtakingly bland response. But that’s not such an inhuman thing to do. I’ve known students to produce stilted, disassociated reflections on certain experiences because they were writing the way they thought they should, instead of simply expressing themselves.

What I was really after was something more specific, and I got it. Within the story was a detail about a skinned knee. Without specifically asking for it, I wanted to see how the AI addressed pain. It made a decent effort, coupling a clinical description that might have been pulled from WebMD with the invocation of some common emotions: “I felt embarrassed, discouraged.” I asked the AI how it could relate this experience despite its lack of pain receptors, or even knees. It cited the general pattern of pairing clinical adjectives with emotions that I described.

Is Machine Learning the Same as Experience?

We learn language, and by extension, writing, through immersion and modeled practice. According to ChatGPT, it relies on algorithms and a process of machine learning that would seem to emulate this process. For now, the key distinction between the machine and the human writer is a matter of invention. The AI’s function is to create the most correct, plausible, and credible piece of content that it can. Though it may regurgitate misinformation, implausibility is its enemy. It seeks to be accurate.

Humans are not beholden to accuracy in the same sense. We can share our thoughts and experiences in colorful and unconventional ways. We say things that make no sense. An iguana is the opposite of a duck. Airports remind me of fish. Losing my father was just like an ice cream cone. Even when humans speak nonsense, when we break all the rules of communication or expression, we can do so credibly, working from the authenticity of our personal experience. As I understand it, no AI can do this. It might achieve this level of invention in the future, but it would still just be guessing.

Human expression recognizes something beyond accuracy: Truth. To write something true, you have to have lived it.