Conversation Starter: Teaching Philosophy in an Age of Large Language Models (guest post)

Over the previous few years we now have seen some startling progress from Massive Language Fashions (LLMs) like GPT-3, and a few of these listening to these developments, similar to thinker John Symons (College of Kansas), consider that they pose an imminent menace to educating and studying (for individuals who missed its inclusion within the Heap of Hyperlinks earlier this summer season, you possibly can learn Professor Symons’ ideas on this here). Within the following visitor publish, Benjamin Mitchell-Yellin (Sam Houston State College) responds to Professor Symons, providing his personal view of the hazards LLMs pose in addition to methods lecturers might make use of to attenuate them.

Dialog-Starter: Instructing Philosophy in an Age of Massive Language Fashions
by Benjamin Mitchell-Yellin

It’s as soon as once more the time of 12 months when these of us who educate philosophy are enthusiastic about the way to construction and ship our programs. In the event you’re something like me, your programs are writing-intensive; your course aims embrace serving to college students enhance their means to assume critically, analyze and assemble arguments, and categorical all of this in writing; and also you’re hoping that the tweaks and revisions you’re making to your syllabus will enhance your personal means to make good on the promise of what a philosophy course has to supply.

I used to be within the thick of planning my fall programs, after I received a little bit of a shock. In “Conversation-Stopper,” John Symons (Kansas) argues that giant language modules (LLMs), just like the comparatively new and much-discussed GPT-3, “will change the normal relationship between writing and pondering.” Because the subtitle places it, they’ll make us “much less clever.” What’s extra, Symons claims, “The LLM marks the top for traditional writing-intensive school programs.” He expects us to see the results clearly as early as this October. (As if we would have liked another excuse to dread midterms!)

My coronary heart skipped a beat. Hasn’t Silicon Valley carried out sufficient harm already?!

Then I believed rigorously about what Symons was arguing and felt quite a bit higher.

Considering via why Symons’ pronouncement in regards to the demise of writing-intensive programs is untimely has been a wholesome train. For one factor, it has made me contemplate the construction of my programs and assignments in a manner I haven’t carried out since early in my educating profession. For one more, it has helped me to see that the troubles he raises, and potential options to them, apply to a number of points. My fundamental competition might be that, regardless of appearances, it’s probably not about LLMs, in any case.

However let’s start with a take a look at Symons’ argument. The center of his case seems to activate the novelty of what we’re now dealing with:

College students have lengthy been tempted by providers that write essays for them and plagiarism is a continuing and annoying function of undergraduate educating, however that is completely different. The LLM marks the top for traditional writing-intensive school programs. Using an LLM has the potential to disconnect college students from the normal technique of writing and analysis in methods that may inevitably reshape their pondering. On the very least, these instruments would require us to rethink the mechanics of writing-intensive programs. How ought to we proceed? Ought to we think about handwritten in-class assignments? Ought to we design extra refined writing initiatives? A number of drafts? 

(I’ll have one thing to say about why it’s unlucky the concluding questions are merely rhetorical. However first, I wish to clarify what I feel it’s that basically drives Symons’ argument. Teaser: it’s fueled by the collapsing of an essential distinction. Learn on, my buddies.)

To his credit score, Symons doesn’t draw back from contemplating the implications of what he’s arguing. He goes on to think about whether or not we should always embrace the disruption and “realign our attitudes to writing,” however demurs, partly, due to his experiences, like these of many people, educating through the pandemic. Technological instruments, similar to Zoom, grew to become the norm as a result of they allowed for socially distant instruction—to the detriment of each instruction and society, it could appear. Symons concludes that the arrival of LLMs implies that “it’s obligatory for school to vary the way in which we consider pupil written work in our programs and extra importantly, to rethink the position of writing in training. … Within the age of the LLM we will be unable to depend on written workouts to make the work of pondering occur. We will even discover that writing abilities which beforehand served as dependable indicators of the virtues we affiliate with pondering can now not achieve this.”

Let the disruption reign!

I don’t imply to be flip. I feel Symons has known as our consideration to one thing essential. It’s simply not what he thinks it’s.

I wish to make it clear that I agree with a lot of what Symons says. For instance, I agree that “writing will be an support to pondering.” I additionally agree with him that it’s seemingly some college students will cross off AI-authored essays as their very own work (some seemingly have already got). And I establish along with his remark that “the arrival of LLMs places me proper again within the place of being a novice instructor.” What I disagree with is Symons’ pessimism in regards to the prospects of writing-intensive programs, and this for causes having to do with that distinction I promised you.

There’s a distinction between educating somebody to assume and write and assessing what they’ve written as a measure of the standard of their pondering. It’s not at all times apparent how the 2 come aside, and so they’re not totally alien from one another. For instance, efficient educating typically entails evaluation. However some fast examples needs to be sufficient to point out that merely grading accomplished essays is just not a dependable measure of pupil studying.

Suppose you could have two college students, Jack and Jill, and so they every flip in essays that obtain the identical grade. Assume, as properly, that there’s no dishonest or bias or something like that. They write the essays to one of the best of their means; you grade them pretty. Though they’ve turned in work of the identical high quality, you shouldn’t take this to exhibit you’ve taught them something, not to mention the identical issues.

Contemplate a primary situation during which each college students flip in A papers. However Jack got here into your class with little to no expertise penning this type of paper, nor did he have a lot in the way in which of background data about philosophy or arguments. He earned his A by paying consideration and placing in plenty of work in your course. Jill, against this, had taken a lot of philosophy programs earlier than, obtained a lot of writing instruction exterior of your course, and mainly phoned it in on this task. It appears protected to say that you just taught Jack quite a bit and Jill nothing.

In a second situation, each Jack and Jill earn Ds on their papers. Once more, Jack got here into your class with little to no related background; Jill got here in with a lot of prior coursework in philosophy. It’s totally attainable on this situation that, as soon as once more, you taught Jack an excellent deal and Jill nada.

What these eventualities exhibit is that when evaluation is used to take a snapshot of a pupil’s abilities and data, it doesn’t let you know whether or not there was progress. To try this, you want greater than time-slice information.

And that brings us to October or, extra exactly, midterms. It’s pretty typical in a writing-intensive course to assign multiple paper. The suggestions (hopefully greater than only a letter grade) on the midterm is formative in nature. College students can use it to enhance on the ultimate. And if we discover ourselves in a 3rd situation, the place Jack goes from a midterm C to a ultimate A and Jill goes from a midterm A- to a ultimate A, maybe we will be fairly assured that you just’ve taught Jack one thing and Jill subsequent to nothing. Typically evaluation may also help us to establish studying.

I’ll guess you’ve already sniffed out the ways in which LLMs (or paper mills or chopping and pasting from the web or no matter) might disrupt this. In a fourth situation, during which one among our preliminary assumptions doesn’t maintain, Jack earns a C on his midterm after which turns to an LLM to “write” his ultimate essay, which receives a B+. An improved evaluation end result, on this context, isn’t a dependable signal of studying.

Up to now, we’ve seen that when writing is used as a mere technique of evaluation, it doesn’t assist us a lot to trace whether or not our college students have realized something in our programs. For this identical motive, LLMs actually do appear to pose a menace to writing-intensive instruction as it’s generally practiced. This new type of dishonest exploits an absence of familiarity with the pondering {that a} pupil’s writing is meant to evince.

However LLMs shouldn’t concern those that use writing (additionally) as a device for educating the kinds of abilities which are typical of philosophy course aims, similar to argument articulation and evaluation. There are a number of acquainted educating methods we are able to make use of in our lecture rooms to assist our college students be taught these abilities and assist ourselves grow to be acquainted with their progress in doing so.

Let’s revisit Symons’ rhetorical questions. Do you have to, as a conscientious instructor of a writing-intensive course, require Jack and his classmates to do all of their writing by hand at school? This may get rid of the specter of utilizing an LLM to cheat. Do you have to require Jack and his classmates to jot down a number of drafts? As many people who already achieve this are conscious, it’s a pink flag every time a pupil turns in a paper that’s so radically completely different than every other written work you’ve seen from them earlier than. However whereas a number of drafts and in-class workouts could also be methods to forestall college students from gaming the system, many readers are seemingly pondering that they’re untenable options. Who has class time to dedicate to permitting college students to pen total essays beneath your watchful eye (or needs to “go full surveillance” on them)? Who has time to present constructive suggestions on obligatory tough drafts?

The excellent news is that in-class writing and a number of drafts don’t must be time-sucks. You possibly can assign college students a brief in-class writing immediate, have them put their names on the tops of their papers, acquire them on the finish of sophistication, and easily use them as a method of taking attendance. After all, it’s also possible to learn what they wrote, or what a few of them wrote, to search out out in the event that they had been greedy the fabric or establishing the argument properly or no matter. However none of this has to take a lot time, and these types of actions can do double-duty—attendance tracker and studying immediate rolled into one.

Who says it’s worthwhile to be the one to learn and supply suggestions on that tough draft? Have college students present suggestions to one another utilizing a rubric you’ve given them (maybe the identical one you’ll use to grade the ultimate model). This may be carried out throughout or between class conferences, and it may be carried out in-person or on-line. Once more, the advantages are a number of: there are alternatives to be taught each within the giving and within the receiving of suggestions. They will flip of their tough draft and peer feedback together with the ultimate model of their essay.

Certain, these could also be among the many best practices relating to efficient writing instruction. However how are you going to ensure they don’t have an AI “write” their tough draft for them?

Attempt utilizing templates to scaffold out your writing assignments. As a substitute of getting college students workshop tough drafts, have them workshop an preliminary define fashioned from filling in sentence stubs on a template you’ve offered. You possibly can even combine templates extra completely into your course. Present them with templates to make use of as guided notes to construction their understanding of the readings—or higher, have them assemble templates like this for one another. Filling out sentence stubs isn’t the identical as writing an essay, however it may be one constructing block with which to erect a cultured paper. And the great thing about templates is that they’ve a built-in construction. For a pupil like Jack, it may be enormously useful to have repeated follow writing inside the confines of a template. The essay then turns into a method of increasing on what he’s realized via doing about the way to construction an argument, versus an opaque request to provide an article modelled on what he’s been studying all time period.

Now, I don’t fake that educating methods like scaffolding, templates, and peer suggestions are surefire methods to get rid of the menace LLMs seem to pose. For one factor, they gained’t get rid of the potential for somebody submitting an AI-generated manuscript to a journal for evaluate. Maybe that is (a part of) what the creators of GPT-3 had in thoughts after they raised the prospect of its getting used for “fraudulent tutorial essay writing.” However I do assume the big range of obtainable methods for educating writing ought to make all of us philosophy professors sleep a bit higher at night time. Maybe we’ll want all of that relaxation, since we’ll have realized we are able to’t merely assign essays and grade them if we wish to be certain our college students are literally studying from us.

Symons notes that educating happens in “embodied and significant social contexts.” I’d add that efficient educating happens in such contexts over time and entails a sure sort of relationship between pupil and instructor. And I’d wish to thank Symons for beginning a dialog that helped me to assume via our craft like a novice.