Assessing Student Submissions in the Era of ChatGPT

May 1, 2023

Anna Johnson

Here’s a typical hallway conversation between colleagues at the college where I teach:

“Hey, have you asked ChatGPT to answer one of your writing prompts?”

“Yes! And I would give its answer a ‘B’ grade, at least. What’s your plan to get around this thing?”

“What’s yours??”

Since the late November 2022 launch of OpenAI’s natural language chatbot ChatGPT (and its earlier release of image generator DALL-E, and, just this month, its more powerful, more accurate GPT-4), many articles have been written about how teachers are handling the new reality that most homework can be done by robots, and done well.

For example, a student can feed a test question into ChatGPT and get a well-written, reasonably accurate paragraph back. This isn’t plagiarism, not technically, because ChatGPT generates its responses rather than linking to existing content like a search engine does.

If one student wrote their own answer to this test question, and another student submitted a ChatGPT-generated paragraph, their instructor would not consistently be able tell the difference.

But don’t panic!

This is not the first time higher education has been disrupted by innovations in information technology. Remember the experts-vs-amateurs furor when Wikipedia first came on the scene? Or the ‘how will you learn if you don’t do it the long way?’ disdain for early versions of Word’s built-citation builder?

Disruptions are always jarring – and this leap in AI capability is certainly a disruption – but disruptions are also opportunities. Think about smartphones in classrooms. Teachers who ten years ago would collect phones in baskets before class could begin now to embrace the reality that every student has a screen in their pocket and build interactive lessons with tools like Kahoot or Mentimeter.

One of my colleagues made this argument with a compelling history lesson: in the 1800s, photography displaced portrait painting, but we still have paintings. Why? Because painters adapted. They shifted from realism-driven portraiture (which photography could do better, faster, and cheaper) to other, more abstract, styles. Maybe, my friend mused, we wouldn’t have Van Gogh’s paintings if we didn’t have photography first.

So, now that AI bots can answer any closed question with a correct answer, what can we ask our students instead? Here are some ideas:

Leverage AI as a learning tool. Search ‘using ChatGPT in the classroom’ for some excellent ideas.
Less ‘what.’ More ‘now what?’ Rather than asking students to compare two concepts from your course, ask which of the two concepts is more interesting to them, and why.
Log off. ChatGPT draws from a huge dataset, but it can’t access offline information. One strategy for reducing the temptation to turn to ChatGPT is to write your own case study and ask test questions about it.