The Supported Essay
Assessment Description: This assessment focuses on academic literacy, specifically the planning required for an essay, prompt engineering, and critical thinking. It would be most beneficial for students early in their degree who are still learning what an academic essay is.
Assessment Example: Investigation of the educational affordances and challenges of digital technologies
What is the task? Investigate the educational affordances and challenges of a digital technology you have used in a formal education setting and determine whether it improved your learning. Use ChatGPT to help you write (1600 words, 6 academic references with hyperlinked dois).
Choose a digital learning experience from your formal education and assess its impact on your learning (improved, didn’t improve).
Use Week 1-4 unit content to outline reasons for your contention in the essay plan form (peer-reviewed in Week 4 tutorial).
Use your plan and peer feedback to compose ChatGPT prompts (workshopped in the Week 4 tutorial) and use the responses to help you create your first draft. Do not let ChatGPT take control of the essay – the final product is your own work
Redraft and proofread your essay to ensure your expression is acceptable, your arguments are logical, that it relates directly to unit content, and that ChatGPT hasn’t made anything up.
Include the ChatGPT conversation link and rubric (/40 - Essay plan & prompts-5, Unit content-15, Strength, logic, and clarity of argument-15, Academic literacy-5) at the end, highlighting self-assessment for a clearer understanding of university expectations.
Unit Context: The first-year unit looks at the use of digital technology in education. The assignment requires students to critique digital technology use in education.
Learning Objectives: The LO from the unit is to “demonstrate an emerging knowledge of strategies that can be used to evaluate teaching programs to improve student learning.” Critiquing the use of digitech in a learning activity is the topic of the essay, but using ChatGPT to help write the essay provides students with an extra layer of digitech in education to critique.
Which AI tools & why: ChatGPT because this is already being used by school students. Forcing future teachers to use ChatGPT as part of a high stakes assessment (40% of unit) means that they have that experience to reflect on when deciding whether to use it and how to approach it with their own students.
How is task & AI structured/scaffolded: Students are forced to provide a detailed plan of their essay which is critiqued by classmates and then used to create ChatGPT prompts. We will spend one 3-hour tutorial preparing students to use ChatGPT, using the following activities:
Giving an overview of how ChatGPT works by using predictive text on phones – the intention is to show that a) when it begins a sentence it does not know how it will end, b) the predictive text suggested is based on probability and c) the texts you do make get fed back into the database making the suggestions more relevant, but also more biased.
Engaging in playful tinkering with AI by creating a political campaign video using PowerPoint, using AI to create images, spoken content (including voiceover) and background music.
Exercising critical thinking skills by getting ChatGPT to write about a unit topic and then critiquing it using the rubric criteria as a class discussion, including accuracy, style of writing (paragraphing, jargon, etc.), logic and connection to unit content.
Discussing what they will do with their own students – is this cheating? Will they give explicit instructions on how to use it? What cautions would they give their own students?
Discussing examples of good and bad prompts and each student workshopping one prompt for their essay in class.
How were responsible use and ethical issues addressed:
Bias and the need for critical thinking will be discussed, and as our students will become teachers, discussions of what they would do with their own students would be discussed.
2-3 things that worked well:
Playful tinkering – many students had not used any AI previously and just being exposed to what is available and how it might be used was a valuable activity.
Forcing students to create a plan (graded) and then workshop their plan to make prompts.
2-3 things that need improvement:
Critiquing the ChatGPT output. While many students did understand how to make a good prompt, they still struggled to modify the output so that it created a solid academic essay. This may be because they were not confident in their understanding of the unit content, or perhaps they weren’t used to creating multiple drafts of assignments. I don’t think it was anything to do with using AI. This was a clear issue with at least 15% of the class performing very poorly. However, it also makes me think that this is a good way to judge critical thinking/deep understanding of the unit. Perhaps more in-class activities requiring critiquing are needed?
Things to consider when adapting to your context:
Time. Students’ preparation for this assessment took an entire 3-hour tutorial, and we could have done more.