Beyond Recall: A New USask Exam Built Around Critical Engagement with AI

As generative artificial intelligence (AI) reshapes higher education, Dr. Kyle Anderson (PhD) of the University of Saskatchewan is responding to a challenge many instructors now face: how do we design assessments that still measure meaningful learning when AI can produce fluent, confident answers with minimal effort? Anderson, an Assistant Professor in Biochemistry, Microbiology & Immunology, is experimenting with a new assessment model designed to test one skill students will increasingly need: verifying AI-generated information.

Rather than asking students to simply recall facts, Anderson's "fact-check" exam questions present them with realistic AI-generated passages containing deliberate errors. Students must identify and correct the mistakes while leaving accurate statements untouched. The approach reflects what Anderson calls the role graduates will play in an AI-enabled workforce: "being the human in the loop who verifies fluent, confident, mostly-right output before it reaches a patient, a dataset, or a decision." In that sense, the assessment prepares students for workplaces where AI may draft the first answer, but humans remain responsible for deciding whether that answer is accurate, appropriate, and safe to use.

Why verification matters

That shift speaks directly to what faculty and students are already noticing. In a global spring 2026 AI-focused survey completed by USask faculty and students, more than 80% of both groups at USask identified decreased learning, especially reduced critical thinking, as a concern related to AI. Faculty also identified assessment design as the most common area where they needed support. Students shared related concerns: 74% worried classmates may misuse AI and gain unfair advantages if assessments are not redesigned.

The course itself becomes the assessment engine

The approach is especially well suited to content-heavy courses, where students must learn rules, generalizations, details, and exceptions. To build the assessment, Anderson used Anthropic's Claude, one of USask's newly available protected AI tools for faculty and staff, and gave it access to a rich record of his own teaching: auto-generated transcripts of more than 26 hours of lectures (roughly 220,000 words), 480 slides carrying his in-lecture pen annotations, and more than 400 past exam questions with keys. That meant Claude was not working from a syllabus or generic biology knowledge alone. It could draw on Anderson's examples, explanations, emphases, and exceptions—the details that shaped how students encountered the material. Anderson estimates only a fifth of the usable information came from slides; the rest from what he said, wrote, and drew in each video. Because the tool is protected, Anderson could use course materials without the same intellectual property concerns associated with public AI tools.

For Anderson, the strength of the approach comes from the constraints he gives the AI. The generator follows more than fifty written rules for what a question may be: errors must be fixable from the mechanism, not memorized trivia; statements that sound wrong but are correct must be verifiable by reasoning; and anything that does not map to a learning outcome gets rewritten. Once those rules are specified, they can be applied to whatever the course covered that term. Claude then produced draft exam passages that mixed accurate statements with carefully designed errors, which Anderson reviewed and checked against his own teaching. The first build took about seven hours of his time: five for design and prompting, and two for writing through the exam himself and checking the AI-generated key against his course materials and teaching. He expects future versions of comparable quality, plus student practice samples, to take about two hours.

Security does not rest on keeping questions secret. Students write in person, on paper, and because each exam is built from that term's lectures, past questions are less useful for students to circulate in later offerings.

What the exam revealed

Anderson used the format in an initial offering with 33 students, who responded as intended: their marked-up exams show them working claim by claim. In some cases, students "corrected" statements that were already true, a behaviour conventional exams give no occasion to reveal, since a student who quietly misunderstands a correct statement is never asked to act on it. Each question was marked against a set key, with some flexibility for answers that corrected the science in a different but valid way; a formal analysis is underway. He also learned that the practical details matter: because students wrote the exam on paper, scanning quality and even pen thickness affected how easily scripts could be reviewed and marked—lessons he has since addressed for the next offering of the course. The format also helped separate different learning challenges: noticing errors in fluent prose is not the same as knowing key terms or interpreting figures.

Adapting to your course

For instructors already rethinking assessment in content-heavy courses, the model offers a practical starting point: use existing course materials to create tasks where students must judge the accuracy of AI-generated explanations, recommendations, summaries, or analyses. Anderson has written the method up as an instructor guide (fairness rules, master prompt, audits, a released exam question with its marking key) and is glad to share it. He recommends starting with the protected AI tools now available at USask.

Faculty interested in adapting the approach can attend a workshop led by Anderson in October and learn more about USask's AI guidelines, resources, and learning opportunities at ai.usask.ca.

---

^{Title image credit: TungArt7 on Pixabay.com
This article was created with the assistance of AI tools, as described in the GMCTL AI Disclosure Statement.}

This resource is shared by the Gwenna Moss Centre for Teaching and Learning (GMCTL), University of Saskatchewan, under a CC BY-NC-SA license.