An AI answer is not evidence of learning

Two different things can look like progress

Imagine a learner who asks an AI assistant to explain a mathematics problem. The explanation is tidy, the answer is correct, and the assignment is submitted. Now imagine asking that learner to solve a slightly different problem without the explanation in front of them. These are different tasks. Success at the first does not tell us whether the second is possible.

That distinction is useful even when no AI is involved. A copied worked example, a polished slide deck, or an encouraging practice score can create the appearance of understanding. The question is not simply whether the tool produced something good. It is what the learner can explain, reconstruct, notice, and apply afterwards. This article proposes a small teaching routine, not a claim that any particular app improves attainment.

What the evidence does—and does not—say

The U.S. Institute of Education Sciences' 2007 practice guide, Organizing Instruction and Study to Improve Student Learning, recommends spaced learning, alternating worked examples with problem solving, and questions that require explanation. It assigns different evidence ratings to its recommendations; they should not be treated as equally established or as a guarantee for every learner. Its relevance here is the distinction between exposure to an answer and opportunities to retrieve and use knowledge.

There is a second issue: answer reliability. NIST's 2024 Generative AI Profile identifies confidently incorrect output as a risk. A convincing explanation or citation is not, by itself, verification. These sources support checking learning and checking information. Neither evaluates SASHA, and neither establishes that a chatbot, retrieval system, or study routine removes every error.

Try an attempt–hint–explain–transfer routine

Start with an attempt. Ask the learner to write what they understand, where they are stuck, and one possible next step. A short attempt is enough; this is not a punishment for needing help. The point is to make the starting position visible before the explanation arrives. A teacher or tutor can adjust the amount of help to the task, accessibility needs, and permitted accommodations.

Then request a hint, not an entire completed assignment. For example: 'Point me to the part of my reasoning I should check. Do not solve the whole problem yet.' This is an instruction to the tool, not a guarantee of its behaviour. If it gives away the answer, choose another practice example or ask the educator how to continue. Keep the learning objective, not the chatbot conversation, in charge.

Next, close or cover the explanation and explain the idea in ordinary language. What changed in your reasoning? Why does that step follow? Which part remains uncertain? Finally, try a fresh example with different details. For a reading task, that might mean identifying the argument in a new paragraph; for a science task, predicting what changes when one condition changes.

Make the check small enough to use

A tutor does not need a surveillance dashboard to begin. A simple record can contain the topic, the help used, one explanation in the learner's own words, and the next practice question. Avoid scoring confidence as competence. 'I feel better about this' is valuable feedback, but it is not the same observation as 'I solved a new example and justified the steps.'

An original five-minute check might be: one minute to name the idea, two minutes to attempt a new example, one minute to compare against a trusted reference, and one minute to choose the next question. That timing is a suggested workshop exercise, not a research-proven optimal schedule. Change it when fatigue, complexity, language, or access needs make another approach more appropriate.

Keep the boundaries visible

Use the course's approved materials to check factual claims. Follow the institution's rules about AI and assessed work. Do not paste private student records into an unapproved service just to personalize a prompt. If the tool and the trusted reference disagree, keep the disagreement visible and ask a qualified person to help resolve it.

For parents, the most useful question may be 'Can you show me what you understand now?' rather than 'Did the app say you finished?' For developers, the equivalent question is whether the product makes uncertainty and independent practice visible. A good-looking answer is an output. Learning needs its own evidence.