The most important sentence in a recent Harvard study of AI tutoring sits in its discussion section, well after the results: “AI chatbots are generally designed to be helpful, not to promote learning.”
The results are what got attention. In an introductory physics course in the fall of 2023, 194 students each learned one lesson from an AI tutor at home and another in an active-learning classroom, the kind of well-run, hands-on class that education research already rates highly. With the tutor, the median student’s gain from pre-test to post-test was more than double the gain in class. They took less time, a median of 49 minutes against a 60-minute lesson, and they reported feeling more engaged.
The sentence explains why. The tutor worked because its builders made it less helpful.
What the builders took away
The team published their instructions to the tutor, and most of them are restrictions. Keep responses brief. Give away only one step at a time. Never give the full solution in a single message. If the student asks for the answer straight away, encourage them to try first. Confirm a right answer, but don’t supply one unless the student insists.
Each problem also came with a step-by-step solution the instructors had written, fed to the tutor so its explanations stayed accurate. And the team found the prompt alone couldn’t keep the tutor in order on multi-part problems, so their platform showed students one part at a time.
Everything in that list pushes against what a chatbot does by default. A general assistant is tuned to resolve your request as completely as it can, as fast as it can. For most work that’s exactly right. For learning, it removes the part that does the teaching, which is you struggling toward the answer.
The same model, both ways
A second trial published the same month shows what the default costs. About a thousand high school students in Turkey practised math with GPT-4. One group used a standard ChatGPT interface. Another used a version with teacher-written hints and instructions not to give answers away.
On practice problems, with the AI in hand, the standard group improved 48% and the tutor group 127%. Then the AI was taken away for the exam. Students who had practised with standard ChatGPT scored 17% lower than classmates who never had it. The tutor group showed no significant difference.
Both groups had the same model. The version that did the work left its students worse at the work, while the version that made them do it cost them nothing on the exam. The researchers put it in their title: generative AI without guardrails can harm learning.
The setting that matters most for learning is how much the AI refuses to do.
What this asks of someone learning alone
When I read the Harvard paper, my first reaction was that it needs to be generalized. Even if a tutor like this only matched a good class, it would still be huge, because it doesn’t need a classroom. It works wherever you are, whenever you have an hour, and at your own pace. The study found the pace mattered too. Students who found the class too fast took longer with the tutor, and the ones who found it too slow finished sooner.
Getting there alone means supplying what Harvard’s instructors supplied.
The restrictions are the easy part, because a prompt carries them. The built-in study modes in ChatGPT, Claude, and Gemini now offer similar behaviour, too.
The answer key is harder. A tutor with no key falls back on whatever it remembers, and it’s as confident when it’s wrong as when it’s right. At home, the key has to come from a source you trust, such as a textbook chapter, official documentation, or the text of a standard, with the AI building problems and solutions from that source rather than from memory. If you let an AI gather the source as well, you’ve handed it both halves of the job the instructors did, and you should trust the result accordingly.
The structure falls to you. When the tutor jumps ahead or starts solving, you’re the one who says “one step only.”
And the honour system matters more than it sounds. If you build a lesson with an answer key and then read the key, you’ve turned studying back into rereading, one of the weakest techniques in the research.
What the trials don’t show
Both studies are short. The Harvard post-test came right after each lesson, so it says nothing about what students remembered a month later. The Turkish study measured one exam. Neither followed anyone for long.
The Harvard sample was one course at one elite university, covering two topics, and a reporter who covered the preprint noted that the sessions were scheduled and proctored over Zoom. Students studying unsupervised at home might engage less. The authors themselves say the approach suits first contact with material that has right answers, and they don’t claim it beats a class for complex synthesis or judgement. They argue against replacing in-person teaching with it.
Nobody has tested a do-it-yourself version against their setup, either. A prompt pack can borrow the design, and whether it earns the same result is an open question.
The gap I’d close first is memory. Testing yourself a day later and again a week later costs a few minutes, and practice testing is one of the few study techniques a major review rated highly. It’s also the only way to find out whether the hour stuck.
The part people will skip
The hardest instruction in the whole setup applies to the learner. The tutor holds back the answer, but you can always ask for it, and it will eventually give in, just as the Harvard prompt allowed.
Every tool in this category is one sentence away from turning back into the helpful assistant it was built to be. The design only holds if the person using it wants the struggle more than the shortcut. That was true of good tutors long before AI, and it’s the one ingredient no prompt can supply.