What it is
An AI tutor for practice is a chatbot configured to behave like a tutor. It asks the student to try first, gives hints instead of answers, walks through a problem in order, and checks its help against solutions you wrote. The configuration matters more than the model.
The strongest evidence so far comes from two randomized studies. In a field experiment with nearly 1,000 high school math students, unrestricted GPT-4 access raised practice scores but lowered scores on a later exam taken without AI, while a version built to hold back answers largely avoided that harm.1 In a randomized trial in a Harvard physics course, students who learned new material with a purpose-built AI tutor learned significantly more in less time than students in an in-class active learning session.2
Evidence and fit at a glance
Two randomized studies point the same way, but each is a single site and a single subject, and each measured outcomes soon after practice. The tutor in the college trial was custom built with expert-written solutions. The evidence is labeled emerging.
- Set-upWriting and testing the tutor's instructions, plus a solution and common errors for each practice item, takes the most time. Start with one problem set.
- Class timePractice sessions of 30 to 90 minutes. In the college trial, the AI group's median time was 49 minutes for a lesson planned at 60.
- DeliveryIn person, Live online, Self-paced online
- Class sizeAny class size
- AI roleBuilt around AI
When it works, and when it doesn’t
Best when
- Students practice problems with checkable answers, such as math, physics, accounting, or programming, and you can supply worked solutions for the tutor to follow.
- Practice happens outside class, where you cannot be present, and students are already using general chatbots on their own.
- You pair the tutor with a no-AI check, such as a quiz or an in-class problem, so you can see what students can do alone.
- You have time to build the tutor's instructions, test them against wrong answers and "just tell me" demands, and review its output.
Weak when
- Students use a general chatbot that gives answers on request. In the high school experiment, unrestricted access improved practice scores but left students about 17% below the control group on the later exam without AI.1
- The goal is synthesis or critical thinking. The authors of the college trial do not presume that AI tutoring will beat active learning for tasks that need complex synthesis or higher-order thinking.2
- You cannot supply or check solutions. The college tutor relied on step-by-step answers written in advance, and the authors say whether it could work without them is an open question.2
- The outcome that matters is retention weeks later. The college trial measured a post-test right after each lesson and did not assess retention.2
- The task is writing or open-ended inquiry. In studies of university students using ChatGPT for these tasks, polished output did not come with matching gains in reasoning or transfer.3,4
How to do it
- Choose practice with checkable answers.A problem set, worksheet, or set of coding exercises where each item has a correct solution you can write down.
- Write the solution and the common errors for each item.Both studies supplied the tutor with expert-written solutions. The high school tutor's instructions also included common student errors and feedback guidance, which the authors say reduced incorrect responses.1 The college tutor was given detailed step-by-step answers instead of relying on the model to solve each problem.2
- Tell the tutor to hint, not tell, and to go in order.The high school tutor was instructed to give hints rather than answers.1 In the college trial, the authors found that instructions alone did not reliably keep multi-part problems on track, so the platform walked students through each part in sequence.2 If you cannot build a platform, use a tool that lets you fix the instructions and attach your solutions, and test it yourself.
- Test it before students do.Give it wrong answers, vague answers, and "just tell me the answer." Revise until it holds.
- Keep a no-AI check.Give a short quiz or problem without AI on the same material. In the high school study, students with unrestricted access did not think they had learned less, even though their exam scores fell, so student self-reports are not a safe check.1
- State the rules and the reason.Tell students what the tutor is for, which tools are approved, and why you ask for AI-free checks.
With AI and without
Without AI
This technique needs AI. If AI is not allowed in your course, the same goals are served by worked examples, retrieval practice, and peer instruction, which are covered on their own pages.
With AI
Use a tool where you control the instructions and can attach your solutions, and one your college has approved, so that student data stays protected. Review a sample of transcripts to see where the tutor gives away answers or gets a step wrong. Do not let the tutor grade anything that counts.
What the research says
What the randomized studies found
In the high school experiment, nearly 1,000 ninth- to eleventh-grade students in about fifty math classes at one large school in Turkey completed four 90-minute review sessions. Classes were randomly assigned to textbooks and notes only, a tool resembling the public ChatGPT interface ("GPT Base"), or a tutoring version of GPT-4 ("GPT Tutor") that gave hints rather than answers and carried teacher-written solutions and common errors. During practice, GPT Base improved scores by about 48% and GPT Tutor by about 127%, relative to the control group. On a later exam without AI, GPT Base students scored about 17% lower than the control group, and GPT Tutor students performed about the same as the control group, with no detectable benefit or harm.1
The same study asked students how much they thought they had learned. GPT Base students did not think they had learned less, even though their exam scores fell, and GPT Tutor students felt they had performed better, though their exam results did not show a gain over the control group.1
In the college trial, 194 eligible students in an introductory physics course for life-science majors at Harvard completed two lessons in consecutive weeks, one with a custom AI tutor at home and one in an in-class active learning session, in a crossover design. The AI group scored higher on the post-test and learned in less time, with a median of 49 minutes, against 60 minutes planned for the in-class lesson. The authors report effect sizes of about 0.63 standard deviations from linear regression and roughly 0.73 to 1.3 from quantile regression, and say many AI-group scores were near the maximum, so the post-test may have underestimated the gain. Students also reported feeling more engaged and motivated.2
What the authors say limits these results
The college lessons covered material students were meeting for the first time, at the understanding, applying, and analyzing levels of Bloom's taxonomy. The authors say results may depend on the student population, high-quality videos, a capable model, expert-written question-specific prompts, and a structured framework, and they call for replication.2 The high school authors note that the study covers one subject, one school, and one country, measured short-term outcomes, and took place when generative AI was new.1
Why open-ended AI use is a different case
In a randomized experiment, 117 university students did a writing task with support from ChatGPT, a human expert, learning analytics tools, or no extra tool. The ChatGPT group improved essay scores but did not show matching gains in knowledge transfer. The authors describe a risk they call "metacognitive laziness," in which learners lean on the AI in ways that reduce their own self-regulation.3
In another randomized study, 91 university students researched a science question about nanoparticles in sunscreen with either ChatGPT-3.5 or Google. Students who used the language model reported lower cognitive load, but their final recommendations and justifications showed lower-quality reasoning and argumentation than those of students who used search engines.4
Pedagogical justification, ready to paste
Use this in a lesson plan, syllabus, or course-design form. Replace the [bracketed] parts with your own course details. Check the claims against the references before you submit.
One-sentence version
Students practice with an AI tutor configured to give hints rather than answers, followed by an AI-free check, because unrestricted GPT-4 access improved practice scores but lowered later exam scores in a randomized field study of nearly 1,000 high school students, while a tutor with guardrails avoided that decline,1 and a purpose-built AI tutor produced more learning in less time than in-class active learning in a randomized trial in a college physics course.2
Full rationale
Because students are likely to use AI for practice with or without a course policy, practice problems are supported by an AI tutor configured to give hints, follow instructor-written solutions, and work through problems in order. In a randomized field experiment with nearly 1,000 high school students, unrestricted access to a general AI tool improved practice scores but lowered scores on a later exam taken without AI, while a version with tutoring guardrails largely avoided that harm.1 In a randomized trial in an introductory college physics course, students who learned new material with a purpose-built AI tutor learned significantly more in less time than students in an in-class active learning session.2 Because the benefit depended on the tutor's design, and because ChatGPT use for open-ended tasks was linked to polished output without matching gains in transfer or reasoning,3,4 each practice set is followed by a short AI-free check, and tutor transcripts are sampled for errors. The evidence is early and comes from single-site studies, so the course compares AI-free check scores with a previous term.
- Learning outcome
- Students will be able to [verb + skill] on [problem type] without AI assistance.
- Evidence of learning
- [AI-free quiz or in-class problems] on the same problem type after each practice set, compared with a previous term or with a comparison section.
- Learning activity
- [N] practice sets with an AI tutor configured to give hints and follow instructor solutions, each followed by [a short AI-free check].
The copy buttons paste plain text. The pasted version renumbers the references from 1 and includes them with a source line, so it works on its own. The numbers shown here match the reference list below.
Limits of the evidence
- Both randomized studies are single-site and single-subject. One is high school math in Turkey, and the other is two topics in one Harvard physics course.
- Both measured outcomes soon after practice. The college trial did not assess retention, and the high school study measured short-term outcomes only.
- The college tutor was custom built, with expert-written solutions and sequenced steps. Results with a general chatbot, or with a tutor you configure in a general-purpose tool, may differ.
- The high school study took place when generative AI was new. Models have improved since, so the exact size of the harm from unrestricted access may differ now.
- The studies of ChatGPT in writing and inquiry tasks show risks for reasoning and transfer. They do not test a guardrailed tutor, and this page does not claim they do.
- Tools change quickly. Treat any specific product as a candidate to test, not an endorsement.
Questions faculty ask
Will letting students use ChatGPT hurt their learning?
It can. In a randomized field experiment with nearly 1,000 high school math students, unrestricted GPT-4 access improved practice scores but left students about 17% below a no-AI control group on a later exam without AI. A version designed to give hints rather than answers largely avoided that harm.1
Does an AI tutor work better than active learning?
In one randomized trial in one Harvard physics course, students learned more in less time with a purpose-built AI tutor than in an in-class active learning session, on material they were seeing for the first time. The authors do not presume this holds for every task and call for replication.2
References
- Bastani H, Bastani O, Sungu A, Ge H, Kabakcı Ö, Mariman R. Generative AI without guardrails can harm learning: evidence from high school mathematics. Proc Natl Acad Sci U S A. 2025;122(26):e2422633122. doi:10.1073/pnas.2422633122
- Kestin G, Miller K, Klales A, Milbourne T, Ponti G. AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting. Sci Rep. 2025;15(1):17458. doi:10.1038/s41598-025-97652-6
- Fan Y, Tang L, Le H, et al. Beware of metacognitive laziness: effects of generative artificial intelligence on learning motivation, processes, and performance. Br J Educ Technol. 2025;56(2):489-530. doi:10.1111/bjet.13544
- Stadler M, Bannert M, Sailer M. Cognitive ease at a cost: LLMs reduce mental effort but compromise depth in student scientific inquiry. Comput Human Behav. 2024;160:108386. doi:10.1016/j.chb.2024.108386
About this page
Written by Jesse Rogers, Learning Lab Coordinator, Palm Beach State College. Last updated October 15, 2026.
AI disclosure. Model used for research, drafting, editing, and verification: claude-sonnet-5-5 (Claude, by Anthropic). The author is responsible for every claim and citation.
How it was checked. Bibliographic details were checked against DOI records. Numbers and findings were checked against abstracts or open full texts. Where that was not possible, the page says so. See the method page.
What I could not confirm.
- Study details for the high school experiment (sample, arms, the 48%, 127%, and 17% figures, and student perceptions) come from the article's full text through Europe PMC. Details for the college trial (194 students, effect sizes, time on task, and the stated limitations) come from the open-access article. The percentages are as reported by the authors, relative to the control group.
- For the two studies of ChatGPT in writing and inquiry tasks (Fan 2025, Stadler 2024), only the headline findings were checked, against the abstracts and the publisher's metadata. Details beyond the abstracts are not reproduced here.
License. CC BY 4.0. Share and adapt it with credit. This is an independent project and is not an official resource of Palm Beach State College.
Corrections. If you find an error, message me on LinkedIn and I will check it and fix it.
Cite this page (AMA). Rogers J. A guardrailed AI tutor for practice. Learn With Jesse. Published October 15, 2026. Updated October 15, 2026. Accessed [date]. https://learnwithjesse.com/techniques/ai-tutor/