Artificial intelligence (AI) companies make the promise of AI tutoring sound simple: every pupil gets a patient personal tutor for the price of a software licence. This autumn the first large, long trials reported, and the results are more modest than the sales pitch. The tutors can help. The trouble is that pupils hardly ever ask them to.
The short version: In a two-year trial in 18 American middle schools, almost every pupil tried the AI tutor, but the typical pupil messaged it in only 17% of the practice sessions where they got something wrong. Gains were small and similar to ordinary maths practice software. The AI helped most when the task gave pupils a reason to use it, so schools should plan how pupils will use a tutor before deciding whether to buy one.
What the largest AI tutoring trial found
The study comes from economists Philip Oreopoulos and Nina Low and was published as a National Bureau of Economic Research working paper in August 2026. Across the 2024 to 2025 and 2025 to 2026 school years, 18 middle schools in Hamilton County, Tennessee, gave pupils access to Khanmigo, the AI tutor built into Khan Academy, during their daily catch-up maths sessions of 25 to 40 minutes. Khanmigo was set up to coach rather than hand over answers, and it sat one click away beside every problem.
Pupils with the tutor practised more, about 30 extra minutes a week, and their maths scores rose by roughly 0.06 to 0.08 standard deviations over a year. That is a small effect. For a sense of scale, the Education Endowment Foundation (EEF) describes an effect size of 0.08 in its recent Maths-Whizz trial as one additional month’s progress. The authors point out that the gains look much like what Khan Academy practice achieves without any AI, and the tutor cost about $15 per pupil per year. The paper has not yet been peer reviewed.
Pupils almost never asked for help
The usage data are the real story. Almost every pupil, 96%, messaged Khanmigo at least once. Yet the median pupil used it on only a third of the days they practised, and in only 17% of the sessions where they had made a mistake.
When pupils did write, it was rarely a conversation. The researchers classified the messages: 39.4% were bare answers to the tutor’s own questions, 24.2% were clicks on suggested prompts and 12.6% were low-effort replies such as “idk” or requests for the answer. Only 14.5% contained a maths question or a step of reasoning, and 9.3% were off task, including requests for raps and jokes. Anyone who has run a Year 9 intervention group will recognise the pattern.
The authors put it plainly: “whatever an AI tutor can do in principle, this one was seldom asked.” According to the paper, Khan Academy has since redesigned Khanmigo so that it switches on automatically during practice. The researchers also suggest that “human attention is the key ingredient”.
An AI tutor that waits to be asked will mostly be left waiting.
When AI tutoring does make a difference
A second paper from the same team, Making AI Tutoring Productive, tested what happens when the task design forces the issue. More than 6,000 middle school pupils in the same district completed a maths review in March 2026 on a purpose-built practice platform. They were randomly given AI help or no AI, and either a mastery rule, where they had to get three questions right in a row before moving on, or free movement through the work.
Pupils with AI moved more slowly and attempted fewer questions, but they were more accurate and recovered better after a mistake. Mastery on its own did not improve results on a test a week later. The best delayed results came when AI was built into the mastery workflow, although the gains were modest and only marginally significant. The authors call what is needed “productive friction”: a reason to slow down and use help at the right moment, without spending a whole lesson on a few questions.
Their other ideas will sound familiar to teachers: ask pupils to explain an error, ask for a confidence rating before feedback, or add a delayed retrieval check. Their conclusion is that AI tutoring “should be evaluated as part of a learning environment” rather than as a product on its own.
The wider evidence is mixed
A trial in rural China, published in the Economics of Education Review and summarised by Stanford’s Freeman Spogli Institute on 17 October 2026, randomly assigned 8,647 pupils in 315 primary classes to AI-personalised maths software, ordinary software or no software. The AI version did not significantly improve achievement, and the researchers say their estimates are precise enough to rule out any meaningful positive effect.
Closer to home, the EEF’s trial of Maths-Whizz, an AI-enabled adaptive tutoring programme, involved 63 English primary schools and 10,369 pupils in Years 2 to 5. Pupils made one month’s additional progress in maths, a result the EEF rates at four out of five padlocks for security. Pupils eligible for free school meals made two months’ additional progress, though that finding is less secure. Pupils averaged only 32 minutes a week against a recommended 45 to 60, and the cost was about £22 per pupil per year. Note that Maths-Whizz sets the work and paces pupils through it rather than waiting for a question. The full evaluation report from the National Foundation for Educational Research is worth reading.
What this means for schools in England
The Department for Education (DfE) is running an AI Tutoring Tools Pioneers Programme, designing generative AI tools for Year 9 and 10 pupils with around 30 partner schools. A parliamentary written answer on 16 October 2026 says this design work runs until March 2027. Tools that pass safety and teaching tests will then be trialled with the EEF through the department’s EdTech Testbed Programme. National evidence on chat-based tutors is some way off.
If your school is already using an AI tutor, or about to, the research points to a few practical steps:
- Put it in the timetable. In Tennessee, the extra practice came from using the tool inside protected session time.
- Give pupils a reason to ask. A mastery rule, or a requirement to explain a mistake before moving on, makes help worth seeking.
- Prompt help-seeking yourself. A teacher saying “you’ve got two wrong, ask the tutor why before you try again” does what the software’s suggested prompts did not.
- Read the chat logs as well as the scores. If most messages are single numbers, pupils are using the tutor as an answer checker.
- Judge small gains against cost and against simpler options, such as ordinary adaptive practice or small group teaching.
- Check the provider’s safety and data practices first, using the questions in our guide to AI chatbots and safeguarding.
Imagine a Year 9 catch-up group of eight working through simultaneous equations on laptops. Rather than leaving the tutor as an optional button, the teacher sets one rule: after any wrong answer, pupils ask the tutor one question in their own words, then write a single sentence in their book explaining the mistake before trying the next question. The teacher circulates and reads the sentences. That small rule turns the tutor from something pupils ignore into part of how the lesson runs.
These studies have limits. Most are in maths, the two Tennessee papers are not yet peer reviewed, and the tools are changing fast. None of them tests a carefully designed AI tutor against a skilled human tutor working with a small group, and the Tennessee authors warn against reading their results as a case for replacing tutors or teachers.
The pattern is consistent with what we found when looking at AI and homework: when AI makes it easy to skip the thinking, many pupils will. It also echoes the Connecticut AI pilot, where pupils trusted the tools their teachers had built around a specific task most. The routine matters more than the software.
Frequently asked questions
Does artificial intelligence (AI) tutoring improve pupils’ results?
Modestly, in the best trials so far. A two-year trial of Khanmigo in Tennessee found gains of about 0.06 to 0.08 standard deviations a year, similar to maths practice software without AI, and a large trial in rural China found no significant effect. How the tutor is used matters more than the tool.
Why don’t pupils use AI tutors more?
In the Tennessee trial the typical pupil messaged the tutor in only 17% of the practice sessions where they made a mistake. Pupils who are struggling are often the least likely to ask for help, and most messages were bare answers or clicks on suggested prompts.
How can teachers make AI tutoring more effective?
Give pupils a reason to ask for help. Use the tutor in timetabled lessons, require pupils to explain a mistake before moving on, use mastery rules or confidence ratings, and read the chat logs to see whether pupils are thinking or just checking answers.
Is the Department for Education trialling AI tutors in England?
Yes. Its AI Tutoring Tools Pioneers Programme is designing tools for Year 9 and 10 pupils with around 30 schools until March 2027. Tools that pass safety and teaching tests will then be trialled with the Education Endowment Foundation.
Have you used an AI tutor with a class, and did your pupils actually ask it anything? Tell us what worked, or what did not, in the comments.
