Most of what reaches teachers about artificial intelligence (AI) comes from vendors or worried headlines. This month brought something more useful: an independent evaluation of a state-funded AI pilot in schools, built on what 735 students and 58 teachers in Connecticut said happened during a year of using AI in class. It is not a glossy success story, which is why it is worth reading.
In short: the AI tools students trusted most were the ones their teachers had built around a specific task, such as a chatbot that checks a draft against the class rubric. Students found AI feedback helpful but still preferred teachers and peers. Many were ambivalent about using more AI and struggled to spot its mistakes, especially in maths. Teachers got more from AI in their own preparation than with students, and their confidence came from training and, above all, time with colleagues.
What the Connecticut AI pilot involved
A 2024 state law asked the Connecticut State Department of Education to run an AI tool pilot during the 2024 to 2025 school year. Six districts took part, with classes from Grade 5 to Grade 12 (roughly Years 6 to 13 in England), though most students were in middle school. Each district chose one of five approved tools: SNORKL, MagicSchool AI, SchoolAI, CK-12 Foundation or Cloud Navigator. Professional development was a condition of taking part.
Researchers from the University of Connecticut and Southern Connecticut State University carried out the evaluation for the Center for Connecticut Education Research Collaboration. The final report and a shorter executive summary came out in October 2026, drawing on the surveys plus focus groups with 32 students and 24 teachers.
The tools students trusted were built by their teachers
The clearest finding is about design rather than brand. Students and teachers got most from resources made for a particular class and assignment, such as teacher-designed chatbots that help students compare their writing with a rubric, study bots and historical character simulations. The report says students were “most enthusiastic about the AI tools that teachers made to align to specific assignments or units.”
One teacher built a chatbot that students used weekly to set personal goals. It “gave them feedback based on the rubric that I had custom-built” and, because students kept returning to it, it could follow their progress before the teacher stepped in.
This is easy to copy. Take a Year 9 History essay on the causes of the First World War. Paste your mark scheme into a custom bot and tell it never to rewrite the student’s work, only to say which level a paragraph meets and what is missing for the next one. The student still writes every word and you still mark the essay. The difference is that the final draft arrives in better shape, so your comments can go on the thinking rather than the structure.
Students valued AI feedback, but not instead of you
Students were broadly positive about AI feedback. On a five-point scale, agreement with “AI feedback helped me improve my work” averaged between 3.6 and 4.3 across the districts. One student called it “like having a teacher in your pocket.” A teacher saw it help “the student who would have been embarrassed to ask me the question”.
Yet the statement with the weakest agreement was “I prefer receiving feedback from AI over other sources”, with averages from 2.6 to 3.2. Students used AI to keep going when a teacher or classmate was busy, not as a replacement for either. Most use came early in the work. Among middle school students, 75.8% used AI to edit or improve writing, while high school students most often used it for background research (61.4%).
The report also punctures the idea that teenagers are natural AI experts. That assumption “did not hold true” in these districts, according to teachers and students alike. The most common view of future AI use was “I don’t care whether we use AI or not”, chosen by 38.4% of students.
The accuracy problem, especially in maths
Students rated their ability to tell when an AI response is incorrect or misleading at 3.4 out of 5, joint lowest of their AI skill scores. They were more confident about checking AI against books, websites and teachers (3.8). Maths came up in every student focus group, with one student describing working a problem by hand and finding the AI had “made a simple arithmetic mistake”.
Teachers had the same frustration. Of those surveyed, 84% used AI to differentiate resources sometimes or often, but only 65% rated those resources good or excellent. One had to correct many errors in AI-written answer keys, and others said checking can wipe out the time saved. More than half (57%) worried about reliability, and the same share about students taking shortcuts or cheating.
There is a classroom opportunity here. Give a Year 10 maths class an AI-generated worked solution containing an error and ask them to find the faulty line. Checking is a skill, and it improves with practice on real mistakes. Some students also liked asking AI to quiz them before a test, which fits what we know about AI retrieval practice, as long as someone has checked the questions.
The most useful AI tool in a classroom is usually the one a teacher has already shaped around the task.
Lessons for leaders planning an AI pilot in schools
For school leaders, the most useful pages are about support rather than software. Before training, 88% of teachers felt only somewhat, slightly or not at all confident with AI tools. Afterwards, 62% felt very or extremely confident. Colleagues mattered even more than courses: 94% said peers improved their use of AI, and 69% wanted more peer collaboration against 49% wanting more training. As one teacher put it, “everybody at some point had some type of snag with the AI. And, it was really nice having each other.”
Inconsistency got in the way. In one school the tool was “locked here and open here”, allowed in the pilot class but not in others, and students said mixed rules made them less likely to use AI. One simply wanted to know “going in that it’s not cheating or it is cheating.” Clear rules matter for assessed work too, as the Joint Council for Qualifications makes plain in its new AI guidance for coursework.
Time was the other recurring theme. “I need the time to play around with the tools myself,” wrote one teacher, “but there is so little time for planning as it is.” England’s Aila trial tells a similar story: teachers saved about 49 minutes a week, but use fell away over the term, as we found in our look at AI lesson planning. Planning your own pilot? These are the steps I would take:
- Start with one task, one rubric and one teacher-built tool, rather than a whole-school licence and a hope.
- Put fortnightly time for pilot teachers to meet on the calendar before the first lesson, and protect it.
- Agree one set of rules on when students may use AI, and apply them in every classroom.
- Check every AI-generated answer key, worksheet and differentiated text before it reaches students, especially in maths.
- Ask students what helped as well as staff. Their ambivalence is useful data.
Schools in England should pay attention. The Department for Education has set out plans for AI tutoring tools for Years 9 and 10, tested in schools under teacher supervision, aiming to make successful tools available nationally from 2027. The Connecticut evidence suggests that how schools prepare teachers will decide as much as which product wins.
What the evaluation cannot tell us
The authors are open about the limits. Participants were keen to explore AI, so they are not typical of every school. The evidence comes from surveys and focus groups, nobody observed lessons, and attainment was not measured, so the report cannot tell you whether students learned more.
Some teachers left unconvinced. “It didn’t solve a problem,” said one. The line I keep coming back to is this: “a trend in education is that if a student can produce the product, they must have mastered the skills to produce the product. That relationship is broken now.” That is the real challenge for any AI pilot: a better product is no longer proof that a student has learned.
Frequently asked questions
What was the Connecticut AI pilot?
It was a state-funded programme in which six Connecticut school districts used approved artificial intelligence (AI) tools with students in Grades 5 to 12 during the 2024 to 2025 school year. Researchers from the University of Connecticut and Southern Connecticut State University evaluated it with surveys of 735 students and 58 teachers, plus focus groups, and published the report in October 2026.
Which uses of AI worked best in the pilot?
Tools that teachers had built around a specific task. Students and teachers rated chatbots that compare a draft with the teacher’s rubric, study bots and historical character simulations most highly. Teachers also found AI useful for preparing and differentiating materials, provided they checked the output.
Did the pilot show that AI improves results?
No. The evaluation relied on surveys and focus groups with schools that were keen to try AI, and the researchers did not observe lessons or measure attainment. It shows how teachers and students experienced the tools, not whether students learned more.
How should a school start its own AI pilot?
Start small, with one task and one rubric, and give pilot teachers regular time to work together. Agree one clear set of rules on when students may use AI, check every AI-generated resource before students see it, and ask students what helped as well as staff.
Has your school run an AI pilot, or built its own rubric chatbot? I would love to hear what worked and what you dropped, so please share it in the comments.
