Most teachers I know have signed up for a free artificial intelligence (AI) tool on a Sunday evening without reading the terms. I have done it myself. This week AI and data protection reached the House of Lords, where the government said it will consult this autumn on certifying AI tools for schools. Until that list exists, the checking falls to us.
The short version: An audit of 28 education technology suppliers by the Information Commissioner’s Office (ICO) found that almost 70% were acting as controllers for some uses of children’s information without realising it. Some had used pupils’ information to train AI features. Contracts with schools were often not detailed enough to say otherwise. The government now plans a certified list of safe AI tools, but schools still sign the contracts, so leaders need a short set of questions to ask before any AI product meets a pupil.
What the Lords heard on 26 October
In a short Lords question on Monday, Baroness Berger asked whether the government would publish a single approved list of AI tools that meet safeguarding and data protection standards. The education minister, Baroness Smith of Malvern, said a planned consultation on safety certification for higher-risk products “would lead to a published list of child-facing generative AI tools” that meet criteria including data protection and safeguarding. Asked for a timetable, she said the consultation and certification framework would launch “very soon”, this autumn.
The minister also said the Office for Standards in Education, Children’s Services and Skills (Ofsted) would have “an important role to play” on safeguarding, with the ICO responsible for data protection. Lord Clement-Jones asked for a “human flourishing test”, checking whether a tool actually improves children’s learning rather than simply avoiding harm. A safe tool can still be a pointless one.
What the regulator found inside edtech companies
Baroness Berger quoted a recent ICO audit. The report, Edtech examined, was published in June. It covers consensual audits in 2024 and 2025 of 28 providers whose products are widely used in schools across the United Kingdom (UK), from management information systems to classroom apps. Information security was mostly strong. The problems were elsewhere.
Almost 70% of the providers were acting as the controller for some uses of children’s information without recognising it. The controller is the party that decides why and how data is used, and it carries extra legal duties that these firms were not meeting. Many used pupils’ information to analyse product performance or develop new products. Some used it to train AI features. One had used children’s information to create anonymised pupil profiles to sell to third parties doing education research. Some suppliers relied on subcontractors whose terms said they would keep a copy of children’s information to train their own AI, and the providers only found out during the audit.
Contracts were a big part of the problem. The ICO found contracts with schools were often not detailed enough, and around 70% of providers could not show that the school or child had instructed or authorised those extra uses. It also found new features switched on by default and cases of AI functionality introduced “without adequate safeguards”. The providers accepted 98% of the 596 recommendations made, which is encouraging. Still, these were the companies willing to be audited.
If the contract does not say what happens to pupils’ data, the supplier usually decides for you.
Why AI makes the old questions sharper
The Department for Education (DfE) gives a telling example in its guidance on procuring edtech. A school wants an AI writing tool to help pupils with spelling and grammar. The tool only asks for a name and email address, but pupils write about their weekend, perhaps a visit to a place of worship or a medical appointment. The school learns that the data is stored outside the UK and that anything pupils type could be used to develop the AI. That feature cannot be switched off, so the school walks away.
That detail never appears on a supplier’s homepage. A maths quiz app knows a pupil’s score. A chat-based AI tool knows whatever a ten-year-old chooses to tell it.
Parents are already uneasy
In a YouGov survey for the ICO in March 2026, only 28% of UK parents were comfortable with AI being used in their child’s school to evaluate learning, while 49% were uncomfortable. On AI that personalises learning, opinion was split, with 35% comfortable and 39% not. Those figures sit awkwardly beside the growing use of AI in marking, which we looked at in AI marking.
A story from Rhode Island this week shows how quickly trust can go. Parents in Providence said they heard about an AI reading program that listens to young children reading aloud only secondhand, and were told they could not opt their children out. They want to know what happened to voice recordings made before the district changed its settings in August, and how to get them deleted. One parent said a child with a lisp had to repeat words because the software did not understand him. That echoes the special educational needs and disabilities (SEND) questions we raised in AI and SEND. District leaders say the system no longer keeps new voice recordings. The settings may now be fine, but the communication clearly was not.
An AI and data protection checklist before you sign
None of this needs a lawyer for every app. It needs the school’s data protection officer (DPO) and designated safeguarding lead (DSL) involved before anyone signs, which is exactly what the DfE guidance asks. These are the questions I would want answered in writing:
- What personal data does the tool collect, including anything pupils type, say or upload, and why is each item needed?
- Is anything pupils enter used to train or improve the AI, and can the school switch that off?
- Which subcontractors see the data, and do any of them keep copies for their own AI?
- Where is the data stored, how long is it kept, and how is it deleted when the contract ends?
- Are new features switched off until the school turns them on?
- Can staff see what pupils have typed, so that the DSL can check use and spot concerns?
Record the answers in a data protection impact assessment (DPIA), the risk assessment that UK data protection law requires for high-risk processing, and update it whenever the tool changes. Our guide to AI chatbots and safeguarding covers the safeguarding questions to add.
For staff using general AI tools, the DfE’s guidance on generative AI and data protection has a simple habit worth copying. An administrator who wants help wording an email to a parent about behaviour gives the AI the notes without any personal data, then adds the pupil’s name and class afterwards. The same works for report comments. Ask for a comment about a Year 7 who has improved her fractions but rushes her working, and type “Maya” yourself.
What happens next
Several pieces are moving at once. The ICO says it is working with the government on secondary legislation that will require it to produce a new code on children’s personal information in educational digital systems. On 26 October the University of Oxford’s Department of Education launched a six-month project with the DfE, Ofqual and Oak National Academy on how pupil data could be used responsibly for AI research. It will recommend a governance framework and ideas for future AI assurance and certification.
Ofsted, meanwhile, says it does not inspect AI as a separate area. Its statement on AI says inspectors may ask about AI and personal data in the same way they would about any other collection and storage of personal data. So a leader should be able to name the AI tools in use and show who approved them.
My view is that a certified list will help, but it will not do the job for us. It will cover higher-risk, child-facing tools, and it will take time. The free apps teachers find on a Sunday evening will not wait. The cheapest protection is a culture where nobody puts a pupil’s name into an unapproved tool, and where trying something new starts with a two-line email to the DPO.
Frequently asked questions
Can teachers put pupils’ names into artificial intelligence (AI) tools?
Only into tools the school has approved for that purpose. Department for Education (DfE) guidance suggests giving a general AI tool the task without personal data and adding names afterwards. If personal data has to be entered, check with the school’s data protection officer (DPO) first and make sure it will not be used to train the AI.
What did the Information Commissioner’s Office (ICO) find in its edtech audits?
Audits of 28 education technology providers in 2024 and 2025 found strong information security but weak contracts. Almost 70% of providers were controllers for some uses of children’s information without recognising it, and some had used children’s information to train AI features. Providers accepted 98% of the 596 recommendations.
Will the government publish a list of approved AI tools for schools?
The minister told the House of Lords on 26 October 2026 that a consultation on safety certification for higher-risk products would launch this autumn. Certification would lead to a published list of child-facing generative AI tools that meet safety criteria, including data protection and safeguarding.
Do schools need a data protection impact assessment (DPIA) for an AI tool?
Usually, yes. A DPIA is required where processing is likely to be high risk, and DfE guidance asks schools to complete one before procuring edtech that processes pupils’ personal data. Review it whenever the supplier updates the tool or adds new features.
Does your school have a list of approved AI tools, and how did you decide what went on it? Share your approach, or your sticking points, in the comments.
