Doctors are using AI software that does not meet minimum standards to record and transcribe patient meetings, according to a Sky News investigation.
NHS bosses have demanded GPs and hospitals stop using artificial intelligence software that could breach data protection rules and put patients at risk.
A warning sent out by NHS England this month came just weeks after the same body wrote to doctors about the benefits of using AI for notetaking – to allow them more time to concentrate on patients – using software known as Ambient Voice Technology, or “AVT”.
Health Secretary Wes Streeting will next week put AI at the heart of the reform plan to save the NHS in the 10-year plan for the health service in England.
But there is growing controversy around software that records, transcribes and summarises patient conversations using AI.
In April, NHS England wrote to doctors to sell the benefits of AVT and set out minimum national standards.
However, in a letter seen by Sky News, NHS bosses wrote to doctors to warn that unapproved software that breached minimum standards could harm patients.
More on Artificial Intelligence
Related Topics:
The 9 June letter, from the national chief clinical information officer of NHS England, said: “We are now aware of a number of AVT solutions which, despite being non-compliant … are still being widely used in clinical practice.
“Several AVT suppliers are approaching NHS organisations … many of these vendors have not complied with basic NHS governance standards.
“Proceeding with non-compliant solutions risks clinical safety, data protection breaches, financial exposure, and fragmentation of broader NHS digital strategy.”
Sky News has previously revealed the danger of AI “hallucinations”, where the technology makes up answers then lies about them, which could prove dangerous in a healthcare setting.
Please use Chrome browser for a more accessible video player
1:59
Is ChatGPT reliable despite its ‘hallucinations’?
NHS England sets minimum standards but does not tell NHS trusts and healthcare providers which software providers to use.
Sky News can now reveal there is growing pressure on NHS England and similar bodies to be more proactive.
Dr David Wrigley, deputy chair of the British Medical Association’s GP committee, said: “Undoubtedly, as a GP myself and my 35,000 colleagues, we’ve got responsibilities here – but in such a rapidly developing market when we haven’t got the technical knowledge to look into this.
“We need that help and support from those who can check that the products are safe, check they’re secure, that they’re suitable for use in the consulting room, and NHS England should do that and help and support us.”
Dr Wrigley continued: “We’re absolutely in favour of tech and in favour of taking that forward to help NHS patients, help my colleagues in their surgeries.
“But it’s got to be done in a safe and secure way because otherwise we could have a free for all – and then data could be lost, it could be leaking out, and that just isn’t acceptable.
“So we are not dinosaurs, we’re very pro-AI, but it has to be a safe, secure way.”
Image: The head of the NHS Confederation says the letter is ‘a really significant moment’
The spectre of dozens of little-known but ambitious AI companies lobbying hospitals and surgeries to get their listening products installed worries some healthcare professionals.
There are huge profits to be made in this technological arms race, but the question being asked is whether hundreds of different NHS organisations can really be expected to sift out the sharks.
Matthew Taylor, chief executive of the NHS Confederation, said the letter was “a really significant moment”.
He said it was right for the NHS to experiment, but that it needed to be clearer what technology does and does not work safely.
“My own view is that the government should help in terms of the procurement decisions that trusts make and should advise on which AI systems – as we do with other forms of technology that we use in medicine – which ones are safe,” Mr Taylor said.
“We’ll need [government] to do a bit more to guide the NHS in the best way to use this.”
When pressed whether in the short term that actually makes it sound like it could be quite dangerous, Mr Taylor replied: “What you’ve seen with ambient voice technology is that kind of ‘let a thousand flowers bloom’ approach has got its limits.”
Please use Chrome browser for a more accessible video player
0:45
Godfather of AI warns of its dangers
Earlier this year, the health secretary appeared to suggest unapproved technology was being used – but celebrated it as a sign doctors were enthusiastic for change.
Mr Streeting said: “I’ve heard anecdotally down the pub, genuinely down the pub, that some clinicians are getting ahead of the game and are already using ambient AI to kind of record notes and things, even where their practice or their trust haven’t yet caught up with them.
“Now, lots of issues there – not encouraging it – but it does tell me that contrary to this, ‘Oh, people don’t want to change, staff are very happy and they are really resistant to change’, it’s the opposite. People are crying out for this stuff.”
Image: GP Anil Mehta says the AI software helps cut paperwork and patients are ‘extremely reassured’
Doctors who use AI that complies with national standards already say there are big benefits.
Anil Mehta, a doctor in the health secretary’s Ilford constituency, told Sky News he backed his MP’s drive for more AI technology in healthcare.
“I spend 30% of my week doing paperwork,” he said. “So I think once I’ve explained all of those features of what we’re doing, patients are extremely reassured. And I haven’t faced anybody that’s not wanted to have me do this.
He added: “(I) think that consultation with your doctor is extremely confidential, so that’s not changed at all.
“That remains confidential – so whether it’s a vulnerable adult, a vulnerable child, teenager, young child with a parent, I think the concept of that confidentiality remains.”
An NHS spokesperson said: “Ambient Voice Technology has the potential to transform care and improve efficiency and in April, the NHS issued guidance to support its use in a safe and secure way.
“We are working with NHS organisations and suppliers to ensure that all Ambient Voice Technology products used across the health service continue to be compliant with NHS standards on clinical safety and data security.”
The Archbishop of York has told Sky News the UK should resist Reform’s “kneejerk” plan for the mass deportation of migrants, telling Nigel Farage he is not offering any “long-term solution”.
Stephen Cottrell said in an interview with Trevor Phillips he has “every sympathy” with people who are concerned about asylum seekers coming to the country illegally.
But he criticised the plan announced by Reform on Tuesday to deport 600,000 people, which would be enabled by striking deals with the Taliban and Iran, saying it will not “solve the problem”.
Mr Cottrell is currently acting head of the Church of England while a new Archbishop of Canterbury is chosen.
Image: Pic: Jacob King/PA Wire
Image: The Archbishop of York, Stephen Cottrell in 2020.
File pic: PA
Phillips asked him: “What’s your response to the people who are saying the policy should be ‘you land here, unlawfully, you get locked up and you get deported straight away. No ifs, no buts’?”
Mr Cottrell said he would tell them “you haven’t solved the problem”, adding: “You’ve just put it somewhere else and you’ve done nothing to address the issue of what brings people to this country.
More on Migrant Crisis
Related Topics:
“And so if you think that’s the answer, you will discover in due course that all you have done is made the problem worse.
“Don’t misunderstand me, I have every sympathy with those who find this difficult, every sympathy – as I do with those living in poverty.
“But… we should actively resist the kind of isolationist, short term kneejerk ‘send them home’.”
Please use Chrome browser for a more accessible video player
2:04
What do public make of Reform’s plans?
Image: Nigel Farage at the launch of Reform UK’s plan to deport asylum seekers. Pic: PA
Asked if that was his message to the Reform leader, he said: “Well, it is. I mean, Mr Farage is saying the things he’s saying, but he is not offering any long-term solution to the big issues which are convulsing our world, which lead to this. And, I see no other way.”
Mr Farage, the MP for Clacton, was asked at a news conference this week what he would say if Christian leaders opposed his plan.
“Whoever the Christian leaders are at any given point in time, I think over the last decades, quite a few of them have been rather out of touch, perhaps with their own flock,” he said.
“We believe that what we’re offering is right and proper, and we believe for a political party that was founded around the slogan of family, community, country that we are doing right by all of those things, with these plans we put forward today.”
Sky News has approached Mr Farage for comment.
Farage won’t be greeting this as good news of the gospel – nor will govt ministers
When Tony Blair’s spin doctor Alastair Campbell told journalists that “We don’t do God”, many took it as a statement of ideology.
In fact it was the caution of a canny operator who knows that the most dangerous opponent in politics is a religious leader licensed to challenge your very morality.
Stephen Cottrell, the Archbishop of York, currently the effective head of the worldwide Anglican communion, could not have been clearer in his denunciation of what he calls the Reform party’s “isolationist, short term, kneejerk ‘send them home'” approach to asylum and immigration.
I sense that having ruled himself out of the race for next Archbishop of Canterbury, Reverend Cottrell feels free to preach a liberal doctrine.
Unusually, in our interview he pinpoints a political leader as, in effect, failing to demonstrate Christian charity.
Nigel Farage, who describes himself as a practising Christian, won’t be greeting this as the good news of the gospel.
But government ministers will also be feeling nervous.
Battered for allowing record numbers of cross- Channel migrants, and facing legal battles on asylum hotels that may go all the way to the Supreme Court, Labour has tried to head off the Reform challenge with tougher language on border control.
The last thing the prime minister needs right now is to make an enemy of the Almighty – or at least of his representatives on Earth.
In the hospital which was supposed to help her, the last moments of 14-year-old Ruth Szymankiewicz’s life were recorded on CCTV.
The teenager, who should have been under constant supervision on the children’s psychiatric ward, was left alone by her support worker at Taplow Manor Hospital in Berkshire. Fifteen minutes later, she had fatally self-harmed.
The worker assigned to her had only one-and-a-half days’ training and had faked his identity using false documents.
Image: CCTV footage showed Ruth Szymankiewicz left alone
Earlier this month, a jury at the inquest into Ruth’s death concluded she was unlawfully killed. Despite this, there have been no criminal prosecutions.
Speaking to Sky News and The Independent in their first TV interview, Ruth’s father, Mark, said: “She went somewhere that was supposed to be helping her, and it made her worse. The isolation and lack of access to her family had a massively negative impact.”
Image: Ruth Szymankiewicz’s parents spoke to Sky News about her death
Her mother, Kate, added: “The children get lost. Ruth got lost. She was lost in the middle of all this chaos.”
Ruth’s parents have said the hospital’s strict visiting regime meant they were unable to see their daughter as often as they had wanted. Her father never saw her room.
“Her access to us was denied,” Mark said. “We were willing and able to give that support. It completely derailed her.”
The family believe that if Ruth had been allowed regular contact with them, she would still be alive.
Image: Ruth’s parents Kate and Mark
History of failings
The failures at Taplow Manor were well-documented. Investigations by Sky News and The Independent uncovered disturbing evidence about the treatment of young people.
There were numerous critical reports, including three from the Care Quality Commission (CQC) regulator in the year leading up to Ruth’s death, each one highlighting unsafe practices.
Despite this, the NHS continued to send vulnerable children there.
Image: Ruth Szymankiewicz died in February 2022. Pic: Family handout via PA
At Ruth’s inquest, an NHS clinician in charge of commissioning her care admitted they knew about the issues at the hospital.
The inquest heard there were no other psychiatric intensive care units close enough to send her to.
Steph Smith was a former patient at Taplow Manor – then known as The Huntercombe Hospital Maidenhead – in 2017, who later went on to work at the unit as a healthcare assistant between September 2021 and February 2022.
She described the ward as “chaotic, scary and intense”.
Image: Steph Smith was a former patient at Taplow Manor
“There was a huge culture of covering things up,” she said.
“Observations weren’t done. People just signed the paperwork at the end of the shift. On paper, it looked fine, but in reality, children were left at risk.
“It was only a matter of time. It breaks my heart that it took a 14-year-old girl dying for the hospital to close. It should have been shut years ago.”
Staff warned managers
Nurse Ellesha Branaghan worked as a clinical team leader on Ruth’s ward. She and colleagues warned managers about shortages on the rota.
“We would often tell them the staffing levels weren’t safe but we just kept getting told these are the numbers,” she said.
She said a lack of staffing often meant patients could not go on leave, or even visit the hospital gardens.
Please use Chrome browser for a more accessible video player
1:56
Teenager’s death in psychiatric care ruled unlawful killing
There were occasions, she said, when patient observation levels were decreased because there were not enough staff on shift.
“Sometimes we would have four or five incidents at the same time,” she added. “We didn’t have the staff to respond, so that becomes unsafe.”
The staffing levels became “so severe” that even patients wrote to senior managers to express concerns.
An NHS England spokesperson said: “All providers must operate to the highest standards and the NHS worked with young people and families to move patients from Taplow Manor to other clinically appropriate services.”
The ‘loophole’
Taplow Manor was finally closed in 2023. The CQC had visited the hospital just 11 days before Ruth’s death.
High-level feedback was given following this, highlighting concerns with the environment, care plans not being followed and staffing levels.
After further inspections in March 2022, the watchdog issued a warning notice about failings in patient observations.
But once a warning notice is issued, that particular issue cannot be the subject of a criminal prosecution – something Ruth’s parents describe as a “loophole”.
Image: Pic: Family handout
Mark said the CQC opened an investigation into his daughter’s death and looked at a “number of different routes to potentially prosecute the Active Care Group”.
Active Care Group acquired the Huntercombe Group, which ran Taplow Manor, in December 2021.
Mark said the regulator was not “allowed or able to prosecute, even though the same failing happened with catastrophic consequences”.
‘No justice for Ruth’
The CQC said it did carry out a full criminal investigation but the evidence “did not meet the threshold”.
It added that there was no suggestion the outcome would have been different if there had been no warning notice.
For Ruth’s parents, this is unacceptable.
“Why did our daughter have to die before anyone paid attention?” Kate asked. “They knew all this before she died.”
The inquest ruling of unlawful killing has brought no comfort to Ruth’s family.
“There can be no justice for Ruth,” her father said. “She’s dead, she’s gone. We’re left with the fallout.”
Please use Chrome browser for a more accessible video player
3:02
‘Gaping hole in our family will never be filled’
A CQC spokesperson said the regulator began a criminal investigation in November 2022 but “found that there was not sufficient evidence to charge”.
“We know that this was disappointing for Ruth’s family, and we met with them to explain how we came to this decision,” the spokesperson added.
“We have a range of enforcement powers available to us and criminal action is only an option when the evidence demonstrates without any doubt that there have been organisational failings that can be proven to the required legal threshold.”
Following Ruth’s death, the CQC continued to visit the unit. A report published just six months later raised more concerns over observations, saying “there had been 22 incidents involving poor practice with observing young people”.
It went on: “The incidents ranged from staff falling asleep, not following young people when they left the room and completing other tasks whilst they were meant to be observing someone.”
It was rated inadequate in December 2022, before its closure.
Ex-patients voice concerns
Ruth’s case echoes concerns raised by other former patients.
Amber Rehman, who was admitted to Huntercombe Hospital in 2019, said: “Ruth’s story – I’ve heard so many similar stories. It could happen to anyone. It could still be happening out there.”
Amber’s mother, Nikki, said: “It was absolutely preventable. No one made changes.”
Image: Amber Rehman
Amber’s family made a formal complaint about the care she received.
An independent review was commissioned by the hospital, which found issues with observations – including missing observation records – and an over-reliance on physical intervention and medication.
The review – which was published exactly a year before Ruth harmed herself – recommended an audit of the observation records, and said the way the hospital communicated and engaged with families should be looked at.
Image: Pic: Family handout
Sky News has seen two other independent reports commissioned by the hospital before Ruth died, raising similar concerns – including engagement and communication with the patient’s family.
Fifty former patients came forward to our investigation in 2022 to share their experience of this hospital and a number of other units run by the same provider.
Many have told us how they still struggle with trauma from what they faced while under its care – some have formal diagnosis of PTSD due to it.
Sky News understands that 58 former patients are now taking legal action against around 30 psychiatrists who worked at various Huntercombe hospitals over two decades.
A statement from Active Care Group said: “We extend our heartfelt condolences to Ruth’s family, friends, and all those affected by her passing. We deeply regret the tragic event that occurred, and we are truly sorry for the distress this has caused
“We directed significant investment in staff training, recruitment, and the hospital estate, spending more than £3m on the physical environment alone over an 18-month period.
“Despite these efforts, by early 2023, it became clear that achieving the high standards of care that reflect our core values would not be possible within an acceptable timescale.
“In recent years, we have made significant improvements to the quality and safety in all of our services.
“We are regrettably unable to comment on historical allegations relating to care provided under previous ownership or management.”
Elli Investments Group, owners of The Huntercombe Group until 2021, previously told us: “We regret that these hospitals and specialist care services, which were owned and independently managed by The Huntercombe Group, failed to meet the expected standards for high-quality care.”
Image: Pictures of Ruth at the family home
‘Our lives are darker without her’
Ruth’s parents, who are both doctors working in the NHS, are calling on the government to close what they see as the “legal loophole” in the powers the CQC has to prosecute.
They also want to strengthen safeguards for children in mental health units by ensuring parents have visitation rights to their children.
“Ruth died under the care of the state,” her mother, Kate, said.
“We very much hope that secretaries of state for health and for mental health are listening to Ruth’s story, and that they can use this opportunity, particularly to make sure that children have unrestricted access to their families.”
A Department of Health and Social Care spokesperson said: “Our deepest sympathies are with Ruth’s family and friends. This is a shocking case and it is clear care at Huntercombe Hospital fell far below the standards we expect.
“Where appropriate the CQC can bring prosecutions where a provider has failed to comply with a warning notice, and we are clear that those that harm patients through negligence or mismanagement should face the consequences.
“We are investing £75m this year to reduce inappropriate out of area placements, increasing family involvement in patient care through the Mental Health Bill, and driving up standards through the 10 Year Plan so everyone receives the level of care they deserve.”
Ruth’s parents are both struggling with the lack of accountability over their daughter’s death, especially the decision by the CQC not to prosecute.
“We don’t have faith the system will make sure changes happen,” Mark said.
“Governance has been completely ineffectual. Until there is real accountability, nothing will stop this happening again.”
Kate added: “Our lives are darker without her. Ruth was unique and wonderful. She kept us wholehearted in everything we did. Now she’s gone.”
Anyone feeling emotionally distressed or suicidal can call Samaritans for help on 116 123 or email jo@samaritans.org in the UK. In the US, call the Samaritans branch in your area or 1 (800) 273-TALK.
The Appeal Court judge who ruled in favour of Home Secretary Yvette Cooper in the Epping migrant hotel case is a long-standing Labour supporter.
Lord Justice David Bean, 71, is a former treasurer of the Society of Labour Lawyers and chaired the left-leaning Fabian Society, which is affiliated to the Labour Party, in 1989 and 1990.
He was also – with Sir Tony Blair’s barrister wife Cherie – a founder member in 2000 of the left-wing Matrix Chambers, whose members include the current attorney general, Lord Hermer.
Image: The Bell Hotel in Epping, Essex, is at the centre of a legal battle. Pic: PA
On its website, the Society of Labour Lawyers describes itself as “a thinktank and affiliated socialist society which provides legal and policy advice to the Labour Party”.
Founded in 1948 by a future Labour lord chancellor Gerald Gardiner, it declares: “Our objectives are to contribute legal expertise to the Labour Party and uphold the principles of justice, liberty, equality, and the rule of law in the UK and around the world.
“We advise Labour MPs and the House of Lords; develop and scrutinise policy and legislation; contribute to debate within the Labour movement by hosting events and discussions; and mentor future members of the legal profession.
More from Politics
“We are open to Labour Party members who are also practising or retired lawyers, law students or graduates, academics, and members of the judiciary.”
The Fabian Society describes itself as “a democratically governed socialist society, a Labour affiliate and one of the party’s original founders”.
Please use Chrome browser for a more accessible video player
5:00
Inside the asylum hotel protests
Sir Stephen Eyre, the High Court judge who ruled in favour of Epping Forest Council earlier this month, was a Conservative parliamentary candidate four times.
His most high-profile bid to become an MP came in the 2004 Birmingham Hodge Hill by-election, won by current Labour MP and former minister Liam Byrne.
Image: Sir Stephen Eyre. Pic: Judicial Appointments Commission/Ministry of Justice
Appointed a High Court judge by then Lord Chancellor Dominic Raab in 2021, Sir Stephen was a Tory candidate while working as a barrister.
His first attempt came in 1987, when he stood in Hodge Hill in that year’s general election, coming second behind Labour’s Terry Davis.
Then in 1992, the year of Sir John Major’s 21-seat election victory, he stood for the Northern Ireland Conservatives in the unionist stronghold of Strangford.
Mr Eyre, as he then was, came fourth behind the official Unionists’ John Taylor, with current Democratic Unionist MP for Antrim East Sammy Wilson in second place.
In 2001, he stood in Stourbridge, where he again came second, this time to Labour’s Debra Shipley, when he cut her majority from nearly 6,000 to under 4,000.
And in the 2004 by-election, he came a distant third as Mr Byrne scraped in by just 460 votes ahead of the Liberal Democrats, who benefited from an Iraq war backlash.