Doctors are using AI software that does not meet minimum standards to record and transcribe patient meetings, according to a Sky News investigation.
NHS bosses have demanded GPs and hospitals stop using artificial intelligence software that could breach data protection rules and put patients at risk.
A warning sent out by NHS England this month came just weeks after the same body wrote to doctors about the benefits of using AI for notetaking – to allow them more time to concentrate on patients – using software known as Ambient Voice Technology, or “AVT”.
Health Secretary Wes Streeting will next week put AI at the heart of the reform plan to save the NHS in the 10-year plan for the health service in England.
But there is growing controversy around software that records, transcribes and summarises patient conversations using AI.
In April, NHS England wrote to doctors to sell the benefits of AVT and set out minimum national standards.
However, in a letter seen by Sky News, NHS bosses wrote to doctors to warn that unapproved software that breached minimum standards could harm patients.
More on Artificial Intelligence
Related Topics:
The 9 June letter, from the national chief clinical information officer of NHS England, said: “We are now aware of a number of AVT solutions which, despite being non-compliant … are still being widely used in clinical practice.
“Several AVT suppliers are approaching NHS organisations … many of these vendors have not complied with basic NHS governance standards.
“Proceeding with non-compliant solutions risks clinical safety, data protection breaches, financial exposure, and fragmentation of broader NHS digital strategy.”
Sky News has previously revealed the danger of AI “hallucinations”, where the technology makes up answers then lies about them, which could prove dangerous in a healthcare setting.
Please use Chrome browser for a more accessible video player
1:59
Is ChatGPT reliable despite its ‘hallucinations’?
NHS England sets minimum standards but does not tell NHS trusts and healthcare providers which software providers to use.
Sky News can now reveal there is growing pressure on NHS England and similar bodies to be more proactive.
Dr David Wrigley, deputy chair of the British Medical Association’s GP committee, said: “Undoubtedly, as a GP myself and my 35,000 colleagues, we’ve got responsibilities here – but in such a rapidly developing market when we haven’t got the technical knowledge to look into this.
“We need that help and support from those who can check that the products are safe, check they’re secure, that they’re suitable for use in the consulting room, and NHS England should do that and help and support us.”
Dr Wrigley continued: “We’re absolutely in favour of tech and in favour of taking that forward to help NHS patients, help my colleagues in their surgeries.
“But it’s got to be done in a safe and secure way because otherwise we could have a free for all – and then data could be lost, it could be leaking out, and that just isn’t acceptable.
“So we are not dinosaurs, we’re very pro-AI, but it has to be a safe, secure way.”
Image: The head of the NHS Confederation says the letter is ‘a really significant moment’
The spectre of dozens of little-known but ambitious AI companies lobbying hospitals and surgeries to get their listening products installed worries some healthcare professionals.
There are huge profits to be made in this technological arms race, but the question being asked is whether hundreds of different NHS organisations can really be expected to sift out the sharks.
Matthew Taylor, chief executive of the NHS Confederation, said the letter was “a really significant moment”.
He said it was right for the NHS to experiment, but that it needed to be clearer what technology does and does not work safely.
“My own view is that the government should help in terms of the procurement decisions that trusts make and should advise on which AI systems – as we do with other forms of technology that we use in medicine – which ones are safe,” Mr Taylor said.
“We’ll need [government] to do a bit more to guide the NHS in the best way to use this.”
When pressed whether in the short term that actually makes it sound like it could be quite dangerous, Mr Taylor replied: “What you’ve seen with ambient voice technology is that kind of ‘let a thousand flowers bloom’ approach has got its limits.”
Please use Chrome browser for a more accessible video player
0:45
Godfather of AI warns of its dangers
Earlier this year, the health secretary appeared to suggest unapproved technology was being used – but celebrated it as a sign doctors were enthusiastic for change.
Mr Streeting said: “I’ve heard anecdotally down the pub, genuinely down the pub, that some clinicians are getting ahead of the game and are already using ambient AI to kind of record notes and things, even where their practice or their trust haven’t yet caught up with them.
“Now, lots of issues there – not encouraging it – but it does tell me that contrary to this, ‘Oh, people don’t want to change, staff are very happy and they are really resistant to change’, it’s the opposite. People are crying out for this stuff.”
Image: GP Anil Mehta says the AI software helps cut paperwork and patients are ‘extremely reassured’
Doctors who use AI that complies with national standards already say there are big benefits.
Anil Mehta, a doctor in the health secretary’s Ilford constituency, told Sky News he backed his MP’s drive for more AI technology in healthcare.
“I spend 30% of my week doing paperwork,” he said. “So I think once I’ve explained all of those features of what we’re doing, patients are extremely reassured. And I haven’t faced anybody that’s not wanted to have me do this.
He added: “(I) think that consultation with your doctor is extremely confidential, so that’s not changed at all.
“That remains confidential – so whether it’s a vulnerable adult, a vulnerable child, teenager, young child with a parent, I think the concept of that confidentiality remains.”
An NHS spokesperson said: “Ambient Voice Technology has the potential to transform care and improve efficiency and in April, the NHS issued guidance to support its use in a safe and secure way.
“We are working with NHS organisations and suppliers to ensure that all Ambient Voice Technology products used across the health service continue to be compliant with NHS standards on clinical safety and data security.”
“It makes me sad. We left when our country had the troubles so we should have in this world… the humanity”.
We’re sitting in a cafe in Tamworth and Noor, 19, is explaining how it feels to know there are people in the town who don’t want him here.
Noor is from Afghanistan and came to the UK on a small boat.
The cafe is close to the asylum hotel where he’s staying.
Image: The group met in a cafe in Tamworth
He’s agreed to come along with four other men from the hotel to speak to locals about the concern in the town over the Staffordshire hotel being used to house asylum seekers.
There was a peaceful demonstration outside the hotel last month. But last year, a protest here turned ugly. Windows were smashed, petrol bombs thrown, and part of the hotel was set alight.
Among the locals in the cafe is Tom, 25, who reveals he was at both protests.
More on Asylum
Related Topics:
Image: Tom (left) has attended anti-migration protests in Tamworth
He says he was persuaded to go by friends and explains to the group why they decided to go.
“They were annoyed, angry, fuming that the government had let them [asylum seekers] live in a hotel,” he says.
Noor, who speaks the best English of the asylum seekers in the group, replies: “What did we do wrong?”
Image: Noor says he is upset by people who do not want him in the UK
“Your government accepts us as asylum seekers,” he continues.
Tom thinks. “I’m more annoyed with the government than you guys,” he tells them.
‘A place to get the golden ticket’
Noor explains to the group how he ended up in the UK. He left Afghanistan four years ago with his family but they were separated on the journey. He doesn’t know where they are.
Heather, a 29-year-old local accountant, speaks up.
Image: Heather says protests outside hotels makes asylum seekers fearful
“When people protest, I’m like, why don’t you protest near the government?” she asks. “Why don’t you take your issue to them rather than being outside the hotel?”
“Those asylum seekers aren’t going to change the policy at all,” she adds. “It’s just going to make them fearful.”
Each of the locals in the cafe has their own take on why some don’t like the asylum seekers living in their town.
“I think they feel like they’re living better than the British people, some of them, and it’s almost like they feel offended,” says Andrew, 47.
“Some people in the UK see how the asylum seekers are coming over to Britain because they see it as a place to get the golden ticket,” he adds.
Please use Chrome browser for a more accessible video player
10:54
UK’s unprecedented immigration figures
Heather agrees. She says the NHS is a draw and the UK also has “different border control regulations that might be seen as weaker than in some other countries”.
“You get to stay in a hotel,” she tells the asylum seekers. “You get the free health care. And so I think that’s why they’re a little bit annoyed.”
Noor replies: “One thing I should tell you is that when we cross the English Channel, it means we don’t care about our life. It’s very dangerous.”
Image: Noor and four other asylum seekers joined the meeting
Links to the UK
I’m keen to know why they chose to come to Britain. Noor tells the group it’s because he has a relative here and speaks the language.
Azim, 22, who is also from Afghanistan, says he came here because people in the UK “have respect to Islam”.
He also has a family member here.
Image: Azim says people in the UK are respectful of Islam
I ask them if they could have claimed asylum in France, but Noor says his “only hope was England”.
He says it’s “better for education” here. All the men agree it’s seen as the better place to come.
The conversation moves to the protests this summer which began in Epping, Essex, after an asylum seeker there was charged with sexually assaulting a schoolgirl – an offence he has now been convicted of.
Please use Chrome browser for a more accessible video player
56:38
In full: The Immigration Debate
Noor believes British people have a right to be angry about that. He tells the group he believes that asylum seekers who commit crimes “should get back to their country”.
“We also [do] not support them,” he says.
Over the course of the meeting, the mood becomes more relaxed. People with different views find some common ground.
Please use Chrome browser for a more accessible video player
3:30
Immigration Debate audience have their say
Noor tells the group that if things improve in Afghanistan he would like to go back there one day.
If not, he hopes he’ll stay in the UK and earn enough to repay in taxes the bill for the hotel he’s staying in now.
It has been a frank exchange. Some in this town will never want asylum seekers here and people like Noor and Azim know it.
But they were placed here by the Home Office and can only wait until their asylum claims are processed.
“Mum is teaching yoga and English to her cellmates in Iranian prison.”
It’s now over eight months since British couple Lindsay and Craig Foreman were detained in Iran.
Last week, during a long-awaited visit from British ambassador Hugo Shorter, it was confirmed that the pair continue to endure tough conditions with no indication of how – or when – the legal process will proceed.
“They’re both coping, making the best of a bad situation. They’re in conditions you can’t even imagine.”
Image: Lindsay Foreman with son Joe Bennett. Pic: Family handout
Speaking to Sky News, their son Joe Bennett explained how the couple have been crammed into cells with more than 50 other prisoners, while suffering constant back pain caused by metal bunk beds.
“The beds are stacked three high. It’s unsanitary. It’s hot. There are often power outages and they’re in 50-degree heat.”
Image: Craig and Lindsay Foreman. Pic: Family handout
Lindsay and Craig, both 52, were arrested in early January in Iran, as they crossed the country on motorbikes as part of an around-the-world adventure. The couple had left Spain just a few weeks earlier and were aiming to drive all the way to Australia.
They were charged with espionage and have been transferred to various prisons around Iran, with little information provided to British diplomatic staff about their whereabouts.
Joe and the rest of the family have only managed to speak to their parents once on the phone. “In a brief conversation that I had with my mum, we managed to share a laugh and a lot of tears as well. But it’s a test of time, how long they can keep this up for.”
Image: Pic: Family handout
The UK ambassador’s meeting with Craig was the first in over four months, and despite suffering from untreated dental pain, he quipped about becoming a “reluctant Arsenal supporter” while watching football on television with other prisoners.
The couple were previously held together in a facility in the Iranian city of Kerman but have been moved to separate prisons in the capital, Tehran. Family members are calling on the Iranians to move Lindsay into the same facility where Craig is being detained.
Image: Pic: Family handout
Their son acknowledged in his interview with Sky News that he was frustrated with his parents when they were arrested in January. Family members had urged them not to travel through the country.
“I had that natural reaction that some of the public do – why did they go? It’s idiotic, you’re going against the advice, and it serves them right. That’s fair enough when you don’t know them [but] just picture your parents having a bit of a sense of adventure… it’s a different story.”
The Foreign, Commonwealth and Development Office says it is “deeply concerned” about the couple, adding, “we continue to raise this case directly with the Iranian authorities”.
Members of the Foreman family are urging the British government and the new foreign secretary, Yvette Cooper, to take direct action to improve harsh prison conditions and urgently organise for Lindsay and Craig’s release.
“I need them home, you know, and I need them home as soon as possible. We need them, the family miss them dearly – so we’re going to do everything we can to make that possible.”
The security services expressed concern about the appointment of Lord Mandelson as ambassador to Washington, but No 10 went ahead anyway, Sky News understands.
Downing Street today defended the extensive vetting process which senior civil servants go through in order to get jobs, raising questions about whether or not they missed something or No 10 ignored their advice.
Sky News has been told by two sources that the security services did flag concerns as part of the process.
No 10 did not judge these concerns as enough to stop the ambassadorial appointment.
It is not known whether all of the detail was shared with Prime Minister Sir Keir Starmer personally.
Sky News has been told some members of the security services are unhappy with what has taken place in Downing Street.
More on Keir Starmer
Related Topics:
Lord Mandelson is close to Sir Keir’s chief of staff, Morgan McSweeney, who is known to have been keen on the appointment – and the pair spoke regularly.
No 10 says the security vetting process is all done at a departmental level with no No 10 involvement.
Shadow foreign secretary Priti Patel described the revelations as “extraordinary”.
“For Keir Starmer, and his Chief of Staff Morgan McSweeney, to have appointed Lord Mandelson despite concerns being raised by the security services shows a blatant disregard of all national security considerations and their determination to promote their Labour Party friends,” she said in a statement.
“Starmer leads a crisis riddled government consumed by a chaos of his own making, because he puts his Party before the needs of our country.
“The country deserves the honest truth this spineless prime minister refuses to give them.”
Image: Priti Patel described the revelations as ‘extraordinary’.
The prime minister, who selected Lord Mandelson for the role, made the decision after new emails revealed the Labour peer sent messages of support to Epstein even as he faced jail for sex offences in 2008.
In one particular message, Lord Mandelson had suggested that Epstein’s first conviction was wrongful and should be challenged.
The Foreign Office said the emails showed “the depth and extent of Peter Mandelson’s relationship with Jeffrey Epstein is materially different from that known at the time of his appointment”.
The decision to sack the diplomat was made by the prime minister and Foreign Secretary Yvette Cooper on Thursday morning, Sky News understands.
This was after Sir Keir had reviewed all the new available information last night.
Please use Chrome browser for a more accessible video player
2:59
Harriet Harman, Ruth Davidson, and Beth Rigby react to the news of Lord Mandelson’s sacking.
It comes after a string of allegations around the diplomat’s relationship with Epstein, which emerged in the media this week, including a 2003 birthday message in which he called the sex offender his “best pal”.
Further allegations were then published in The Telegraph on Wednesday morning, suggesting that Lord Mandelson had emailed Epstein to set up business meetings following the latter’s conviction for child sex offences in 2008.
Additional emails were then published detailing how the diplomat wrote to Epstein the day before he went to prison in June 2008 to serve time for soliciting sex from a minor. Lord Mandelson said: “I think the world of you.”