Doctors are using AI software that does not meet minimum standards to record and transcribe patient meetings, according to a Sky News investigation.
NHS bosses have demanded GPs and hospitals stop using artificial intelligence software that could breach data protection rules and put patients at risk.
A warning sent out by NHS England this month came just weeks after the same body wrote to doctors about the benefits of using AI for notetaking – to allow them more time to concentrate on patients – using software known as Ambient Voice Technology, or “AVT”.
Health Secretary Wes Streeting will next week put AI at the heart of the reform plan to save the NHS in the 10-year plan for the health service in England.
But there is growing controversy around software that records, transcribes and summarises patient conversations using AI.
In April, NHS England wrote to doctors to sell the benefits of AVT and set out minimum national standards.
However, in a letter seen by Sky News, NHS bosses wrote to doctors to warn that unapproved software that breached minimum standards could harm patients.
More on Artificial Intelligence
Related Topics:
The 9 June letter, from the national chief clinical information officer of NHS England, said: “We are now aware of a number of AVT solutions which, despite being non-compliant … are still being widely used in clinical practice.
“Several AVT suppliers are approaching NHS organisations … many of these vendors have not complied with basic NHS governance standards.
“Proceeding with non-compliant solutions risks clinical safety, data protection breaches, financial exposure, and fragmentation of broader NHS digital strategy.”
Sky News has previously revealed the danger of AI “hallucinations”, where the technology makes up answers then lies about them, which could prove dangerous in a healthcare setting.
Please use Chrome browser for a more accessible video player
1:59
Is ChatGPT reliable despite its ‘hallucinations’?
NHS England sets minimum standards but does not tell NHS trusts and healthcare providers which software providers to use.
Sky News can now reveal there is growing pressure on NHS England and similar bodies to be more proactive.
Dr David Wrigley, deputy chair of the British Medical Association’s GP committee, said: “Undoubtedly, as a GP myself and my 35,000 colleagues, we’ve got responsibilities here – but in such a rapidly developing market when we haven’t got the technical knowledge to look into this.
“We need that help and support from those who can check that the products are safe, check they’re secure, that they’re suitable for use in the consulting room, and NHS England should do that and help and support us.”
Dr Wrigley continued: “We’re absolutely in favour of tech and in favour of taking that forward to help NHS patients, help my colleagues in their surgeries.
“But it’s got to be done in a safe and secure way because otherwise we could have a free for all – and then data could be lost, it could be leaking out, and that just isn’t acceptable.
“So we are not dinosaurs, we’re very pro-AI, but it has to be a safe, secure way.”
Image: The head of the NHS Confederation says the letter is ‘a really significant moment’
The spectre of dozens of little-known but ambitious AI companies lobbying hospitals and surgeries to get their listening products installed worries some healthcare professionals.
There are huge profits to be made in this technological arms race, but the question being asked is whether hundreds of different NHS organisations can really be expected to sift out the sharks.
Matthew Taylor, chief executive of the NHS Confederation, said the letter was “a really significant moment”.
He said it was right for the NHS to experiment, but that it needed to be clearer what technology does and does not work safely.
“My own view is that the government should help in terms of the procurement decisions that trusts make and should advise on which AI systems – as we do with other forms of technology that we use in medicine – which ones are safe,” Mr Taylor said.
“We’ll need [government] to do a bit more to guide the NHS in the best way to use this.”
When pressed whether in the short term that actually makes it sound like it could be quite dangerous, Mr Taylor replied: “What you’ve seen with ambient voice technology is that kind of ‘let a thousand flowers bloom’ approach has got its limits.”
Please use Chrome browser for a more accessible video player
0:45
Godfather of AI warns of its dangers
Earlier this year, the health secretary appeared to suggest unapproved technology was being used – but celebrated it as a sign doctors were enthusiastic for change.
Mr Streeting said: “I’ve heard anecdotally down the pub, genuinely down the pub, that some clinicians are getting ahead of the game and are already using ambient AI to kind of record notes and things, even where their practice or their trust haven’t yet caught up with them.
“Now, lots of issues there – not encouraging it – but it does tell me that contrary to this, ‘Oh, people don’t want to change, staff are very happy and they are really resistant to change’, it’s the opposite. People are crying out for this stuff.”
Image: GP Anil Mehta says the AI software helps cut paperwork and patients are ‘extremely reassured’
Doctors who use AI that complies with national standards already say there are big benefits.
Anil Mehta, a doctor in the health secretary’s Ilford constituency, told Sky News he backed his MP’s drive for more AI technology in healthcare.
“I spend 30% of my week doing paperwork,” he said. “So I think once I’ve explained all of those features of what we’re doing, patients are extremely reassured. And I haven’t faced anybody that’s not wanted to have me do this.
He added: “(I) think that consultation with your doctor is extremely confidential, so that’s not changed at all.
“That remains confidential – so whether it’s a vulnerable adult, a vulnerable child, teenager, young child with a parent, I think the concept of that confidentiality remains.”
An NHS spokesperson said: “Ambient Voice Technology has the potential to transform care and improve efficiency and in April, the NHS issued guidance to support its use in a safe and secure way.
“We are working with NHS organisations and suppliers to ensure that all Ambient Voice Technology products used across the health service continue to be compliant with NHS standards on clinical safety and data security.”
A care worker who reported the alleged abuse of an elderly care home resident, which triggered a criminal investigation, is facing destitution and potential removal from Britain after speaking up.
“Meera”, whose name we have changed to protect her identity, said she witnessed an elderly male resident being punched several times in the back by a carer at the home where she worked.
Sky News is unable to name the care home for legal reasons because of the ongoing police investigation.
“I was [a] whistleblower there,” said Meera, who came to the UK from India last year to work at the home.
“Instead of addressing things, they fired me… I told them everything and they made me feel like I am criminal. I am not criminal, I am saving lives,” she added.
Image: ‘Meera’ spoke up about abuse she said she witnessed in the care home where she worked
Like thousands of foreign care workers, Meera’s employer sponsored her visa. Unless she can find another sponsor, she now faces the prospect of removal from the country.
“I am in trouble right now and no one is trying to help me,” she said.
More on Migrant Crisis
Related Topics:
Meera said she reported the alleged abuse to her bosses, but was called to a meeting with a manager and told to “change your statement, otherwise we will dismiss you”.
She refused. The following month, she was sacked.
The care home claimed she failed to perform to the required standard in the job.
She went to the police to report the alleged abuse and since then, a number of people from the care home have been arrested. They remain under investigation.
‘Migrants recruited because many are too afraid to speak out’
The home has capacity for over 60 residents. It is unclear if the care home residents or their relatives know about the police investigation or claim of physical abuse.
Since the arrests, the regulator, the Care Quality Commission (CQC), carried out an investigation at the home triggered by the concerns – but the home retained its ‘good’ rating.
Meera has had no reassurance from the authorities that she will be allowed to remain in Britain.
In order to stay, she’ll need to find another care home to sponsor her which she believes will be impossible without references from her previous employer.
She warned families: “I just want to know people in care homes like these… your person, your father, your parents, is not safe.”
She claimed some care homes have preferred to recruit migrants because many are too afraid to speak out.
“You hire local staff, they know the legal rights,” she said. “They can complain, they can work anywhere… they can raise [their] voice,” she said.
Image: Sky’s Becky Johnson spoke to ‘Meera’
Sky News has reported widespread exploitation of care visas and migrant care workers.
Currently migrants make up around a third of the adult social care workforce, with the majority here on visas that are sponsored by their employers.
As part of measures announced in April in the government’s immigration white paper, the care visa route will be closed, meaning care homes will no longer be able to recruit abroad.
‘Whole system is based on power imbalance’
But the chief executive of the Work Rights Centre, a charity that helps migrants with employment issues, is warning that little will change for the tens of thousands of foreign care workers already here.
“The whole system is based on power imbalance and the government announcement doesn’t change that,” Dr Dora-Olivia Vicol told Sky News.
She linked the conditions for workers to poor care for residents.
Image: Work Rights Centre CEO Dr Dora-Olivia Vicol
“I think the power that employers have over migrant workers’ visas really makes a terrible contribution to the quality of care,” she said.
Imran agrees. He came to the UK from Bangladesh, sponsored by a care company unrelated to the one Meera worked for. He says he frequently had to work 14-hour shifts with no break because there weren’t enough staff. He too believes vulnerable people are being put at risk by the working conditions of their carers.
Migrant workers ‘threatened’ over visas
“For four clients, there is [a] minimum requirement for two or three staff. I was doing [it] alone,” he said, in broken English.
“When I try to speak, they just directly threaten me about my visa,” he said.
“I knew two or three of my colleagues, they are facing the same issue like me. But they’re still afraid to speak up because of the visa.”
A government spokesperson called what happened to Imran and Meera “shocking”.
“No one should go to work in fear of their employer, and all employees have a right to speak up if they witness poor practice and care.”
James Bullion, from the CQC, told Sky News it acts on intelligence passed to it to ensure people stay safe in care settings.
Donald Trump may be denied the honour of addressing parliament on his state visit to the UK later this year, with no formal request yet submitted for him to be given that privilege.
Sky News has been told the Speaker of the House of Commons, Sir Lindsay Hoyle, hasn’t so far received a request to invite the US president to speak in parliament when he is expected to visit in September.
It was confirmed to MPs who have raised concerns about the US president being allowed to address both houses.
Kate Osborne, Labour MP for Jarrow and Gateshead East, wrote to the speaker in April asking him to stop Mr Trump from addressing parliament, and tabled an early-day motion outlining her concerns.
“I was happy to see Macron here but feel very differently about Trump,” she said.
“Trump has made some very uncomfortable and worrying comments around the UK government, democracy, the Middle East, particularly around equalities and, of course, Ukraine.
“So, I think there are many reasons why, when we’re looking at a state visit, we should be looking at why they’re being afforded that privilege. Because, of course, it is a privilege for somebody to come and address both of the houses.”
But the timing of the visit may mean that any diplomatic sensitivities, or perceptions of a snub, could be avoided.
Image: France’s President Emmanuel Macron addressed parliament during his state visit this month
Lord Ricketts, a former UK ambassador to France, pointed out that parliament isn’t sitting for much of September, and that could help resolve the issue.
In 2017, he wrote a public letter questioning the decision to give Donald Trump his first state visit, saying it put Queen Elizabeth II in a “very difficult position”.
Parliament rises from 16 September until 13 October due to party conferences.
The dates for the state visit haven’t yet been confirmed by Buckingham Palace or the government.
However, they have not denied that it will take place in September, after Mr Trump appeared to confirm they were planning to hold the state visit that month. The palace confirmed this week that the formal planning for his arrival had begun.
With the King likely to still be in Scotland in early September for events such as the Braemar Gathering, and the anniversary of his accession and the death of Queen Elizabeth on the 8th September, it may be expected that the visit would take place sometime from mid to the end of September, also taking into consideration the dates of the Labour Party conference starting on the 28th September and possibly the Lib Dem’s conference from the 20th-23rd.
Image: Mr Trump has said he believes the trip to the UK will take place in September. Pic: Reuters
When asked about parliamentary recess potentially solving the issue, Ms Osborne said: “It may be a way of dealing with it in a very diplomatic way… I don’t know how much control we have over Trump’s diary.
“But if we can manoeuvre it in a way that means that the House isn’t sitting, then that seems like a good solution, maybe not perfect, because I’d actually like him to know that he’s not welcome.”
A message from the speaker’s office, seen by Sky News, says: “Formal addresses to both Houses of Parliament are not automatically included in the itinerary of such a state visit.
“Whether a foreign head of state addresses parliament, during a state visit or otherwise, is part of the planning decisions.”
Image: Mr Trump made his first state visit to the UK in June 2019 during his first presidency. File pic: Reuters
It’s understood that if the government agrees to a joint address to parliament, the Lord Chamberlain’s office writes to the two speakers, on behalf of the King, to ask them to host this.
It will be Mr Trump’s second state visit.
During his first, in 2019, he didn’t address parliament, despite the fact that his predecessor, Barack Obama, was asked to do so.
It was unclear if this was due to the fact John Bercow, the speaker at the time, made it clear he wasn’t welcome to do so.
However, it didn’t appear to dampen Mr Trump’s excitement about his time with the Royal Family.
Speaking earlier this year, he described his state visit as “a fest” adding “it’s an honour… I’m a friend of Charles, I have great respect for King Charles and the family, William; we have really just a great respect for the family. And I think they’re setting a date for September.”
It is expected that, like Mr Macron, the pageantry for his trip this time will revolve around Windsor, with refurbishment taking place at Buckingham Palace.
Liverpool have retired the number 20 shirt in honour of Diogo Jota – the first time it has made such a gesture.
The club said it was a “unique tribute to a uniquely wonderful person” and the decision was made in consultation with his wife and family.
The number 20 will be retired at all levels, including the men’s and women’s first teams and academy squads.
A statement said: “It was the number he wore with pride and distinction, leading us to countless victories in the process – and Diogo Jota will forever be Liverpool Football Club’s number 20.”
The club called it a “recognition of not only the immeasurable contribution our lad from Portugal made to the Reds’ on-pitch successes over the last five years, but also the profound personal impact he had on his teammates, colleagues and supporters and the everlasting connections he built with them”.
Image: Jota’s wife joined Liverpool players to view tributes at Anfield on Friday. Pic: Liverpool FC
Image: Pic: Liverpool FC
Newly-married Jota died alongside his brother when his Lamborghini crashed in northern Spain on 3 July.