The technology behind ChatGPT scored better at assessing eye problems and providing advice than non-specialist doctors, a new study has found.
A study led by the University of Cambridge has found that GPT-4, the large language model (LLM) developed by OpenAI, performed nearly as well as specialist eye doctors in a written multiple-choice test.
The AI model, which is known for generating text based on the vast amount of data it is trained on, was tested against doctors at different stages of their careers, including junior doctors without a specialism, as well as trainee and expert eye doctors.
Each group was presented with dozens of scenarios where patients have a specific eye problem, and asked to give a diagnosis or advise on treatment by selecting from one of four options.
The test was based on written questions, taken from a textbook used to test trainee eye doctors, about a range of eye problems – including sensitivity to light, decreased vision, lesions, and itchy eyes.
The textbook on which the questions are based is not publicly available, so researchers believe it is unlikely the large language model has been trained on its contents.
GPT-4 scored significantly higher than junior doctors, whose level of specialism is comparable to general practitioners, at the test.
The model achieved similar scores to trainee and expert eye doctors, but it was beaten by the top-performing experts.
Follow Sky News on WhatsApp
Keep up with all the latest news from the UK and around the world by following Sky News
The research was conducted last year using the latest available large language models.
Advertisement
The study also tested GPT-3.5, an earlier version of OpenAI’s model, Google’s PaLM2, and Meta’s LLaMA on the same set of questions. GPT-4 gave more accurate responses than any of the other models.
The researchers have said that large language models will not replace doctors, but they could improve the healthcare system and reduce waiting lists by supporting doctors to deliver care to more patients in the same amount of time.
Dr Arun Thirunavukarasu, the lead author of the paper, said: “If we had models that could deliver care of a similar standard to that delivered by humans, that would help overcome the problems of NHSwaiting lists.
“What that requires is trials to make sure it’s a safe and effective model. But if it is, it could be revolutionary for how care is delivered.”
He added: “While the study doesn’t indicate deployment of LLMs in clinical work immediately, it gives a green light to start developing LLM-based clinical tools as the knowledge and reasoning of these models compared well to the expert ophthalmologists.”
The use of drones to fly drugs and weapons into UK prisons needs to be tackled “urgently” because it has become a threat to “national security”, the chief inspector of prisons has said.
Charlie Taylor added that police and the prison service had in effect “ceded the airspace” around HMP Manchester and HMP Long Lartin to organised crime gangs who are using the gadgets to deliver contraband to inmates.
The two high security prisons hold some of the most dangerous men in the country, including terrorists and organised crime bosses.
Mr Taylor’s warning comes after inspections of HMP Manchester, based in the city centre, and HMP Long Lartin, in Evesham, Worcestershire, found both prisons had “thriving illicit economies” of drugs, mobile phones and weapons.
Inspectors also found that basic security measures such as protective netting and CCTV had fallen into disrepair.
Some inmates at HMP Manchester, a category B jail which holds a small number of category A prisoners, had burned holes in windows so that they could receive regular deliveries by drone, the HM Inspectorate of Prisons watchdog said as it published the findings of the inspections.
It added that many of the drones had “increasingly large payloads” which “had the potential to lead to serious disruption and even escape”.
More from UK
Mr Taylor said: “It is highly alarming that the police and prison service have, in effect ceded the airspace above two high-security prisons to organised crime gangs which are able to deliver contraband to jails holding extremely dangerous prisoners including some who have been designated as high-risk category A.
“The safety of staff, prisoners and ultimately that of the public, is seriously compromised by the failure to tackle what has become a threat to national security.
“The prison service, the police and other security services must urgently confront organised gang activity and reduce the supply of drugs and other illicit items which so clearly undermine every aspect of prison life.”
Inspectors found prisoners had been using the elements from their kettles to burn holes in their “inadequately protected” Perspex windows to allow the “entry of drones laden with contraband”.
The inspections at HMP Manchester and HMP Long Lartin, which took place across September and October 2024, also revealed other serious concerns around safety and security at both sites.
Mr Taylor felt the situation was so bad at HMP Manchester that he issued an urgent notification for improvement to the Justice Secretary Shabana Mahmood.
At Manchester, 39% of prisoners tested positive in mandatory drug tests, while at Long Lartin, 50% of those who responded to the watchdog’s survey said it was easy to get drugs and alcohol.
Violence and self-harm at both prisons was also found to have increased, which the watchdog said was partly driven by drugs and the accompanying debt prisoners found themselves in.
Meanwhile there had been six “self-inflicted deaths” at Manchester since the watchdog’s last inspection in 2021, with a seventh taking place after the most recent visit.
It is now one of the most violent prisons in the country, with a high number of serious assaults against prisoners and staff.
Many officers “lacked confidence, were demoralised, and were struggling to manage experienced prisoners who were serving long sentences for serious crimes”, the watchdog said.
Manchester was also found to have a chronic rodent infestation, while there was widespread dirt, damp and litter at both sites.
Prisoners at Manchester were also found to have used torn-up foam from mattresses and pillows to keep out the cold.
Inspectors found 38% of prisoners there were locked up during the working day and poor attendance at education and work was further fuelling the boredom, drug-taking, self-harm and violence.
At Long Lartin, which houses both category A and B prisoners, a continued lack of in-cell toilets for many prisoners led them to use buckets and throw bags of excrement out of the windows, many of which were not cleared up, the watchdog said.
The Ministry of Justice said in a statement: “This government inherited prisons in crisis – overcrowded, with drugs and violence rife.
“We are gripping the situation by investing in prison maintenance and security, working with the police and others to tackle serious organised crime, and building more prison places to lock up dangerous criminals.”
Reform UK has grown in support to within one percentage point of Labour according to a new poll for Sky News by YouGov which suggests Britain has entered a new era of three-way party politics.
Sir Keir Starmer looks set to spend the parliament locked in a fight with two right-wing parties after Labour support dropped sharply in the first YouGov poll since the general election.
This is the first of YouGov’s weekly voting intention polls for Sky News, shared with The Times.
It reflects a drop in satisfaction with the government, a rise in support for Reform UK, and shows how the Labour vote has split in all directions since the election.
Labour has retained 54% of their vote at the general election – 7% have gone to the Lib Dems, 6% to the Green Party, 5% to Reform UK, 4% to the Tories – while 23% of those polled did not say, did not know or would not vote.
Reform UK’s vote has grown since the general election at the expense of all other parties, with 16% of voters who backed the Tories at the ballot last year now saying they’d support Reform.
The judgement on Sir Keir’s first six months in office is damning, however.
Some 10% say the government has been successful while 60% say unsuccessful.
Older voters have turned away from Labour. Just 14% of over 65s would now vote Labour, down from 22% around the time of the election.
However, there are signs the Tory party remains a toxic brand. Reform UK are the least unpopular party, with a net favourability rating of -32, Labour a touch worse on -34 and the Tories down on -45.
YouGov interviewed 2,279 voters in Great Britain on Sunday 12 January and Monday 13 January.
A woman in her 40s has been arrested on suspicion of murder after the body of a man was found in Greater Manchester.
The man, in his 50s, was found dead at an address in Hope Hey Lane, Little Hulton, on Sunday morning after reports of concern for his welfare.
Following a post-mortem examination, Greater Manchester Police (GMP) said it had launched a murder investigation as his injuries were consistent with an assault.
Officers subsequently arrested the woman and she remains in police custody for questioning.
Detective Chief Inspector Neil Higginson, from the force’s Major Incident Team, said: “Sadly, following the discovery of a body at a property in Little Hulton, we have now launched a murder investigation, and we have a team of detectives working around the clock to understand the circumstances.
“We do not believe there to be a threat to the wider public, but you will likely see an increased presence of police in your area whilst we conduct further enquiries.
“If you have any information which may assist our investigation, or any dashcam, CCTV, or doorbell footage from the area in the last 24 hours, please get in touch with us.”
He added: “No matter how small the information may seem, it could be crucial to our investigation.”