Connect with us

Published

on

Apr 17 2024 University of Cambridge

The clinical knowledge and reasoning skills of GPT-4 are approaching the level of specialist eye doctors, a study led by the University of Cambridge has found.

GPT-4 – a 'large language model' – was tested against doctors at different stages in their careers, including unspecialized junior doctors, and trainee and expert eye doctors. Each was presented with a series of 87 patient scenarios involving a specific eye problem, and asked to give a diagnosis or advise on treatment by selecting from four options.

GPT-4 scored significantly better in the test than unspecialized junior doctors, who are comparable to general practitioners in their level of specialist eye knowledge.

GPT-4 gained similar scores to trainee and expert eye doctors – although the top performing doctors scored higher.

The researchers say that large language models aren't likely to replace healthcare professionals, but have the potential to improve healthcare as part of the clinical workflow.

They say state-of-the-art large language models like GPT-4 could be useful for providing eye-related advice, diagnosis, and management suggestions in well-controlled contexts, like triaging patients, or where access to specialist healthcare professionals is limited.

"We could realistically deploy AI in triaging patients with eye issues to decide which cases are emergencies that need to be seen by a specialist immediately, which can be seen by a GP, and which don't need treatment," said Dr Arun Thirunavukarasu, lead author of the study, which he carried out while a student at the University of Cambridge's School of Clinical Medicine

He added: "The models could follow clear algorithms already in use, and we've found that GPT-4 is as good as expert clinicians at processing eye symptoms and signs to answer more complicated questions.

"With further development, large language models could also advise GPs who are struggling to get prompt advice from eye doctors. People in the UK are waiting longer than ever for eye care.

Large volumes of clinical text are needed to help fine-tune and develop these models, and work is ongoing around the world to facilitate this. Related StoriesStudy reveals crucial insights into the ocular effects of Zika virus infection during pregnancyEye movement reflex reveals genetic association with autismNaturally-occurring material is an effective disinfectant for contact lenses, study suggests

The researchers say that their study is superior to similar, previous studies because they compared the abilities of AI to practicing doctors, rather than to sets of examination results.

"Doctors aren't revising for exams for their whole career. We wanted to see how AI fared when pitted against to the on-the-spot knowledge and abilities of practicing doctors, to provide a fair comparison," said Thirunavukarasu, who is now an Academic Foundation Doctor at Oxford University Hospitals NHS Foundation Trust.

He added: "We also need to characterise the capabilities and limitations of commercially available models, as patients may already be using them – rather than the internet – for advice."

The test included questions about a huge range of eye problems, including extreme light sensitivity, decreased vision, lesions, itchy and painful eyes, taken from a textbook used to test trainee eye doctors. This textbook is not freely available on the internet, making it unlikely that its content was included in GPT-4's training datasets.

The results are published today in the journal PLOS Digital Health. Even taking the future use of AI into account, I think doctors will continue to be in charge of patient care. The most important thing is to empower patients to decide whether they want computer systems to be involved or not. That will be an individual decision for each patient to make."

Dr. Arun Thirunavukarasu, lead author of the study

GPT-4 and GPT-3.5 – or 'Generative Pre-trained Transformers' – are trained on datasets containing hundreds of billions of words from articles, books, and other internet sources. These are two examples of large language models; others in wide use include Pathways Language Model 2 (PaLM 2) and Large Language Model Meta AI 2 (LLaMA 2).

The study also tested GPT-3.5, PaLM2, and LLaMA with the same set of questions. GPT-4 gave more accurate responses than all of them.

GPT-4 powers the online chatbot ChatGPT to provide bespoke responses to human queries. In recent months, ChatGPT has attracted significant attention in medicine for attaining passing level performance in medical school examinations, and providing more accurate and empathetic messages than human doctors in response to patient queries.

The field of artificially intelligent large language models is moving very rapidly. Since the study was conducted, more advanced models have been released – which may be even closer to the level of expert eye doctors. Source:

University of CambridgeJournal reference:

Thirunavukarasu, A. J., et al. (2024) Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study. PLOS Digital Health. doi.org/10.1371/journal.pdig.0000341.

Continue Reading

Sports

Oshie would ‘love to play’ 17th year, if back obliges

Published

on

By

Oshie would 'love to play' 17th year, if back obliges

ARLINGTON, Va. — T.J. Oshie lingered on the ice long after many of his Washington Capitals teammates departed for the locker room following the loss that eliminated them from the playoffs.

His family was at all four games, including in New York, just in case. They could be the final ones he plays in the NHL.

Oshie, 37, isn’t calling it a career just yet, but the winger acknowledged Tuesday he and doctors must find a solution to his chronic back problems before committing to return for a 17th season.

“I’d love to play next year, but I will need to come back with somewhat of a guarantee that my back won’t be — it’s hard putting everyone through the situation,” Oshie said. “I’d like to find just an answer and a fix to the problem before I make another run at it.”

Oshie’s back and other injuries limited him to 52 of 82 games during the regular season, and he played the series finale against the Rangers with a broken left hand.

“A broken hand is like a hangnail compared to what I go through with my back,” Oshie said, adding he’s often on the floor unable to move when it gives out. “There’s been a lot of just extra things that I have to do to make sure that my back is ready to play an NHL game.”

Already expecting center Nicklas Backstrom to remain on long-term injured in the final year of his contract after stepping away in November because of a nagging hip issue, general manager Brian MacLellan said the team will support Oshie whichever decision he makes and doesn’t have a date by which he’d like an answer.

“If he determines at some point that he feels good, he wants to come back to play, let’s go that way,” MacLellan said. “If he doesn’t, we’ll work it out that way, too.”

Oshie’s presence or lack thereof would certainly affect how the Capitals affect the offseason, whether they can use his $5.75 million salary space or need to fill a major void.

“He’s a big part of our team,” MacLellan said. “You have him, or you go out and find a guy.”

Oshie, who reached the 1,000 games milestone and scored the empty-net goal that clinched a playoff spot, long ago emerged as a popular teammate and fan favorite and was the Capitals’ emotional engine for nearly a decade while playing through injuries.

“He’s a warrior,” captain Alex Ovechkin said. “He’s a guy who brings energy on the ice and off the ice. He have so much respect from the coaching staff to the fans and from the players, as well. He’s a guy who brings everything to this group on the ice and off the ice.”

As for Ovechkin, after scoring just eight goals in his first 43 games of the season, the superstar went on a tear down the stretch to finish with 31 before being held without a point in a playoff series for the first time. At 853, he’s 42 away from Wayne Gretzky’s NHL career record, turns 39 in September and has two years remaining on his contract.

“If I make it, it’s good, but it’s still 42 goals,” Ovechkin said. “It’s kind of a long way. But it’s possible if you’re going to be healthy, yeah.”

MacLellan joked about penciling in Ovechkin for 42 next season, knowing full well it’s unfair to expect one of the oldest players in the league to carry Washington offensively. His hope is also to add through trades and free agency to give Ovechkin some help.

Continue Reading

Sports

Jets won’t have Namestnikov, Dillon for Game 5

Published

on

By

Jets won't have Namestnikov, Dillon for Game 5

WINNIPEG, Manitoba — Winnipeg forward Vladislav Namestnikov and defenseman Brenden Dillon are recovering from injuries and won’t be available for Game 5 against Colorado on Tuesday night as the Jets try to avoid playoff elimination.

Namestnikov fractured his cheekbone in Game 4 on Sunday when he was hit by a deflected puck on a shot. Dillon suffered a deep laceration on his hand in a scrum after the final horn of Game 3 when he appeared to be cut by a skate blade. The Jets trail 3-1 in the first-round, best-of-seven series.

Jets coach Rick Bowness said Tuesday following the morning skate in Winnipeg that he checked in with Namestnikov the night before. Namestnikov stayed an extra day in Denver to recover.

“He’s nauseous. He hasn’t eaten. So he’s not feeling great,” Bowness said. “Dilly, of course, you know him — two broken hands and he’d still want to play, but we’ll give him a couple days.”

Forward Cole Perfetti, a first-round pick in 2020, is expected to make his NHL playoff debut. It’s a difficult moment for the 22-year-old after seeing the injury to Namestnikov.

“He’s one of our brothers and such a great guy. He plays his heart out every single game,” Perfetti said. “To see that happen, I mean, for the people that were there in the rink it was silent. You could hear a pin drop. It was tough to watch, to see one of your really good friends and teammate and brother go down like that and kind of not know really what was happening.”

Namestnikov, 31, was trying to jump out of the way of a shot from teammate Nate Schmidt when the puck hit a stick and struck him. Namestnikov stayed down on the ice as trainers rushed out. He went to the hospital for further evaluation.

“It was great to see him after he saw the doctors and what-not, and he was doing relatively OK,” Perfetti said. “That was a big relief for us.”

Continue Reading

Sports

Bedard, Faber, Hughes named Calder finalists

Published

on

By

Bedard, Faber, Hughes named Calder finalists

Connor Bedard, Brock Faber and Luke Hughes are this year’s finalists for the Calder Trophy, the NHL announced Tuesday.

The Calder Trophy is awarded to “the player selected as the most proficient in his first year of competition” and is determined through voting by the Professional Hockey Writers Association.

Bedard, who was the first pick of the 2023 NHL draft, has long been considered to be the league’s next great generational talent. His first season with the Chicago Blackhawks only added to those expectations, as the 18-year-old showed he could handle the demands of playing as a top-line center, tying for the team lead with 22 goals in 68 games and leading the Blackhawks with 39 assists and 61 points.

Despite missing 14 games with a fractured jaw, Bedard led all rookies in goals, points and was tied for first in assists.

Faber, who was a second-round pick in 2020, played his first full season for his hometown Minnesota Wild. Already in a top-four role, the 21-year-old defenseman took on additional importance as the team was battling injuries on the back end.

Faber finished with 8 goals, 39 assists and 47 points while averaging just under 25 minutes of ice time — leading all first-year players as well as the Wild — over 82 games. He finished tied with Bedard for the most rookie assists and was tied for second in points.

Hughes, a first-round pick by the New Jersey Devils in 2021, also played his first full campaign this season. The 20-year-old defenseman’s role continually grew throughout the season as he finished with 9 goals, 38 assists, 47 points and averaged 21:28 of ice time over 82 games. Hughes led all rookies in power-play points, was tied for second in points with Faber, finished third in assists and was second in ice time among newcomers with more than 20 games.

Hughes also led the Devils in ice time while finishing fifth on the team in points.

Continue Reading

Trending