Connect with us

Published

on

Apr 17 2024 University of Cambridge

The clinical knowledge and reasoning skills of GPT-4 are approaching the level of specialist eye doctors, a study led by the University of Cambridge has found.

GPT-4 – a 'large language model' – was tested against doctors at different stages in their careers, including unspecialized junior doctors, and trainee and expert eye doctors. Each was presented with a series of 87 patient scenarios involving a specific eye problem, and asked to give a diagnosis or advise on treatment by selecting from four options.

GPT-4 scored significantly better in the test than unspecialized junior doctors, who are comparable to general practitioners in their level of specialist eye knowledge.

GPT-4 gained similar scores to trainee and expert eye doctors – although the top performing doctors scored higher.

The researchers say that large language models aren't likely to replace healthcare professionals, but have the potential to improve healthcare as part of the clinical workflow.

They say state-of-the-art large language models like GPT-4 could be useful for providing eye-related advice, diagnosis, and management suggestions in well-controlled contexts, like triaging patients, or where access to specialist healthcare professionals is limited.

"We could realistically deploy AI in triaging patients with eye issues to decide which cases are emergencies that need to be seen by a specialist immediately, which can be seen by a GP, and which don't need treatment," said Dr Arun Thirunavukarasu, lead author of the study, which he carried out while a student at the University of Cambridge's School of Clinical Medicine

He added: "The models could follow clear algorithms already in use, and we've found that GPT-4 is as good as expert clinicians at processing eye symptoms and signs to answer more complicated questions.

"With further development, large language models could also advise GPs who are struggling to get prompt advice from eye doctors. People in the UK are waiting longer than ever for eye care.

Large volumes of clinical text are needed to help fine-tune and develop these models, and work is ongoing around the world to facilitate this. Related StoriesStudy reveals crucial insights into the ocular effects of Zika virus infection during pregnancyEye movement reflex reveals genetic association with autismNaturally-occurring material is an effective disinfectant for contact lenses, study suggests

The researchers say that their study is superior to similar, previous studies because they compared the abilities of AI to practicing doctors, rather than to sets of examination results.

"Doctors aren't revising for exams for their whole career. We wanted to see how AI fared when pitted against to the on-the-spot knowledge and abilities of practicing doctors, to provide a fair comparison," said Thirunavukarasu, who is now an Academic Foundation Doctor at Oxford University Hospitals NHS Foundation Trust.

He added: "We also need to characterise the capabilities and limitations of commercially available models, as patients may already be using them – rather than the internet – for advice."

The test included questions about a huge range of eye problems, including extreme light sensitivity, decreased vision, lesions, itchy and painful eyes, taken from a textbook used to test trainee eye doctors. This textbook is not freely available on the internet, making it unlikely that its content was included in GPT-4's training datasets.

The results are published today in the journal PLOS Digital Health. Even taking the future use of AI into account, I think doctors will continue to be in charge of patient care. The most important thing is to empower patients to decide whether they want computer systems to be involved or not. That will be an individual decision for each patient to make."

Dr. Arun Thirunavukarasu, lead author of the study

GPT-4 and GPT-3.5 – or 'Generative Pre-trained Transformers' – are trained on datasets containing hundreds of billions of words from articles, books, and other internet sources. These are two examples of large language models; others in wide use include Pathways Language Model 2 (PaLM 2) and Large Language Model Meta AI 2 (LLaMA 2).

The study also tested GPT-3.5, PaLM2, and LLaMA with the same set of questions. GPT-4 gave more accurate responses than all of them.

GPT-4 powers the online chatbot ChatGPT to provide bespoke responses to human queries. In recent months, ChatGPT has attracted significant attention in medicine for attaining passing level performance in medical school examinations, and providing more accurate and empathetic messages than human doctors in response to patient queries.

The field of artificially intelligent large language models is moving very rapidly. Since the study was conducted, more advanced models have been released – which may be even closer to the level of expert eye doctors. Source:

University of CambridgeJournal reference:

Thirunavukarasu, A. J., et al. (2024) Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study. PLOS Digital Health. doi.org/10.1371/journal.pdig.0000341.

Continue Reading

Sports

The Gators, a top QB and a $13.85 million NIL deal gone wrong: What to know and what’s next

Published

on

By

The Gators, a top QB and a .85 million NIL deal gone wrong: What to know and what's next

Jaden Rashada, a former ESPN 300 quarterback recruit, sued the University of Florida and coach Billy Napier on Tuesday. Rashada is claiming he was defrauded of millions of dollars in name, image and likeness money.

The lawsuit, among other things, alleges that Napier promised the player’s father a $1 million “partial payment” upon signing. Rashada never got the money, and the boosters never fulfilled the deal, the lawsuit states.

It is the latest in a long saga involving the player and the school. Now, Rashada is the first college athlete known to sue his coach or a booster due to a dispute over an NIL deal.

What’s next for Rashada and Florida? Our reporters break it down.

How did we get here?

Rashada was ranked No. 31 overall in the 2023 class and had a prep career that featured several transfers. His freshman season in high school was played at Liberty High School (Brentwood, California). He then transferred to IMG Academy in Florida before going back to California to play three seasons at Pittsburg (California) High School. His college decision came down to two Florida schools and two big NIL numbers.

Rashada committed to the Miami Hurricanes in June 2022. The lawsuit states that Rashada had a $9.5 million NIL promise with the Canes. But Rashada decommitted that November and promptly flipped to the Florida Gators and Coach Napier. He signed his national letter of intent during the December signing period. However, he didn’t enroll at Florida, and his arrival in Gainesville was contingent on a four-year, $13.85 million NIL deal. Rashada asked for a release from his letter of intent when the deal fell through.

The Gator Collective, an independent fundraising organization that distributed money to UF athletes at the time, was responsible for the deal. However, the financial backing did not materialize, and the Gator Collective terminated the contract. Rashada was released from his letter of intent. He then took a visit to Arizona State and committed to play for the Sun Devils, arriving on campus in July 2023.

Rashada started the first two games of the season for Arizona State, but an injury kept him out for most of the season. In three games, he was 44-of-82 for 485 yards, with four touchdowns and three interceptions.

Rashada entered the transfer portal on April 18 and is now transferring to Georgia, where he will have four years of eligibility remaining. — Tom VanHaaren


What’s next for Rashada?

Georgia emerged as Rashada’s likely transfer destination soon after he entered the portal. He selected the Bulldogs on April 25, captioning his Instagram announcement, “Compete with the BEST.” Rather than vying for the starting job at ASU with emerging Michigan State transfer Sam Leavitt, Rashada enters a situation where he will play behind Georgia’s Carson Beck, a top Heisman Trophy contender and NFL hopeful.

Barring an injury to Beck, Rashada will use the 2024 season for developmental purposes, while absorbing a different offense under coordinator Mike Bobo in Athens.

A realistic goal would be to start in 2025 for a Bulldogs team always in the national title hunt. He would need to beat out primary challenger Gunner Stockton, who enters his third season in the Georgia program and gained valuable experience this spring. Both Stockton and incoming freshman Ryan Puglisi ranked among ESPN’s top 110 recruits in their respective classes. Coach Kirby Smart likes to have four scholarship quarterbacks on his roster at all times, and the team has a 2025 commitment from ESPN 300 recruit Ryan Montgomery.

Rashada is the most decorated quarterback prospect of the bunch, but he will need to adjust quickly, especially with Stockton ahead on the learning curve. — Adam Rittenberg


How did this affect Florida on the field?

The Gators signed Rashada in December 2022, anticipating he would compete for the starting job with veteran Graham Mertz, who transferred in from Wisconsin. Without him, the job went to Mertz, who threw for 2,903 yards, completed 73% of his passes and threw 20 touchdown passes to three interceptions last season. Losing Rashada did not help Napier from a public relations standpoint, as he drew ire from the Florida fan base — especially after a 5-7 finish to 2023.

Florida did have a plan for its future, though, as Napier had a longstanding commitment from D.J. Lagway, the top-rated quarterback in the class of 2024. Lagway committed to Florida just weeks before Rashada signed his letter of intent, and the Gators were in a massive fight to keep Lagway as other schools went after him as signing day approached last December. Lagway has said Clemson, USC and Texas A&M all made late pushes, but he ultimately signed with the Gators and enrolled early, going through spring football.

His decision to come to Florida was absolutely huge for Napier, who has yet to find solid footing headed into Year 3 and desperately needed Lagway to sign. Mertz is back for one more season and is the presumptive starter. But there is a lot riding on this season for Florida and for Napier in particular. With the season opener against rival Miami, Florida fans want to see progress and results immediately. If Mertz struggles, do not be surprised if fans start calling for Lagway. — Andrea Adelson


Could the Gators face any NCAA repercussions from Rashada’s claims?

The NCAA alerted Florida almost a year ago that it was investigating claims that the football program had violated recruiting rules, according to documents obtained by The Associated Press. ESPN’s Mark Schlabach confirmed the investigation was related to Rashada’s recruitment.

Rashada claims in his lawsuit that Napier directly promised Rashada a $1 million payment from a booster if he signed with Florida, which would be a violation of the NCAA’s policies. The NCAA penalized Florida State and suspended one of its coaches in January for getting involved in NIL offers made to a prospect.

However, the NCAA sent a letter to its schools in late February explaining that it was pausing all open enforcement cases “involving third-party participation in NIL-related activities” after a federal judge in Tennessee granted an injunction that prohibited the NCAA from enforcing some of its rules. That case was filed by Tennessee’s attorney general after the NCAA opened an investigation into the Vols’ recruiting tactics.

Any other high-profile enforcement actions taken by the NCAA — such as a penalty for Napier or the Gators — could invite additional lawsuits at a time when the association remains vulnerable to antitrust scrutiny, which makes it unlikely that any NCAA punishments could be coming soon. — Dan Murphy

Continue Reading

Sports

Yankees ace Cole faces hitters, eyes June return

Published

on

By

Yankees ace Cole faces hitters, eyes June return

NEW YORK — New York Yankees ace Gerrit Cole says he thinks he could return to a major league mound in June after facing hitters for the first time since spring training.

The reigning AL Cy Young Award winner, wearing a full pinstriped uniform, threw 20 pitches to Oswaldo Cabrera and Jamai Jones before Tuesday night’s game against Seattle.

Cole had thrown five bullpen sessions before the batting practice. The 33-year-old right-hander, sidelined by elbow nerve inflammation and edema, said he will have one or two more BP sessions before starting a minor league rehabilitation assignment.

Continue Reading

Sports

Cubs activate Swanson ahead of Braves series

Published

on

By

Cubs activate Swanson ahead of Braves series

CHICAGO — The Chicago Cubs on Tuesday activated shortstop Dansby Swanson from the 10-day injured list and recalled infielder Luis Vázquez from Triple-A Iowa.

Swanson, 30, was placed on the injured list May 8 because of a right knee sprain. He’s batting .209 with three doubles, one triple and four home runs in 37 games.

Vázquez, 24, earned his first major league call-up after batting .262 with 10 doubles and three homers in 39 games with Iowa.

The Cubs opened a home series against Swanson’s former team, Atlanta, on Tuesday night. He was in the lineup, batting eighth and playing shortstop. In the corresponding roster moves, the Cubs optioned Pete Crow-Armstrong and infielder Miles Mastrobuoni to Iowa.

Information from The Associated Press was used in this report.

Continue Reading

Trending