Microsoft on Thursday announced new health-care data and artificial intelligence tools, including a collection of medical imaging models, a health-care agent service and an automated documentation solution for nurses. 

The tools aim to help health-care organizations build AI applications quicker and save clinicians time on administrative tasks, a major cause of industry burnout. Nurses spend as much as 41% of their time on documentation, according to a report from the Office of the Surgeon General. 

“By integrating AI into health care, our goal is to reduce the strain on medical staff, foster the collective health team collaboration, enhance the overall efficiency of healthcare systems across the country,” Mary Varghese Presti, vice president of portfolio evolution and incubation at Microsoft Health and Life Sciences, said in a prerecorded briefing with reporters. 

The new tools are the latest example of Microsoft’s efforts to establish itself as a leader in health-care AI. Last October, the company unveiled a series of health features across its Azure cloud and Fabric analytics platform. It also acquired Nuance Communications, which offers speech-to-text AI solutions for health care and other sectors, in a $16 billion deal in 2021.

Many of the solutions Microsoft announced on Thursday are in the early stages of development or only available in preview. Health-care organizations will test and validate them before the company rolls them out more broadly. Microsoft declined to share what these new tools will cost.

Health-care AI models 

Roughly 80% of hospital and health system visits include an imaging exam because doctors often rely on images to help treat patients.

Microsoft is launching a collection of open-source multimodal AI models that can analyze data types beyond just text, such as medical images, clinical records and genomic data. Health-care organizations can use the models to build new applications and tools.

For example, digitizing a single pathology slide can require more than a gigabyte of storage, so many existing AI pathology models have trained on small pieces of slides at a time. Microsoft and Providence Health & Services built a whole-slide model that improves on mutation prediction and cancer subtyping, according to a paper published in the peer-reviewed journal Nature.

Now, health systems can build on it and fine-tune it to meet their needs. 

“Getting a whole-slide foundation model for pathology has been a challenge in the past … and now we’re actually able to do it,” Sara Vaezy, chief strategy and digital officer at Providence, told CNBC in an interview. “It was really sort of a game changer.” 

The models are available in the model catalog within Azure AI Studio, which serves as Microsoft’s generative AI development hub. 

Health-care agent service

Microsoft also announced a new way for health systems to build AI agents.

AI agents vary in complexity, but they can help users answer questions, automate processes and perform specific tasks. 

Through Microsoft Copilot Studio, these organizations can create agents equipped with health-care-specific safeguards. When an answer contains a reference to clinical evidence, for instance, the source is shown, and a note indicates if the answer is AI-generated. Fabrications and omissions are also flagged, Microsoft said. 

For example, a health-care organization could build an AI agent to help doctors identify relevant clinical trials for a patient. Microsoft said a physician could type the question, “What clinical trials for a male 55-year-old with diabetes and interstitial lung disease?” and receive a list of potential options. It would save the doctor the time and effort of finding each trial. 

AI agents that can help patients answer basic questions have been popular among the health systems that have already begun testing the service, Hadas Bitran, general manager of health AI at Microsoft Health and Life Sciences, said in a Q&A with reporters. Agents that can help doctors answer questions about recent guidelines and patients’ history are also common, she added.

Microsoft’s health-care agent service is available in a preview capacity starting Thursday.

Bringing automated documentation to nurses

In August, Microsoft announced that the next phase of its partnership with Epic Systems would be dedicated to building an AI-powered documentation tool for nurses, and the company detailed those plans on Thursday. 

Epic is a health-care software vendor that houses the electronic health records of more than 280 million people in the U.S. It has a yearslong relationship with Microsoft. 

Microsoft’s Nuance already offers an automated documentation tool for doctors called DAX Copilot, which it unveiled last year. It allows doctors to consensually record their visits with patients, and AI automatically transforms them into clinical notes and summaries.

Ideally, this means doctors don’t have to spend time typing out these notes themselves every time they see a patient. 

The technology has exploded in popularity this year. Nuance announced that DAX Copilot was generally available within Epic’s electronic health record in January – a coveted stamp of approval within the health-care industry. Integrating a tool like DAX Copilot directly into doctors’ EHR workflow means they won’t need to switch apps to access it, which helps save time and reduces administrative workload.

But so far, DAX Copilot has only been available to doctors. Microsoft said that’s changing. It’s building a similar tool optimized for nurses.

“The nursing workflow is very different from that of physicians, and any solution developed for nurses needs to integrate with the way they work,” Presti said during the briefing. “Our team has spent hours shadowing nurses during their shifts to see how they carry out their tasks and to discover where the greatest points of friction exist throughout their day.”  

Microsoft is working with organizations like Stanford Health Care, Northwestern Medicine and Tampa General Hospital to develop it.