Using multimodal large language models (LLMs), hospital systems can develop advanced virtual doctor assistants to proactively monitor and diagnose patient conditions, according to Dr. Ashley Beecy, the medical director of AI operations at NewYork-Presbyterian (NYP).
Speaking at a recent AI Impact Tour event in New York, Dr. Beecy highlighted how NYP is already leveraging generative AI in low-risk areas such as summarizing patient visit conversations. She expressed hope that the growing enthusiasm for generative AI will drive workflow changes that enable hospitals to create comprehensive assistants that will revolutionize patient care.
Multimodal LLM Technology for Proactive Care
Dr. Beecy, also a practicing cardiologist, stated that while she didn’t specify a timeline for implementing these advancements, she aims for progress within the next year. Currently, patients are referred to her after experiencing chest pain, but she'd prefer to anticipate heart attack risks before they occur. “We can utilize this technology and our patient data to gain insights from various modalities—imaging, echocardiograms, and electrocardiograms—that may not be visible to human eyes but can be recognized by AI, allowing timely intervention,” she explained.
While the technical foundations for this capability are largely established, Dr. Beecy emphasized the need for change management within hospital workflows. Collaborating with national health organizations will be crucial to effectuate broader structural changes. The approach involves initially targeting low-risk administrative tasks, such as summarizing patient conversations, before advancing to clinical diagnostics for detecting conditions like heart disease.
“I envision a model that integrates all these functions seamlessly, enabling me to know when my next patient is arriving, how long their appointment should be, and offering summaries of their previous visits—all populated into electronic records for easy access and action,” she shared.
Employee Engagement and Workflow Refinement
Dr. Beecy noted that NYP employees are generally receptive to generative AI, demonstrating eagerness to engage with the technology. NYP employs approximately 49,000 staff members and affiliated physicians linked to Cornell and Columbia University.
In her discussion, moderated by AI writer Sharon Goldman, Dr. Beecy underscored AI's potential in enhancing key applications like pattern recognition, data extraction, and summarization—all pivotal in reducing the administrative load on doctors. One of her preferred applications involves recording patient visits, enabling real-time transcription into notes, thus alleviating the manual transcription burden. However, she acknowledged the necessity of obtaining patient consent for recording visits, prioritizing transparency.
Transitioning from administrative applications to clinical diagnostics is more complex, yet NYP is exploring using electrocardiograms for identifying structural heart diseases. “Many individuals receive electrocardiograms, which can also detect heart disease, enabling earlier intervention and care,” she said.
Balancing Risks and Innovations
When asked about potential risks associated with generative AI’s implementation, Dr. Beecy recognized several concerns but emphasized that doctor oversight of visit summaries and diagnoses could mitigate many risks. "The technology is not perfect; it hovers around 90 percent accuracy, which is why provider review is critical," she explained.
Another potential issue is the risk of overreliance on AI technology. Dr. Beecy noted the rapid advancements in LLMs, citing the evolution from GPT-3.5 to GPT-4, and cautioned against complacency in human oversight.
NYP is adopting a cautious yet progressive approach, ensuring the technology aligns with both stakeholders interested in AI tools and those utilizing them. While there are valid apprehensions about integration into existing workflows, Dr. Beecy reported a prevailing excitement among staff eager to pilot new initiatives.
Democratizing Technology in Healthcare
Dr. Beecy highlighted a shift in how technology is integrated within organizations, noting that for the first time, healthcare professionals at NYP have direct access to generative AI tools like ChatGPT. This decentralization fosters engagement as providers discover practical use cases relevant to their work.
NYP is also reaching out to patient groups to gauge transparency around AI's application. Questions revolve around whether patients wish to be informed when AI is utilized, prompting a need for multidisciplinary input, including insights from sociologists and bioethicists.
Following Dr. Beecy’s presentation, Sarah Bird, Microsoft’s global lead for responsible AI engineering, discussed the foundational role of technology in crafting an all-encompassing doctor’s assistant. She highlighted that while generative AI can streamline processes, accuracy in summaries remains crucial, as missing critical information could significantly alter a patient's diagnosis.
In another session, Promiti Dutta from Citi shared how generative AI is reshaping the finance sector, emphasizing a cultural shift towards data-informed decision-making, yet noted that external-facing LLM-driven chatbots are still considered too risky for implementation.
Event Note: Microsoft sponsored this segment of the AI Impact Tour, but the speakers were independently selected. Stay tuned for upcoming stops on the AI Impact Tour, including events in Boston on March 27 and Atlanta on April 10.
Ranking
