As I look at startups and incumbents using LLMs / Generative AI in healthcare for provider workflows, I am trying to separate fact from hope, and current reality vs. the likely progress in the next 1 - 2 years. Wondering if there is anything in the "it's real now" bucket or in the "it seems very likely in the near future" one. 3 basic questions I am hoping to be educated on - if you have a strong POV, I'd love to learn from you: 1. Ambient listening: Given Nuance knows a thing or two about dictation and speech, and Msft about LLMs, what is the real angle for new players? Is it fewer hallucinations? More accuracy? More actionable guidance based on proprietary medical knowledge? I am assuming everything needs to go back into the EHR so why is this not simply Epic + Msft wins? I get the stuff about elephants not dancing and such but still...? 2. Clinical Diagnoses: Is this for real? Will we really see a day that my historical and real time medical record, (not just the stuff I see on Apple Health) can be fed into a fine-tuned model with all the RAG, RLHF bells and whistles, and out will pop helpful guidance for medication or other meaningful interventions I should take? And for all the talk of "provider in the loop / AI is an assistant not the agent" talk, is there a real example of this capability saving time / $$ / creating access? (Outside of radiology - I get how images are a well solved problem) 3. Workflow integration: Related to 2. above, has anyone seen a real LLM use case integrated into nursing / clinician / other provider workflow in a useful way that doesn't make them use multiple screens, create new and often artificial alerts and triggers, and add burden as opposed to take it away? Above all, are hospitals and staff buying into / trusting any specific use cases? I'd love to see a case study or two of real ROI delivered with ecstatic users who would hate to lose the capability. This is revealing my ignorance, so please educate me on the reality and near term (1 - 2 year) potential, I get all the stuff about the long term promise. Thanks in advance!
Megan Eubanks
2mo
Hey Casey Bryson, just thought I'd tag you in Sanjeev's post. Maybe you two could chat about your thoughts about AI solutions that could make life easier for doctors. Could be interesting!
Sidhartha Sinha
2mo
We're building on the TPA/payer side (self-insured ecosystem), focused primarily on #3 above--there's similarities to the workflows you called out but the end user/beneficiary is different. We're early so no ROI to show yet, but two value props where we're seeing traction among potential customers: Most of our ICPs are already using 3-5 different programs for their workflows/operations (alternatively, they're sitting on a legacy architecture...think Access dbs with apps, COBOL, etc.). The time that it takes an end user to synthesize the information and provide a response results in lower operating margins and directly limits the number of customers that can be served. So what we're learning is that the tradeoff of one more program (a web app) is worth it if the right info can be surfaced quicker, minimizing the need to access multiple other sources. Workflows in this category have become the ideal wedge feature. These orgs. take weeks to months to transform data into a usable/unified format for workflows. Leveraging a RAG approach significantly reduces that burden, but it creates the challenge we're working to address now--incorporating feedback from our end users to train the disparate structured and unstructured data sets.
On #2 - the puzzle here is, I think, in how hospital data lakes can get all unstructured data structured, and made available at varied permission level, to provide context for LLMs, RAG, etc. One such effort is ongoing at Intermountain Health. See this outline of their HIMMS talk: https://www.linkedin.com/posts/andrei-radulescu-banu_himss24-ai-automation-activity-7174165916589420544-ud6X Intermountain also gave a follow-up talk at the JohnSnowLabs conference a month ago, describing their work with Databricks to create ready-available structured data in their data lake. My guess is, other hospitals need same; this is a great opportunity for the right software tools vendors. Thus, I would not take LLMs and RAG in isolation, but the whole back end infrastructure in the data lake that can enable LLMs. #3 I have less direct experience with workflows, but would bet #2 is a prerequisite to simplify/automate staff work, and ultimately automate workflow steps.
Patrick Stroh
2mo
What I always find interesting are the attempts and failures we don't hear about. That self censoring, for obvious reasons, is such a loss of information that could accelerate workable solutions.
Pooja Solanki
2mo
Have the same curiosity around the crowding of ambient technologies and also the hype of ROI in clinical workflow. Especially on 2 and 3 I am hearing most integrations are not flawless or nonexistent, with most work being additive to the workflow and “enhancing” the same workflow rather than truly taking anything away making the change management and adoption still highly variable.
Ruth Smith
2mo
Sanjeev Agrawal, your questions are similar to many I've heard from companies across healthcare (and other verticals). At Rōnin Consulting we have developed prior authorization letter automation using LLMs that is integrated into workflows. LLMs can assist with diagnosis but with a doctor in the loop. The workflow integration piece holds the most solvable puzzle piece now. I'd love to schedule a time to show you what we have developed and the actual use cases where it is being used. I'll send you a DM.
My 2 cents on #1 - Nuance, Microsoft may be great, but there could be opportunity in smaller deployments that Nuance or Microsoft are not looking into. What these smaller, niche deployments are - I don't know. What I do know, though, is that good solutions tend to come from startups up, rather than from big players down. But startups need early adopters, to iterate over the product. You said it - elephants not dancing.
Jared Pelo
2mo
Sanjeev Agrawal I’d be happy to discuss where I believe we are today and where we will be in a few years. In short, I agree with point 1 that Epic/MSFT ultimately win. Question 2 it’s coming but not as quickly as it should with it being a messy and siloed data problem. Question 3 the truly integrated efficiency is dictated by the EHR and their willingness for integration until we unlock the data from question 2.
John Edwards
2mo
Extremely insightful questions. It's crucial to distinguish between current capabilities and future possibilities in the LLMs healthcare landscape.
CEO & Co-Founder at Kythera Labs
2moSanjeev Agrawal let's connect soon and talk about some ideas