Want to build a readmissions model? Research papers are a great reference, but for real world applications, there are a few questions to consider before diving into replicating one of these research projects.
So, you want to build a readmissions model? Well, you’re not alone.
According to the Agency for Healthcare Research and Quality (AHRQ), 30-day readmissions are associated with $41.3B in hospital costs per year. As a result, many organizations are researching the best predictor of future readmissions and publishing their findings. These research papers are a great reference and comparison tool, but for real world applications, there are a few questions to consider before diving into replicating one of these research projects.
It all starts with building a model that can actually be used in your organization. Too often, data scientists replicate published papers only to find out these “ideal” machine learning models can’t be deployed to production. Academic papers don’t take into account the data lag that exists in the real-world and/or your specific organization’s needs. Rather than spending time and resources building a model that is unusable by your company, wouldn’t it be great to build a model that can be deployed and has an impact beginning day 1?
At ClosedLoop, we’ve developed a framework for data scientists and clinical teams to follow when creating any new predictive model. Before building a readmissions model, ask yourself these 5 questions:
Let’s walk through each of these five questions. To help illustrate how healthcare organizations would approach these, we’ve created a made up group called ABC Health. Although the organization is fictional, the situations are very real and highlight the complexities of real-world healthcare predictive models.
Before you jump to exact details, think about the big picture for a moment. Brainstorm how you’re going to use these predictions. Does your organization have interventions in place for patients that are deemed “high-risk” for readmissions? Will the patients be assigned a dedicated nurse while they are in the hospital? Will a follow up appointment be scheduled before they are even discharged? Is tele-monitoring technology placed in the homes of “high-risk” patients?
An intervention for at-risk members might not be in place today, which if fine, but if you find yourself reading this article, you most likely have some intended use for the predictions. Think about what those are. Our fictional organization, ABC Health, uses home-visits as an intervention for patients that are labeled “high-risk” for a readmission.
The intervention in place at your organization will naturally give a baseline for your new model to beat. Ask yourself, how are patients identified for these programs today? Is it based off certain parameters the patient needs to meet? Does your organization use LACE scores or another type of rules-based scoring system? Beating the existing baseline helps to demonstrate ROI and the impact your readmissions model will have.
If the lack of an intervention reveals there isn’t a formal baseline, think for a moment what informal rules could be applied to identify at risk patients. Would a clinician manually have to go through charts to find “high-risk” patients? Would it be based on intuition when a patient is being discharged?
At ABC Health, patients are characterized as high-risk if they’ve had 3 or more admissions in the last six months. This means our new model needs to be better at predicting than our existing rules-based approach.
I know what you’re thinking, “This one is easy, skip”? Think again!
Most readmissions research papers predict the likelihood of someone getting admitted within 30 days of another inpatient admission, but what is considered to be a readmission? Is it a readmission if the hospital stay was planned or unrelated to the original admission? How much does it cost your organization if the readmission is after 31 days of discharge? Do you want to account for any adverse events (mortality, health-acquired infections, etc.) in that time frame? Think of your specific organization when answering this question. Odds are, what you specifically want to predict will differ from the research papers online.
ABC Health is held contractually responsible for all readmissions within 45 day of another inpatient admission, except for admits related to pregnancy or cancer. ABC Health needs to predict the likelihood of someone having an unplanned readmission or death within 45 days of another inpatient admissions, ignoring pregnancy and cancer-related admissions.
Does every single patient need a prediction for readmissions? What about patients who had a discharge disposition of Acute Care Facility or Hospice? To answer this question, think back at what you actually want to predict.
ABC Health only applies the In-Home Nurse Visit Intervention to patients who were discharged at Home. Meaning, the patients discharged at Home are the only ones that will receive a risk score for readmissions.
This question is actually two questions in one, just think of it as a bonus. First, at what time do you want to make the prediction? Most papers predict readmissions at discharge, but there are a number of different options. Think back to the actual intervention. If the intervention is scheduling a follow up appointment as they check out of the hospital, the time of prediction is made at discharge. If the intervention is to provide additional care during the patients stay, the time of prediction is right at admission.
ABC Health nurses call high-risk patients to set an in-home appointment after the patient is discharged. If the nurse is unable to reach a patient, another call is made 2 days later. This means we need to make readmission predictions starting at discharge and every day until 45 days since the initial admission.
The second question is, what data sources do you have on hand at the specific time the prediction is made? Be thorough when answering this question and don’t be afraid to ask your data or IT team for help. Does your organization have an ADT (Admission Discharge Transfer) feed? Which claims are available at the time of prediction? Are those claims timely, or are they delayed by 90 days or more? How much history do you have on the patient? Do you have access to the EMR data from the inpatient stay?
At the time the prediction is made, at discharge, ABC Health has prescription claims, near real-time ADT feeds, and CMS blue-button patient history, but they do not have access to the EMR data from the hospital stay. Also, their medical claims have a 3-month delay meaning all the medical claims they have for a patient would be from a previous admission – if there was one.
By answering these five questions, we’re able to define our model in much more detail than just predicting readmissions. Let’s see how our model has evolved.
Initially: I want to predict readmissions at ABC Health
After answering the 5 questions: I want to optimize the in-home nurse visit intervention by predicting who will either get readmitted or have an adverse reaction like death within 45 days of discharge to home, but ignoring admissions related to pregnancy or cancer. These predictions will be made every day starting when the patient is discharged until 45 days after the initial admission. The machine learning model will be powered from prescription claims, ADT feeds, CMS blue-button data, and 3-month delayed medical claims
This framework is extremely effective and valuable for creating an AI model that provides a real return on investment because you’ve defined the model with your specific organization in mind. Give it a try and let us know how it goes!
ClosedLoop’s AI & data science platform is built to get you thinking about these questions early on. Our HIPAA compliant platform enables data scientists to quickly build, train, and deploy machine learning models with tools like data normalization, healthcare ontology mapping, automated feature engineering, explainable predictions, and version control. Feel free to review our platform overview for more built-in features.
Not a data scientist? No problem! ClosedLoop has the industry’s best collection of predictive models, and our catalog of off-the-shelf models can be customized and extended based on your local population. Sign up for a live demo to speak with one of our data experts and walk through the platform.
Interested in learning more about building models for Healthcare? Check out these related posts:
CONTINUE READING
Artificial intelligence (AI) and machine learning (ML) are increasingly used in healthcare to combat unsustainable spending and produce better outcome...
13 min read
Implementing a comprehensive strategy to advance health equity is a moral and financial imperative for healthcare organizations (HCOs). Persistent...
12 min read
COVID-19 simultaneously exacerbated existing health disparities and introduced entirely new ones. The pandemic disproportionately impacted people of c...
11 min read
Get in touch today to see the ClosedLoop platform in action.
