New tool for early identification of COVID-19 surges

Kaiser Permanente develops method to help hospitals, communities identify early signs of community spread

Health systems could use data from routine clinical care to identify the onset of upcoming COVID-19 surges as many as 6 weeks before they occur, Kaiser Permanente researchers found in a study published July 26 in the journal BMJ Open.

Even though the impact on hospitals of COVID-19 cases has subsided in Northern California with successful vaccination campaigns, there are concerns that the virus and hospitalizations may surge again locally as they are elsewhere in the world, says lead author Vincent Liu, MD, MS, a research scientist with the Kaiser Permanente Division of Research in Northern California.

“Over the course of 2020, COVID-19 surprised us at nearly every turn, making longer-term predictions of its impact on our patients, health system, and communities extremely challenging,” says Dr. Liu, who is also a practicing intensivist with The Permanente Medical Group and regional director of hospital advanced analytics for Kaiser Permanente in Northern California.

“At the same time, shorter-term predictions — looking only 1 to 3 weeks out — left little time to respond adequately,” he says. “By examining hundreds of millions of data points across our entire health system, we have now identified key leading indicators that forecast impending COVID-19 surges as much as 6 weeks before they hit our medical centers.”

The team tested dozens of different data elements from the electronic health record system and data used by Kaiser Permanente in Northern California to find any elements that might be indicative of an impending surge in COVID-19 admissions. To determine what data points might be leading indicators, they looked at seasonal surges of influenza as far back as 2015.

Ultimately, the investigators chose 10 indicators and determined how many weeks of lead time they could provide health system and community officials. While individual indicators could forecast an upcoming surge in the next 1 to 3 weeks, the combined CHOTS — the “COVID-19 HotSpotting Score” — significantly increased the lead time to as far as 6 weeks prior to a surge.

CHOTS includes 4 major variables — cough and cold calls, relevant subject headers from patient emails, positive COVID-19 test rates, and current COVID-19 hospital census — along with 6 minor variables, including COVID-19-related calls to the call center, both routine and urgent clinic visits for respiratory infections, clinic and urgent clinic COVID-19 visits, and respiratory virus tests ordered.

Public health officials and individual health systems could use the information to boost local testing, expand contact tracing, adjust policy measures, and prepare expanded hospital space and supplies for anticipated patients. The authors found early indicators of waning COVID-19 activity were also valuable because they could help identify when relief from the surge of COVID-19 was on the horizon.

“The development of CHOTS is an inspiring example of how Kaiser Permanente’s integrated care delivery system and robust research capabilities can come together to provide the greater community with vital public health tools,” says co-author Stephen Parodi, MD, national infectious disease leader for Kaiser Permanente. “What is even more impressive is that this work was carried out while our medical centers were caring for large numbers of COVID-19 patients. Our researchers worked alongside operational leaders to better prepare our management of future surges.”

Developers chose simplicity

Because of the urgency to develop an early warning system for the COVID-19 hospital surge in May 2020, the authors chose to keep CHOTS relatively simple. They wanted to ensure the tool was available in real time by June 2020 and could be duplicated by other health care organizations. The authors are making the CHOTS methodology freely available for others to use.

“We use machine learning and artificial intelligence every day in our research group to develop predictive models to improve patient care. We applied these tools when we were developing CHOTS but didn’t find that they improved the tool’s value,” says co-author Patricia Kipnis, PhD, principal statistician at the Division of Research. “Our research group focuses on pairing the right algorithm with the right use case and, in this case, a simpler tool showed excellent performance and could be readily implemented and shared.”

CHOTS went live in June 2020 and the team evaluated it against actual COVID-19 hospital activity through December 31, 2020, throughout Northern California, and for each of 20 Kaiser Permanente hospitals in the region. The correlation of the regional CHOTS with hospital census was very strong, peaking with a 28- to 35-day lead time, but with continued correlation when tested out to 6 weeks. Because the impact of COVID-19 varied tremendously within individual medical centers over time, the team also evaluated CHOTS at each hospital and found strong, leading correlation at most sites.

To understand whether the CHOTS was better than individual leading indicators, the investigators compared the performance of each indicator with COVID-19 census. Current hospital census and new positive tests were the strongest indicators of census in the following 1 to 2 weeks, while call center and email volumes were strong indicators of census in the following 2 to 3 weeks. However, the signal from the individual indicators diminished over time and the CHOTS proved to be the most indicative of future activity with the most lead time.

The authors note a limitation to CHOTS: It relies on comprehensive data from a robust electronic health record system, which may not be available to health systems in some locations and those without regional call centers. A simplified tool, without these indicators, also show strong, but modestly lower, correlation.

The study was funded by The Permanente Medical Group, Kaiser Foundation Hospitals, and the National Institute of General Medical Sciences.

Co-authors also included Gabriel Escobar, MD, Laura Myers, MD, Lawrence Gerstley, PhD, and Khanh K. Thai, MS, of the Division of Research; and Yi-Fen “Irene” Chen, MD, Nancy Goler, MD, and Jessica Galin, MPH, of The Permanente Medical Group.

This is reprinted from the AboutKP site.