Finding Early Warning Signals in Public Health Data
Before an issue occurs, reporting data might show early warning signals. These should be noticeable, like a low battery alert on your smartphone.
Measuring the distance between the statistical characteristics of historical and newly reported data provides a practical way to detect early warning signals.
To explore this idea, consider the following example.
Propose the Data
To detect early unusual hospital workload in the region, the Public Health Expert proposed patient enrollment data. Can you detect the signal in the newly reported data? Put your attention on extremely small samples.
Figure 1 Patient enrollment data.
Explore the Data
The raw time series are hardly usable for signal detection. For this example, the following statistical features have been selected as the signal carriers, while the Standardized Euclidean distance is the similarity measurement.
Determine signal level detection.
The signal level detection is limited by the variability coefficient of the historical data.
VC=Standard Deviation/Mean
Build and run the model
Get Alex Kurasoff’s stories in your inbox
Join Medium for free to get updates from this writer.
When newly reported data differ from historically grounded expectations, it signals the need for further investigation. In statistics, historical grounded expectations are a set of plausible new data simulated using a statistical model based on historical data.
The Data Scientist chose the ARIMA(1,1,0) statistical model, trained it on historical data.
Δyₜ = 184.713494 − 0.326575 × Δyₜ₋₁ + εₜ
To establish historically grounded expectations, the model simulates 10,000 outcomes using the Monte Carlo method because εₜ is a normally distributed random variable. Such approaches are widely used for quantitative risk analysis and forecasting.
Detect and measure the signal.
A signal is measured by how unusual the historical or newly updated data appear compared with the modeled outcomes. The measurement is a percentile. For example, the percentile of 74.2 means that the newly reported data is more unusual than 74.2% of the modeled outcomes, while 25.8% of the modeled outcomes are even more unusual.
Comparison of newly updated data with the historically grounded expectations allows detection and measurement of the signal.
Comparing observed historical data with the modeled historical data allows assessment of model adequacy. It is vital for small historical samples.
Investigate Cause
The cause investigation is outside the scope of signal detection. Based on experience, over 80% of signals are caused by dirty data or reporting delays. Consider these first.
Conclusion
The usage of distance measurement is very practical for signal detection because it provides data that is accessible at the national and hospital level, without relying on international databases that may be limited at any time.
This method has been effectively used for nearly a century in quality control and risk management within industry. The use of Monte Carlo estimations is common in risk management and forecasting.
The implementation of this approach cannot be a one-size-fits-all solution. It requires the hard mutual efforts of public health experts and data scientists. Indeed, the reliability of the signal detection depends not only on performing the calculations correctly but also on recognizing and evaluating the assumptions on which those calculations are based. The good news is that, with AI assistance, implementation typically takes one to two months.
