‘The devil is in the details’ interpreting raw data from COVID-19 statistics

Those who are looking closely at the numbers related to local COVID-19 cases may wonder why it seems that the reports from different sources don’t seem to add up. Numbers reported by Kingston, Frontenac, Lennox and Addington (KFL&A) Public Health often indicate hospitalization numbers that differ from those reported by the Kingston Health Sciences Centre (KHSC).

The numbers alone don’t give the whole picture, said Dr. Gerald Evans, Chair of the Division of Infectious Diseases and Professor with Queen’s Department of Medicine, and an infectious disease specialist with KHSC. “Some people are going to say that if the numbers are not the same, that must mean that someone’s not telling the truth, but that’s not the case. If you’re an accountant, yes, you want there to be a debit line and a credit line that balance each other out perfectly, but that’s just not how this operates, or how it can be used in the most informative and helpful way.”

In an interview, Dr. Evans noted that he’s found an increase in public interest in the hospitalization numbers and what they indicate.

“I do a lot of tracking of numbers and data, and earlier on in the pandemic, I saw those different numbers, too, but not too many people were talking about that then, because our collective focus was a bit less on hospitalizations and more just on numbers of cases. But now, everyone’s looking at hospitalizations because of Omicron,” he explained.

“What we’re seeing is that cases are rising very, very quickly, but that hospitalizations are not jumping at the same level. So, people are now looking particularly at hospitalization rates to try and gauge the seriousness, saying ‘can I track this myself, can I see if hospitalizations are going up or down, and calculate some of the risk?'”

However, Dr. Evans cautioned, there can be a misleading sense of clarity in leaning on numbers without knowing how those numbers were determined. “The expression that should be part of everyone’s approach to understanding this is ‘the devil is in the details.’ There’s a high-level view that gets reported in any set of data, but when you get down into it, you need to sort through how those numbers were generated, what’s the nuance, what do they actually mean.”

Dr. Evans explained that Public Health reporting locally gives numbers on any cases from individuals who live within the KFL&A Public Health region, even if they are hospitalized outside of the region. “If, for some reason, they got really sick while they were in another area, say they’re in hospital in Belleville, but their home address is in Kingston, they’ll still be on the KFL&A numbers. KFL&A would say that’s a KFL&A patient, but KHSC would not.”

Conversely, the KFL&A numbers do not include individuals receiving care at local hospitals who are deemed to be from outside the KFL&A Public Health region.

“In this region, with KGH being a tertiary hospital, we may have people from all over,” he noted, referring to Kingston General Hospital (KGH).

“So, we would see KHSC hospitalization numbers reflecting any cases whose home address is outside the KFL&A region, but those [patients] would not be showing up in the KFL&A reporting,” said Dr. Evans.

Major hospitals designated as tertiary care facilities provide specialized healthcare, inpatient care, and serve as academic and teaching facilities. Because of the scope of community care tertiary hospitals provide, it is common for them to treat individuals from outside of their own Public Health region.

Additionally, KHSC numbers only include hospitalizations where the individuals are still positive for COVID-19, while KFL&A Public Health still includes the cases in their reporting as long as the individuals are still hospitalized with complications, but are no longer COVID-19-postive.

“What KFL&A does is look at the address and see that’s it’s someone in our Public Health region, and once they’re in hospital, they keep them on their list until they’re discharged,” Dr. Evans explained. “So, for example, let’s say there could be someone who got COVID-19 in October and hasn’t been a positive case, infection-wise, for a long time, but they’re still in hospital because they’ve suffered some longer-term consequences. They’d still be on the KFL&A numbers, but KHSC would have cleared them from our numbers.”

Dr. Evans acknowledged that it’s challenging to determine how to designate a case as being resolved, and that it is helpful to keep both sets of numbers in mind accordingly. “Some of the people in ICU could be not actively sick with COVID-19, but still on a ventilator because of the longer effects of an earlier infection. We have people in different stages of the illness. We may call them ‘recovered’, but are they recovered? This is where the perspective can be entirely different.”

Provincial reporting on ICU hospitalizations related to COVID-19, he noted, is based on individuals in ICU that are currently positive for the infection, and not on continued ICU stays from the after-effects of especially debilitating cases.

On an even broader level, Dr. Evans noted that numbers reported from international sources are also going to reflect what their own region has determined is the best method for reporting.

“In England, during the onset of the Delta variant, the British government was publishing data on case numbers and hospitalization rates, and we were all looking at this saying, ‘this is awful, Delta is awful.’ And we learned that their reporting is approached differently than ours is. In England, anyone who comes to the emergency department is counted as a hospital admission, which is not the same here,” Dr. Evans explained.

“Also, they were testing everyone for COVID-19, regardless of what they came to the hospital for. So, when someone who, say, broke their ankle, came in for treatment, and in the process was also tested for COVID-19 and the test came back positive, they were counted as a hospital admission for COVID-19, even though their actual COVID-19 symptoms themselves would not have brought them to the hospital. But for their reporting methods, this was accurate and was well-understood in their context; it’s just very different from how it would be approached here.”

“It’s a global transparency thing, and we need to keep in mind what each reporting source is being transparent about,” he added. “Everything is about perspective, and it depends what lens you’re looking through, and being aware of the lens that the reporting source was using, as well.”

Dr. Evans was cautiously optimistic about the current status of KHSC facilities. “So far, we’ve been very fortunate; we’ve had no major issues with availability of beds or staffing, but of course that can change so quickly. At the moment, we’re okay as we come up to the new year, but we’ll have to see what the early part of January is going to look like to get a sense of how that mid-winter period is going to go.”