The State Of California: Moving Forward From The State's Data Meltdown

BELO HORIZONTE, BRAZIL - MARCH 24: Anna Raquel Ribeiro dos Santos, works in the standardization of a rapid test for the diagnosis of the disease on March 24, 2020 in Belo Horizonte, Brazil.
Photo credit Pedro Vilela/Getty Images

California’s public health director has resigned under fire, days after a series of mistakes led to an undercount of cases of COVID-19.

Gov. Gavin Newsom has split that job among two interim directors, and Monday he accepted full responsibility for the data meltdown. How could something like this happen? What does it say about California’s overall management of the coronavirus pandemic? How trustworthy will the state’s COVID-19 data be going forward?

To explore those questions and more, we were joined by Epidemiologist Brad Pollock on KCBS Radio’s "The State Of California." He’s chairman of the Department of Public Health Sciences and Associate Dean of Public Health at the UC Davis School of Medicine.

How colossal a failure was this, in terms of how it hamstrung the state, and the counties that depend on this information? Was it as bad as it sounds, or are we making too big a deal out of this data meltdown?

It’s pretty serious. I think a lot of decisions are made at the country level that depend on having reliable data about new cases. We were having erroneous data, missing data for some period of time. I’m not sure how long it went on, probably not a super long period of time. Unfortunately, the data are very critical for managing what we do here, including contact tracing and trying to get the pandemic under control. I think that a large extent, it’s important to point out that the CalREDIE system is quite old and it’s in need of infrastructure development quite a long time ago, before the governor took office. It’s been recognized as kind of being a weak link. This is a function of not having up-to-date systems not maintained at the level they need to be maintained at and the extra load added by this pandemic, which has certainly taxed the system.

How concerned should we be about how dependent we are on technology?

In general, we have so much more capacity over the last 10 years with our electronic systems. Ten years ago, there were many more physicians and health organizations that were not using electronic health records. I think they’ve obviously revolutionized how we manage our information, how patients are treated, the timeliness of corporate reporting. So we’ve become a lot more dependent on electronics, on these large data sources for running our normal health operations. They’re very, very important. You had something like this pandemic, which has stressed out some of these public reporting systems like CalREDIE in particular. It’s a function of needing to invest in our future to be able to leverage those technologies.

The governor insists things are improving. Do we really know that’s the case?

I think hospitalizations data is actually more reliable than testing data. Partially, because testing was so erratic and limited in the beginning of the epidemic. As time as gone on, we’ve increased our testing capacity enormously. In the early parts of the epidemic to try and rely on the test positive cases to try and give you a clue what was going on was very difficult. Hospitalizations really are not subject to the same type of bias, the same force of disease in the population. That leads to a certain subset of the population needing to be hospitalized. The other metric we use is deaths. Mortality is a very good indicator. Of course, it’s still relatively rare. But again, mortality hospitalizations tend to affect the true shape of the epidemic. Of course, there is a lag between people getting infected and the outcome. They tend to be more reliable indicators in terms of the shape and the direction of the epidemic. I think the governor is right. When you see hospitalizations starting to come down and the death rate come down, that’s an indication that the epidemic is changing direction, hopefully toward the better. But the problem with the CalREDIE system and in general is subject to who gets tested, who gets offered tests. They have not been uniformly available across the state. There have been pockets where it’s very easy to get a test if you’re asymptomatic and other places where you just can’t do it. There isn’t enough capacity. That’s why the reliance on hospitalizations and deaths tends to be a little bit better reflection of the true incidence, that is the true force of new cases coming on board. Understanding that there is a lag time between when they’re infected and when they are hospitalized.

Is it possible the rate of positive cases is significantly higher than what we’re seeing?

It’s based on the number of tests that are performed, not on how many are out there in the general population. The thing is if you’re missing some of the test results, it’s the proportion of the people who are tested that have tested positive that makes up that rate. It’s also a relatively good metric to use to look at what’s going on with the epidemic. When the case positivity rate goes down, it doesn’t mean that there’s less cases in the population, it means there are less positive results among the people who got tested. So the ones that do get tested, if you see those rates coming down, that’s a good indication that the epidemic is sort of going in the right direction.

How disruptive is it for the state’s public health director to leave in the middle of a pandemic like this?

There are a lot of people that make up the California Department of Public Health and really good folks at all levels. The director is important, obviously, so it’s not great when you have to change horses in the middle of the race, but you understand there are so many very dedicate professionals that work in the agency. The show goes on. People will be working hard to ensure that we do things right and help the state. I think it’s not as critical, it’s important that it’s happening now. The timing couldn’t be worse. But in terms of the show going on, I think we’re going to be okay.