Data Snapshot: Are we overcounting Covid-19 deaths? No.

It's a fair question. A recent Washington Post column didn't look at the data. A few times during the pandemic, we have overcounted Covid deaths. Mostly, though, we undercount.

Our weekly Data Snapshot is curated and wrangled by Benjy Renton. In this case, we’re literally fact-checking the Washington Post—since nobody else bothered to. Usually our weekly Data Snapshot is a quick hit. This one is a deep dive which I hope people can dig into.

With the help of readers like you, we’ll be able to keep this going for a long time. Thanks for supporting Inside Medicine…

---

One of the earliest questions I had when Covid-19 erupted was whether this new fangled disease we were hearing about was actually causing an increase in the number of total deaths, regardless of cause. If it was not, (i.e., there was no all-cause “excess mortality”), that would have been great news. It would have meant that things were far less serious.

As we know, however, there have been well over 1.1 million more deaths since March of 2020 than would have otherwise been expected over that span, but for the Covid pandemic. Studying this has become the main focus of my academic work.

I’ll never forget the punch-in-the-gut I felt, the first time I saw the numbers coming out of New York and my own state of Massachusetts in the spring of 2020. Not only were there thousands more deaths than usual in just a month, we were probably undercounting Covid deaths. I wrote about it in the Washington Post back in 2020, and spoke about it on C-SPAN. While things have gotten better, we are still recording excess mortality in Massachusetts (and elsewhere), which tracks with rises in Covid.

All-cause excess deaths in Massachusetts, weekly, through January 1st, 2023. Data are >99% complete, and are provided by the Massachusetts Department of Health. Model by me and my colleagues.

Last week, a column in the Washington Post by Dr. Leana Wen claimed that things have changed. The column was entitled, “We are overcounting Covid deaths and hospitalizations.” That’s factually inaccurate—at least the first half of the sentence. I’ve told Dr. Wen this (who I maintain a friendly relationship with, even though we have disagreed on some major issues.) She was receptive. I shared with her that I believe the headline should be corrected by the Post.

I can’t say for sure whether we are overcounting hospitalizations. It’s certainly possible, though its not black-and-white. But I can say with confidence that we are not overcounting Covid deaths right now, including in Massachusetts, the state highlighted by the Post, because this is literally what I study.

In the most recent 4 weeks of data we have (through January 1, 2023*) Massachusetts has had 643 all-cause excess deaths (23 per day). That’s somewhat higher than the 428 Covid-19 deaths recorded during that period (15 per day). We’re not overcounting Covid deaths in the state that the Post chose to use as an example. If anything, we’re probably undercounting Covid deaths, as of late (though explaining that is more complicated).

In this week’s Data Snapshot, Inside Medicine’s Benjy Renton and I pulled a few lines of code from our massive mortality databases to make the following graph to show how we did this past fall compared to the last couple of fall seasons.

Are we overcounting Covid-19 deaths? When more Covid deaths are recorded than all-cause excess deaths, it’s likely—and that happened in Massachusetts in the fall of 2020 (left bars), for example. But that has not been the case recently (middle, right bars). Image: Benjy Renton for Inside Medicine.

If we were truly overcounting Covid deaths, an important piece of evidence to support that would be if Covid deaths (blue bars) were exceeding all-cause excess deaths (yellow bars). As you can see above, that has happened at some times during the pandemic (albeit not often). Interestingly, in the fall of 2020 (far left duo of bars), we apparently did overcount Covid-19 deaths here in Massachusetts. There were 593 more deaths from September 1–November 30, 2020 from all causes than there “should have been,” but for the pandemic. During that same period, there were 850 Covid-19 deaths recorded. That means that there were 257 people whose death certificates list Covid as the cause that, statistically speaking, would have happened around the same time anyway. This means that during that time, 30% of Covid deaths were possibly overcounts. Again, that has happened at various times and places during the pandemic. For most places, such situations—periods when officials recorded more Covid deaths than all-cause excess deaths—have been the exception, not the rule. It has happened at times in other states in the Northeast, although never in the South. Meanwhile, in both the fall of 2021 (middle duo of bars above) and 2022 (right duo of bars), there were fewer Covid-19 deaths than all-cause excess deaths. That makes it extremely difficult (if not impossible) to imagine that we are overcounting Covid deaths.

So, my problem with the Post column is that there was no evidence offered for a claim for which we have excellent contradictory data.

If the claim were true, by the way, it would have fairly important implications. Instead, the entire column hinges on a couple of infectious diseases doctors who have not looked at all-cause mortality data, making claims based on anecdote, without offering a single piece of data. One of the experts seems to think that because it’s possible for medical examiners to add Covid to a death certificate when reviewing a death, that this means it is happening so much that we’re overcounting. There’s no indication that is happening at any significant rate. And remember: if it were happening, what’s the first thing we’d see? More Covid deaths than all-cause excess deaths. Do we see that? Nope.

In fact the gap between deaths blamed on Covid-19 and all-cause excess deaths— evidence that we are probably undercounting Covid deaths—became a bit of a puzzle to my team in recent months, after White House officials asked us to look under the hood on this. In the fall of 2021, our most basic analysis would suggest that Covid-19 could only be responsible for 64% (1,058/1,652) of the excess mortality in adults over 50 in Massachusetts (we choose to look at older adults to factor out most of the increases from non-medical causes such as accidental overdoses; if you think about it carefully, doing that hurts our case, but it’s more honest in this situation). In the fall of 2022, Covid-19 deaths could only be responsible for 71% (970/1,358) of excess mortality.

This means two things:

1. There is no evidence that we are overcounting Covid deaths lately. If anything, we are probably undercounting Covid deaths.

2. Researchers (like us) need to explain this gap.

Many people think that the gap between Covid and all-cause excess mortality might be explained by non-Covid deaths due to deferred care. For example, people think that missed cancer screenings in 2020 that were never rescheduled might be haunting us now. Others think that Long Covid may be responsible.

We think both of those are unlikely explanations. Why? Because medical causes of death other than Covid (heart attacks, cancer, strokes, etc.,), conspicuously rise and fall with Covid waves.

More deaths than usual from all causes (“excess mortality”) have occurred throughout the pandemic (green lines). Covid deaths (purple lines) track tightly with that. In a few instances, Covid death counts are higher than excess mortality (purple line goes above the green line; e.g., see Northeast in parts of Spring of 2021 and Spring of 2022). Note: Covid deaths for recent months are not yet fully reported in the source CDC database. IR=incident rate, or deaths per 100,000 people. Image: Benjy Renton for Inside Medicine.

We can’t directly prove that heart attack, cancer death, and stroke rates that spike during Covid waves are really all Covid deaths (which again would be undercounted Covid deaths, not overcounted ones). However, other evidence we have uncovered has convinced my colleagues and me that Covid is likely behind most of them. (Sorry to be coy here, but I can’t completely scoop myself out of a forthcoming paper in a medical journal. Besides, that’s not the point of this piece.)

The point of this post is that anecdotal reports about people dying “with Covid” as opposed to “of Covid” are not reliable and they are not evidence-based.

That said, I understand why my fellow docs might be confused. Covid deaths look diverse these days. Some die of pneumonia (à la 2020 and 2021, before the vaccines); but most die of some other condition like a heart failure exacerbation, triggered by Covid. To physicians, these deaths look like heart failure deaths, not “old-school” Covid deaths. But had they not been infected, the heart condition would not have been instigated.

Even when I’m at work in the ER, I can’t tell whether someone who died while infected with SARS-CoV-2 died because of Covid or not, or to what extent (it’s not all-or-none). I simply have to look at the population-level data. That’s why I study excess mortality; it cuts through the subjectivity that comes with assigning causes of death.

Figuring out whether we are undercounting Covid deaths is challenging, but doable. (Again, my colleagues and I are on it.) Refuting the idea that we are currently overcounting Covid deaths, however, is not as hard. The number of all-cause excess deaths remain higher than Covid deaths—and these numbers rise and fall in tight parallel; it’s not credible to say that we are overcounting Covid deaths right now.

---

Thanks as always to Benjy Renton for helping make Data Snapshot happen.

*Mortality data are generously provided to me by the Massachusetts Department of Health. We are generally able to have data which are >99% within 1-2 weeks.