Covid19 “Science” from CMU, Squeee!

The Delphi Group at Carnegie Mellon University recently released data on mask-wearing in the United States via a project called a COVIDMap and public API. This is a valuable contribution to the understanding of Covid19 models; actual mask wearing data has been difficult to find, and there is much speculation about compliance. Indeed overlaying mask wearing with case and death data might be insightful as to how effective masks are, and over what timeframes and geographies.

Ignoring the quality issues with opt-in self reporting from a fb feed prompt and a fairly small sample set, the most significant thing this data shows is that as of now, mask wearing when in public in the United States is consistently above 90%, and often 95%. This has been true for about a month. As of 12 weeks ago, the range was 60% to 90%.

Unfortunately 2 ½ months of data is not nearly enough to observe correlations between mitigations and results. Almost any 10 week window in 2020 would have led to grossly incorrect results.

That didn’t stop The Delphi Group at Carnegie Mellon from doing exactly this, in the form of a map infographic that shows current mask wearing compliance, and new “cases” (i.e. PCR-RT positives),  a scatter plot showing the same thing, and text calling out North and South Dakota as example of bad mask compliance leading to higher “cases”.

There are few things to note about this content:

The difference in compliance rates are a few percentage points, indicating that mask wearing is a highly sensitive variable. It wasn’t in my models (with estimated compliance numbers), and I can find no studies indicating that anyone believes it is. That said, there could be an as yet undiscovered threshold that produces non-linear results, so we should remain curious.

If you choose just about any other 12 week window to compare cases or deaths with mask wearing in The Dakotas, you would observe the exact opposite outcome in the data – not wearing masks keeps a population safe from Covid19. What is likely going on here is that 12 week window chosen – because they had data – happens to coincide with the geographic movement of the virus. Indeed, I believe most assertions of correlations between mitigations and outcomes are random based on where the data window falls on the geographic progression of the virus.

The usual disclaimer is in the University’s text: “Correlation does not imply causation”, although this point seems to be lost quickly in the wave of press referencing the data from the project.

Now what would solve the last issue would be a comparison of two demographics, carefully selected to have highly similar potentially dominant variables but differing in mask compliance, rather like DANMASK-19 did. That study concluded that masks didn’t in practice work. I think the same meta-study will be possible with the Delphi Group’s data, given 2 or 3 more months of collection.

With the caveats aside, let’s look at the actual data. This graph shows daily new deaths due to Covid19 and daily mask wearing in North Dakota during the window we have data.

From this data, assuming any correlation at all exists between masks and deaths (a statistically unsound assumption), you would conclude that masks cause deaths, or Covid19 causes masks.

If we shift the mask wearing series back 3 weeks to account for effect, a common adjustment these days when forecasting high death counts from high PCR-RT positives, the correlation is more pronounced.

Of course, the informed data scientist understands that this data shows us nothing.

Nothing at all.

Google recently donated $1,000,000 to this project. One wonders if a project showing that we know nothing at all about the effectiveness of masks would be equally valuable.


This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s