In this week’s #SkepticalTuesdays post, I take a look at a recent study of the microbiome of the NYC subway, and urge calm on the populous while trying not to type the words “anthrax” and “plague” … aw, crap.
***
Basically everybody was reporting on a new study out last week by the folks behind PathoMap, a project undertaken by the Weill Cornell Medical College with the aim of studying “the microbiome and metagenome of the built environment” of New York City. The coverage was overall pretty good, but everybody had to have their little bit of eye-catching text, like the claim that “67 types of disease-causing bacteria, bubonic plague and anthrax among them” were found, or the words “associated with” that show up all over this (actually pretty sweet) Wall Street Journal interactive map and “tie” the bacteria found to everything from the insides of our intestines to making mozzarella. Sarah Zhang over at Gizmodo actually did a pretty good job at reporting this, so hats off to her for pointing out a lot of good stuff and not going for the “OMG Anthrax” angle. Refreshing — check it out.
But you’re not here for links, you’re here because you like things explained with a skeptical bent. So here we go:
The study, “Geospatial Resolution of Human and Bacterial Diversity with City-Scale Metagenomics” is just out in the journal Cell Systems. Their team swabbed every open subway station in the city, as well as one closed station (the South Ferry station in lower Manhattan, closed since hurricane Sandy swamped it), twelve sample sites along the Gowanus Canal, four public parks, and the Staten Island Railway. They geotagged and photographed all the sample sites, and then used some fairly new technology to sequence all the samples.
The results were interesting. Almost half (48%) of the samples didn’t correspond to what the study calls “any known organism,” and they did find what looked like plague and anthrax, as well as what looked like other pathogens. They also found a whole lot of everything else, and teasing out associations — like the mozzarella one (lactococcus lactis, Wisconsin’s state bacterium!) — isn’t always simple. Let’s have a look at some of the most important text not reported on from the Cell Systems article:
“…although the evidence is strong that these organisms were detected based on the current databases, it is always possible that improved bacterial annotations and newly completed genomes can move the ‘‘best-hit’’ evidence to a different species in the Yersinia or Bacillus genera, or a different genus altogether.” [emphasis mine]
Basically, the databases they were comparing their data with aren’t complete. They don’t even have the genome for the cockroach yet, which you really have to expect is going to be found in the other 48% when they do. Add to that the facts that we haven’t been doing this sort of thing for very long and that to make up for it they’re using a best fit model for matching the samples to known species, and you get a not insignificant chance that the results aren’t exactly concrete. Like maybe it wasn’t the badass anthrax at all, but rather anthrax’s second cousin (you know, the one with the weird multi-level marketing scheme that nobody wants to talk to at the family reunion) who’s actually pretty harmless.
So when New York’s health department calls the study “deeply flawed” and the MTA seems perturbed, it’s probably not because of things the study’s said, but rather because of the way the study’s been represented to them. Even the caveats that made it to the media — like the fact that, even if they were plague and anthrax, they were most certainly dead and there haven’t been any cases in a long time — don’t state the simple truth of it: this is the first time anyone’s done this, and it’s a far cry from perfect.
And caveats are a little beside the point when the WSJ is saying staphylococcus aureus is “associated with” staph infections that “can cause skin infections, sinusitis and food poisoning.” Guys, staph lives on your skin. Normally. And if that’s how readers react to that news, imagine how much gets lost when you start throwing around the words “anthrax” and “plague.”
So kudos to Scientific American for giving the researchers a rebuttal (“Most of the bacteria identified are types that placidly thrive on our skin and are of no concern to [primary investigator Christopher] Mason. “If anything, I’ve become much more confident riding the subway,” he says.”) and focusing on the weirdness that is South Ferry station (after Sandy its microbiome has become totally different from the rest of the system). Congratulations, too, to the folks at PathoMap for putting something so interesting together.
But let’s not pretend that this is anything more than a beginning, a proof of concept for an exciting new methodology that will probably reveal impressive results in the future.
So don’t panic, your subways are healthy, probably not full of alien life, and maybe don’t even have anthrax in them. Or maybe they do.
As for me, I’ll be over here waiting for the cockroach genome.