Ron Doering and I disagree about lots of things, but we apparently agree on this: ranking countries by food safety can be a mildly amusing parlor game, but is meaningless.

Doering, a past president of the Canadian Food Inspection Agency, writes in his Food Law column:

How safe is our food? What is the economic cost of foodborne illness? How does Canada’s food safety performance compare to other countries? In spite of what you may have read recently, we don’t have clear answers to any of these questions, nor will we anytime soon. Not everything that counts can be counted, as they say, but that doesn’t stop the economists from trying.

Estimating the incidence of foodborne illness is back in the news again because the U.S. Centers for Disease Control (CDC) recently announced that due to “new data and improved methodology,” it is revising its longstanding estimate of the incidence of foodborne illness from 76 million to 48 million annual episodes. Not to be outdone, the Public Health Agency of Canada (PHAC) is now working on changes to its methodology, and we are told already by the Canadian Food Inspection Agency (CFIA) that the longstanding estimate of foodborne illness in Canada “will likely be reduced.” To get safer food, you just have to change your methodology.

Quick off the mark, the Conference Board of Canada adopted part of the new CDC methodology, made a number of assumptions and, voila, we are told that Canada has only half the annual cases of foodborne illness that PHAC has estimated and widely reported for well over a decade.

The Conference Board estimate is still based on the old data from three telephone surveys (one of which is from over a decade ago) in which Canadians were asked if they had a recent episode of acute gastrointestinal illness and then a number of debatable assumptions are made to determine which of these may have come from food.

Notoriously unreliable telephone surveys for these kinds of matters are used because actual data based on authenticated reports are even more unreliable as a true indicator of the incidence of foodborne illness. The most common symptoms of foodborne illness include stomach cramps, nausea, vomiting, diarrhea and fever, and because these symptoms also resemble hangovers, stomach flu and many other ailments, most cases of foodborne illness go unreported. Only the serious cases or large outbreaks of illness are investigated. To be included in Canada’s national statistics, a person must be infected, become ill enough to consult a doctor, and then be sent for lab tests.

The lab test must identify the illness-causing bacteria, recognize it as foodborne, be reported to the local health department (most are unreportable) and then these results must be sent to PHAC.

Estimates among epidemiologists vary widely on the percentage of actual cases that are reported, but there is a clear consensus that foodborne illness is seriously underreported.

The World Health Organization has estimated “that the reported incidence of foodborne diseases represents less than 10 per cent, or maybe even less than one per cent, of the real incidence.” Surveys in a few countries estimate that foodborne illness might be 300 to 350 times more frequent than the reported cases suggest.

What is the economic cost? With the deep uncertainty on the basic matter of incidence, and with the many other obstacles to meaningful measurement, only a few hardy souls have tried to estimate the economic costs. To the Conference Board’s credit, on this issue at least, it admits that “no credible estimate of the costs could be attained…with the existing paucity and variance in data, we do not believe that existing attempts to measure the economic impact are reliable.”

Can we meaningfully compare Canada’s food safety performance to that of other countries? No, but that hasn’t stopped academics from trying and the media reporting on their “scientific” results.