Measuring the old emperor’s clothes: Meta-audit of laboratory ISO accreditation inspections

Accreditation to ISO/IEC 17025 is required for EC official food control and veterinary laboratories by Regulation (EC) No. 882/2004.

emperor’s clothesMeasurements in hospital laboratories and clinics are increasingly accredited to ISO/IEC 15189. Both  of these management standards arose from command and control military standards for factory inspection during World War II. They rely on auditing of compliance and have not been validated internally as assessment bodies require of those they accredit. Neither have they been validated to criteria outside their own ideology such as the Cochrane principles of evidence-based medicine which might establish whether any benefit exceeds their cost.

We undertook a retrospective meta-audit over 14 years of internal and external laboratory audits  that checked compliance with ISO 17025 in a public health laboratory. Most  noncompliances arose solely from clauses in the standard and would not affect users. No effect was likely from 91% of these. Fewer than 1% of noncompliances were likely to have consequences for the validity of results or quality of service. The ISO system of compliance auditing has the performance characteristics of a poor screening test. It adds substantially to costs and generates more noise (false positives) than informative signal.

Ethical use of resources indicates that management standards should not be used unless proven to deliver the efficacy, effectiveness, and value required of modern healthcare interventions.

 Microbiology Open

Ian G. Wilson, Michael Smye, Ian J. C. Wallace

http://onlinelibrary.wiley.com/store/10.1002/mbo3.314/asset/mbo3314.pdf?v=1&t=ii7upas9&s=93258e79b42f69fe8e1bbe0f5d81d3ae56b42e23

Because third-party audits can suck: Journey should not be mentioned in food quality, it should be there

Journey is/was a terrible band.

I saw them open for the Stones in 1981 and they were booed off the stage, hence the Journey effect: surround yourself with incompetent others so you look better.

audit.checklistDespite the recent passing of legislation by the National People’s Congress of China in 2009, many food businesses in China have yet to implement a third party certified food safety management system (FSMS).

While the extent literature identifies a number of internal and external barriers and benefits, the extent to which these impact on the business is thought to be dependent upon how much progress the firm has made on its journey towards quality assurance and the environment within which the firm operates.

To test this proposition, the barriers and the benefits accrued from the implementation of a third party certified FSMS were explored by segregating the participating firms into three distinct groups; (i) those that have yet to implement a third party certified FSMS; (ii) those that were in the process of adopting a third party certified FSMS; and (iii) those that were already operating under a third party certified FSMS. Contrary to expectations, in what is a highly competitive market, those firms which were operating under a third party certified FSMS were more likely to question the benefits they had derived than those firms that were either in the process of adoption or had chosen not to adopt a third party certified FSMS.

Irrespective of the stage of adoption, the major constraint to the implementation of a third party certified FSMS was the need for the organisation to focus on more immediate issues and the lack of any strategic long-term planning.

 Barriers and benefits to the adoption of a third party certified food safety management system in the food processing sector in Shanghai, China

Food Control, Volume 62, April 2016, Pages 89–96

Jiang Qijun, Peter J. Batt

http://www.sciencedirect.com/science/article/pii/S0956713515302425

No testing, but an A+ on audits: Lenient sentences for ex-peanut officials in Salmonella outbreak

USA Today reports that two ex-officials of Peanut Corporation of America drew lenient sentences Thursday for their self-admitted roles in a Salmonella outbreak blamed for killing nine and sickening hundreds.

AIB.audit.eggsGeorgia U.S. District Court Judge W. Louis Sands sentenced Samuel Lightsey, 50, a former operations manager at the peanut firm’s Blakely, Ga. plant, to serve three years in prison. Daniel Kilgore, 46, another ex-manager at the plant, drew a six-year sentence from the judge.

Sands allowed them to remain free, pending voluntary surrender after the U.S. Bureau of Prisons designates the correctional facilities where they will serve their sentences.

Both reached plea agreements with prosecutors that limited their punishment when they pleaded guilty last year to conspiracy, mail and wire fraud, along with sale of misbranded and adulterated food.

They later testified as government witnesses during the 2014 federal trial that ended with criminal convictions of ex-Peanut corporation of America owner Stewart Parnell and two other former top executives.

The case stemmed from findings by the Food and Drug Administration and the Centers for Disease Control and Prevention that traced a national salmonella outbreak to the Parnell firm’s peanut roasting plant in Blakely. The incident sickened 714 people in 46 states and may have contributed to nine deaths, the CDC reported.

The illnesses erupted in January 2009 and prompted one of the largest food recalls in U.S. history.

Parnell, 61, was sentenced to a virtual life term — 28 years behind bars — on Sept. 21. His brother, Michael Parnell, received a 20-year term, and former quality control manager Mary Wilkerson, drew a five-year sentence.

Sands ordered the Parnell brothers to surrender immediately, denying defense arguments that they should be permitted to remain free on bond pending appeals of their convictions.

Government evidence presented at the trial established that Lightsey and Kilgore knowingly helped the top executives fabricate certificates of analysis in a scheme that falsely showed peanut butter from the Blakely plant was free of Salmonella and other pathogens. In fact, there had been no testing of the product, or tests had confirmed contamination, prosecutors showed.

New Hampshire audit finds state falling behind on food inspections, one facility went 15 years without review

Live free or die.

nh.live.free.or.dieAccording to yet another feature about restaurant inspections, nearly one in five of the New Hampshire’s highest-risk food establishments weren’t inspected at all in 2013 or 2014, despite a federal recommendation they be inspected several times a year.

During that two-year period, the 474 highest-risk establishments in New Hampshire – high-volume food processing plants and large restaurants and dining halls – went an average of 427 days since their last state inspection. One unnamed facility went uninspected for 5,270 days, nearly 15 years.

Those are just a few findings outlined in a recent state audit that reveals flaws in the Food Protection Section, a branch of the New Hampshire Department of Health and Human Services charged with preventing food illnesses and ensuring consumer safety.

State auditors found food inspectors lacked oversight of towns that run their own food protection programs, rarely inspected agricultural fairs or soup kitchens, and didn’t properly collect and manage fees, resulting in an estimated loss of $1.2 million to state coffers between 2013 and 2014. During the two-year audit period, officials found state inspections decreased and complaints rose.

The Department of Health and Human Services agreed with many of the audit’s recommendations, and said it is pursing a new public health accreditation and has plans to launch a database this fall that will shift some critical operations online and give the public electronic access to inspection results.

But the department also pointed out that in recent years the Legislature has been rolling back food safety regulations and the Food Protection Section, made up of 15 employees, has faced staff reductions.

Since 2008, the food inspection program, which oversees roughly 5,352 food establishments in the state, has lost a shellfish supervisor, a food inspector and a food emergency response specialist and inspector.

Australian audit being conducted into third party certifiers including halal and kosher organisations

The Islamic organisations paid to certify halal meat in Australia are being reviewed as part of an overall audit of food certifiers, which includes organic and kosher organisations.

halaldThe Senate hearing into food certification has been told that there are 22 different halal certifying organisations covering about 70 abattoirs which export to Islamic markets.

Greg Read, from the Federal Agriculture Department, told the hearing the latest review came on top of the regular six-monthly audit carried out on processing plants and a regular review of the Islamic organisations responsible for certification.

“(It is ) just to give us high levels of confidence that’s (the system is) accurate nationally, then the department will effectively do a higher level review over the top of all of those to ensure that we haven’t got any plants that are differing and thereby exposed to the international markets should there be a further review,” he said.

The inquiry, charged with looking at food labelling and third party certification, has attracted over 1,400 submissions, with many focussing on halal certification.

Opponents argue the certification fee is effectively a tax, while prior to the hearing a number of critics claimed the monies raised could be channelled to extremist organisations.

UK minister says cut food safety audits

George Eustice, DEFRA’s newly appointed Minister of State, told the British Meat Processors Association’s annual conference in London that  meat manufacturers – and food businesses at large – must be freed from the “burden” of audits.

Eustice argues the need for fewer audits was one of the key findings to come out of Professor Chris Elliott’s report into 2013 horsemeat scandal. By the end of 2015 the government hopes to have finalised a strategy that paves the way for fewer inspections from both retailers and government agencies.

audit.checklistBut could it backfire?

Consumer trust in food safety is at an all time low

Thanks to wholesale media coverage of some notable cases of food fraud and breaches of food safety, we are living in an age of major consumer scepticism. The public wants to see tighter regulations on the food industry and it’s easy to see how the call for reduced audits could be perceived as a step backwards, not forwards.

It’s a stance that’s difficult to begrudge. Little has been done to ratchet-up traceability and safety measures since horsegate.

One thing that most food businesses agree on is the need for food safety procedures to be more streamlined. Paper-based checks are easy to falsify, annoying to complete and time-consuming to review. Eustice appeared to recognise as much when he noted that a greater use of technology must be central to any plans to reduce the incidence of audits.

Any government strategy to remove unnecessary burdens from food businesses will be warmly received. But food businesses must remember their obligation of safety to the end consumer. Technology that offers cloud integration presents the opportunity for food businesses to share safety data with one another on an open platform, paving the way towards transparent food chains.

Inspections and audits are not synonymous with safety. Beyond sharing data amongst companies, share it with everyone – especially consumers.

Three years ago, a group of us came out with a paper we could all (mostly) agree with and got it published. The main points were:

  • food safety audits and inspections are a key component of the nation’s food safety system and their use will expand in the future, for both domestic and imported foodstuffs., but recent failures can be emotionally, physically and financially devastating to the victims and the businesses involved;
  • many outbreaks involve firms that have had their food production systems verified and received acceptable ratings from food safety auditors or government inspectors;
  • while inspectors and auditors play an active role in overseeing compliance, the burden for food safety lies primarily with food producers;
  • there are lots of limitations with audits and inspections, just like with restaurants inspections, but with an estimated 48 million sick each year in the U.S., the question should be, how best to improve food safety?
  • audit reports are only useful if the purchaser or food producer reviews the results, understands the risks addressed by the standards and makes risk-reduction decisions based on the results;
  • sunnybrook-auditorthere appears to be a disconnect between what auditors provide (a snapshot) and what buyers believe they are doing (a full verification or certification of product and process);
  • third-party audits are only one performance indicator and need to be supplemented with microbial testing, second-party audits of suppliers and the in-house capacity to meaningfully assess the results of audits and inspections;
  • companies who blame the auditor or inspector for outbreaks of foodborne illness should also blame themselves;
  • assessing food-handling practices of staff through internal observations, externally-led evaluations, and audit and inspection results can provide indicators of a food safety culture; and,
  • the use of audits to help create, improve, and maintain a genuine food safety culture holds the most promise in preventing foodborne illness and safeguarding public health.

Audits and inspections are never enough: A critique to enhance food safety

30.aug.12

Food Control

D.A. Powell, S. Erdozain, C. Dodd, R. Costa, K. Morley, B.J. Chapman

http://www.sciencedirect.com/science/article/pii/S0956713512004409?v=s5

Abstract

Internal and external food safety audits are conducted to assess the safety and quality of food including on-farm production, manufacturing practices, sanitation, and hygiene. Some auditors are direct stakeholders that are employed by food establishments to conduct internal audits, while other auditors may represent the interests of a second-party purchaser or a third-party auditing agency. Some buyers conduct their own audits or additional testing, while some buyers trust the results of third-party audits or inspections. Third-party auditors, however, use various food safety audit standards and most do not have a vested interest in the products being sold. Audits are conducted under a proprietary standard, while food safety inspections are generally conducted within a legal framework. There have been many foodborne illness outbreaks linked to food processors that have passed third-party audits and inspections, raising questions about the utility of both. Supporters argue third-party audits are a way to ensure food safety in an era of dwindling economic resources. Critics contend that while external audits and inspections can be a valuable tool to help ensure safe food, such activities represent only a snapshot in time. This paper identifies limitations of food safety inspections and audits and provides recommendations for strengthening the system, based on developing a strong food safety culture, including risk-based verification steps, throughout the food safety system.

Iowa editorial says egg conviction insufficient

Iowa egg producers Austin “Jack” DeCoster and his son, Peter, stood in a Sioux City courtroom April 13 and were sentenced for their role in the nation’s largest egg-related salmonella outbreak.

egg.dirty.feb.12That outbreak, which sickened at least 56,000 people (2,000 confirmed) and led to a record-setting recall of more than half a billion eggs, stands as one of the worst cases of corporate negligence in Iowa history.

At the hearing, U.S. District Court Judge Mark Bennett said the DeCosters “created a work environment where employees not only felt comfortable disregarding regulations and bribing USDA officials, but may have even felt pressure to do so.”

Given the magnitude of those crimes, and the tens of thousands of people who suffered as a result, the sentence that was handed down seems seriously lacking: The DeCosters were each fined $100,000 and sentenced to 90 days in jail, with a year of supervised release.

But the fact is, it’s not often that corporate executives are held criminally responsible for their companies’ actions, even when the nation’s food chain is poisoned. Most cases are resolved with corporate fines, and in this case the DeCosters’ Quality Egg had to pay almost $7 million in fines, restitution and forfeitures.

What’s more worrisome than the 90-day sentences is the fact that the DeCosters flouted federal regulations for years without ever being caught. The regulatory system that is supposed to prevent — not simply respond to — violations of food-safety regulations failed us completely. It wasn’t until consumers started becoming ill that investigators took any sort of meaningful action against the DeCosters.

According to federal authorities, the company deliberately and routinely provided false paperwork to an independent auditing firm that periodically inspected the plant and reviewed the company’s records to ensure the eggs were safe. On the eve of each impending audit, workers were given blank, signed audit forms and told to fabricate data for the reports. This went on for at least three years, at a time when the DeCosters were producing more than 1 million eggs per day.

For at least eight years, Quality Egg regularly shipped its customers eggs that were labeled with falsified processing dates and expiration dates to conceal the fact that the eggs were old. According to court records, this mislabeling of DeCoster eggs “was a common practice, and was well known among several Quality Egg employees.”

In 2010, federal inspectors conducted on-site visits to the company’s egg-laying facilities and feed mill. Inside, they found frogs; wild birds; a chicken skeleton; mice, beetles, maggots and flies; and manure that was piled to the rafters inside one building. Salmonella contamination was pervasive and widespread “throughout the entirety” of the Decosters’ Wright County egg operations.

On at least two occasions, Quality Egg officials bribed a USDA inspector to overlook regulatory violations — in one case, paying $300 from the company’s petty cash account.

Given the DeCosters’ long history of alleged regulatory violations related to salmonella outbreaks, the minimum wage, pollution, workplace safety, animal cruelty, child labor and the hiring of undocumented immigrants, government regulators should have been particularly vigilant in their oversight of this family’s Iowa operation. But they were not.

Safety don’t have much to do with it: Food safety audits are just what retailers wanted

I’m not Dr. Doug.

doug.coach.happy.feb.15Kids call me coach, others call me asshole, all fine by me.

Dr. Bob and Dr. Dan exist, and probably many others in the misguided belief they are enhancing the public understanding of science, when it’s just demeaning and arrogant.

Dr. Bob says during grower food safety events, “we often talk about why having a food safety program is important and how it is critical to have a program to protect your own business, protect your customers and, ultimately, public health. We talk about emerging science, the importance of foundational food safety programs such as sanitation practices and worker hygiene and how to identify and manage potential cross-contamination hazards on the farm and in the packinghouse.

“After going through this information and basically laying out the why, how, and what of food safety, often some brave soul in the audience will raise their hand and ask, So what score do I need to get in order to pass the audit? And that’s when the frustration sets in. How did passing an audit become a substitute for actually building a risk-based food safety program?”

Oh, Oh, Dr. Bob, I can answer that.

Because back in the late 1990s, as fresh fruit and vegetable outbreaks took on national prominence, retailers decided, we want third-party audits, rather than food safety programs promoted by grower groups.

I chaired a national committee in Canada about 2002 to look at the issue, came up with a solution that would be advantageous to growers and consumers, and was then overruled behind the scenes so the grower groups could keep their Canadian Food Inspection Agency funding (and the bureaucracy).

I walked away.

Thirteen years later and Dr. Bob is wondering how this happened?

But will fewer people get sick? USDA seeks alignment of GAPs audit with food safety law

The Good Agricultural Practices audit offered by the U.S. Department of Agriculture for fruit and vegetable growers will be updated to reflect the requirements of the regulations from the Food Safety Modernization Act.

sunnybrook-auditorThe new joint GAPs review project includes staff from the USDA Agricultural Marketing Service, the Food and Drug Administration and state partners, according to Leanne Skelton, USDA liaison to the Food and Drug Administration on food safety issues.

“We’re just getting started, and the first meeting will be in a couple of weeks,” Skelton told the U.S. Department of Agriculture’s Fruit and Vegetable Industry Advisory Committee March 10.

She estimated the process may take one year to 18 months to complete.