Investigation Failures, Root Cause Problems Continue To Bedevil Manufacturers

Investigations of manufacturing failures and detection of their root causes might be improved by a more systematic but open-ended approach to reviews or evaluation through a lens of best practices, experts suggested Sept. 13 at the Parenteral Drug Association/FDA joint regulatory conference in Washington.

They suggested that new more holistic approaches are needed given that health inspectors worldwide continue to see problems with inadequate investigations of why manufacturing lots fail to meet specifications and the inability of the pharmaceutical industry to detect the root cause of quality defects found in manufacturing operations – and there has been little progress in more than two decades.

As David Jaworski, a senior policy advisor in the FDA drug center’s Office of Manufacturing and Product Quality, noted, despite the industry’s adoption of International Council on Harmonization's Q10 quality systems guideline in 2010 and quality guru Joseph Juran’s strategies to achieve optimal world class quality in the early 1990s, the industry has not made much progress over the past 25 years in achieving this vision.

He said that “pharmaceutical quality systems still lag in effectiveness for many of our companies; there are still too many recalls and adverse events that are traced to quality issues.”

The Right And Wrong Way

Karen D’Orazio, a consumer safety officer for FDA’s Center for Drug Evaluation and Research, shared two case studies showing the right – and wrong – way of conducting failure investigations.

These case studies were based on a review of 26 warning letters, untitled letters and regulatory meetings over three years, from July 2013 to July 2016. She found that failure to adequately extend investigations was found 13 times under 21 CFR 211.192, while failure to identify root cause was found six times. Not extending the investigation to other products was found four times and corrective action not adequate found three times.

One successful failure investigation involved a finished sterile powder fill operation for which FDA's district received field alert reports for complaints of cracked bottles.

The firm's practices called for inspecting glass bottles for stoppers and labels and placing those rejected for missing labels in a bin – and cracks occurred when the bottles hit the bin. In reviewing the cracks, the firm hypothesized that shipping was responsible for the cracks.

But it subsequently identified the correct cause by bringing in a material science expert, who examined the bottles and observed that tossing the bottles in the bin caused the cracks.

In contrast, D'Orazio cited an over-the-counter acne control product manufacturer that hypothesized on several causes of a failure, but neglected to then prove those were the right causes or to look beyond isolated incidents.

This firm rejected a batch for failing established microbial specifications. It concluded that the failure of the initial lot was due to the insufficient cleaning of a transfer pump, but had no data to support this conclusion. The pump was sanitized but the first lot afterwards also failed the established microbial specifications.

The firm then concluded that the source of the contamination was in the filling process, but was unable to specify why or where in the process that contamination occurred. It also didn't extend its investigation to other products.

D’Orazio said that “what they did was they said it was somewhere in the filling process. Then it was everywhere. They did not extend the investigation to other products to see if there were any other problems.”

She commented that “this firm really did not have any good quality system to speak of. The water had been leaking and there were multiple excursions from microbial contamination from the water as well. The lab methods were not validated and the clearing method was not validated. The persons there did not understand their processes even well enough to understand the root cause.”

Information First, Not Hypotheses

Acknowledging the difficulties of conducting failure investigations, D’Orazio described her own experience outside of pharma to show how investigator bias can cloud the effort.

In her first job as an environmental inspector, her duties were to ensure appropriate air quality. She received complaints of poor air quality from the lower floors of an urban medical center. She was convinced the problem stemmed from vents circulating air that were located near a loading dock and a bus stop.

“I could not get out of my head that it had to be the same two intakes.” She told her supervisor that the problems could be fixed for $115,000. The supervisor told her to get another opinion.

They consulted an air quality control expert who disagreed with D’Orazio’s assessment, and said "‘let’s go across the street and see what’s going on.’” The odors actually emanated from an incinerator across the street, which D'Orazio said was not supposed to operate during daytime hours. "Yet the smoke was always coming out. That picture was worth a thousand words … I was presuming I knew what the root cause was even before I did the inspection.” D'Orazio realized she was caught up in proving a theory and fitting the facts to that theory.

D’Orazio drew some lessons from the experience: don't assume to know the root cause before beginning the investigation; bring in appropriate expertise early on to identify all the potential processes contributing to the problem and interview the right people.

Mark Paradies, the president of System Improvements in Knoxville, TN, echoed D'Orazio's advice to develop as much information as possible before hypothesizing on the source of failures.

He focused specifically on effective root cause analysis and incident investigation. Among the "secrets" of effective reviews, he said, is understanding that “your root cause is only as good as the information you collect.” His comments suggested investigators should try to ask more questions on more varied issues, because often they ask the wrong questions.

For example, he showed a video of a driver nodding off before finally going to sleep. The accident report on the driver's subsequent car crash never mentioned that he fell asleep. Investigators incorrectly attributed the accident to distracted driving, never asking if the driver was fatigued.

Paradies' comments suggest the importance as well of wording questions in such a way as they are more likely to elicit needed information. People "never admit that they fall asleep, they say that ‘a dog ran out in front of me’ or ‘I don’t know what happened and the car rolled over,' because they really don’t know what happened." He noted that "one of the things I seldom see in industrial incident investigations is whether someone asks ‘are you fatigued?'"

Paradies, an electrical engineer, spent 31 years doing root cause work, and is the author of eight books on root cause analysis. He described the second "secret" as understanding what happened before probing why it happened.

He outlined a seven-step process to follow:

1. Plan the investigation.
2. Determine the sequence of events.
3. Define causal factors.
4. Analyze each causal factor's root cause.
5. Analyze each root cause's generic cause.
6. Develop and evaluate corrective actions.
7. Report and implement corrective actions.

“It was not just one guy’s fault. It is not that Joe screwed up. It is a whole bunch of things that led to this accident. We need to fix this and come up with a root cause for all of them. Figuring out what happened before why it happened gets you beyond assigning blame. You usually have multiple factors that have to be looked into.”

Don't Play Favorites

Reinforcing D'Orazio's cautions about the bias introduced by starting with a hypothesized cause, Paradies said the third “secret” is to understand that the investigator's knowledge, or the lack of it, can get in the way of a good root cause analysis.

Paradies said that “I am going to say this is obvious but maybe it is not. Karen’s example is a good one. Her first investigation where she knew what the answer was, so that her knowledge got in the way of an investigation. … If you don’t know the cause of the effect you observed, you probably will not find that cause because you will not have any knowledge of it. Should you be looking for things that you don’t know you should be looking for? That means your root cause effect is limited to your current knowledge.”

Paradies said that some of the tools that the pharmaceutical industry uses to conduct a root cause analyses – the fishbone diagrams, the five whys and fault trees – depend on this cause and effect relationship, which is dependent on investigator knowledge.

This model assumes a level of expertise and also training that may not always be there. For example, many seasoned investigators lack training in human factors.

During his employment at DuPont, Paradies observed “very few” of the experienced system engineers who conducted failure investigations had training in human factors. “Guess what causes most accidents and quality problem? Someone screws up. Someone makes a mistake. That is what we get and we get problems and we get mistakes, so if we don’t have training on what causes those mistakes, how likely is it that we will understand the cause and effect relationship that result in these errors if you have never had training in this?”

Investigators also should be wary of having a "favorite cause" for most problems, leading them to continue to search for that to be a cause until they find it, meanwhile ignoring many other questions that should be asked.

Paradies said that he developed a new definition of root cause to shift the emphasis from cause and effect, which is dependent on investigator knowledge, to a more systemic and holistic approach.

• The old root cause definition he used in 1986: “The most basic cause (or causes) that can be reasonably identified that management has control to fix.”
• The new root cause definition he started using in 2006: The "absence of a best practice or the failure to apply knowledge that would have prevented the problem.”

The new definition “allows manufacturers to focus on the lack of best practices in evaluating the problem. So what is the missing best practice and where is the knowledge that we should be applying to prevent these problems.”

Going Beyond Root Cause

Paradies suggested that the nuclear industry uses a more robust root cause analysis method that is worth considering in pharmaceutical manufacturing. Its model examines the generic cause of an accident and uses two tools: the “extent of the condition” and the “extent of the cause.”

The “extent of condition” evaluations are done soon after an accident to decide if additional immediate actions are needed to address the risk of additional failures while a root cause analysis is being conducted. Usually extent of condition looks for similar equipment related conditions. For example if a bearing fails are similar bearings used in similar circumstances or equipment that could also fail?

It can also be applied to human factors. For example if a valve is opened accidentally, are there similar valves that may be opened accidentally?

The “extent of cause” is usually performed after the specific root causes are identified that led to an accident. The extent of cause is used to decide if a specific root cause needs to be analyzed to find the generic cause behind the specific root cause.

Using this approach in drug manufacturing, he continued, “if you have product contamination, you would say 'is this the only product that we have that gets contaminated or can there be other products that are contaminated and if they are contaminated is this cause in other places?' So you are looking for the extent of the cause and the extended conditions.” If "you have the same cause in other places your corrective actions have to be more than just fix it here.”

Major Global Problem

The PDA/FDA conference presentations documented the extent of the failure investigations challenges as well as looking for new approaches to better investigations.

For example, Merck’s Anil Sawant said that the lack of adequate failure investigations was the top deficiency found in GMP inspections of Merck facilities worldwide.

A report prepared internally for Merck from inspectional observations said that inadequate “deviation management” was found in 37 inspections. The problems included failures to investigate discrepancies, inadequate investigations and ineffective corrective and preventive actions.

This was followed by inadequate documentation found in 18 inspections and then inadequate environmental control found in 16 inspections. Merck’s data was completed from inspectional data from FDA, the European Medicines Agency, the UK’s Medicines and Healthcare Regulatory Agency, and COFEPRIS, the Mexican health regulator. This intelligence was gathered and analyzed for inspections from the second quarter of 2015 to the first quarter of 2016.

Inadequate investigation of out-of-specification results continues to be a top deficiency found by FDA inspectors as well. Of the 42 drug GMP warning letters sent to pharmaceutical manufacturers in FY 2015, 11 cited inadequate investigation of out-of-specification results under Part 211.192, making it the fifth leading deficiency in that year's drug GMP warning letters. (Also see "FY 2015 Drug GMP Warning Letters Hit Compounders and Foreign Sites" - Pink Sheet, 29 Jan, 2016.)

In FY 2013, this deficiency was noted in 15 warning letters, and in 2014, it was noted in four warning letters. This blip was due to FDA’s focus on sterility practices at compounding pharmacies. (Also see "FDA’s Blizzard of Enforcement at Compounding Pharmacies Evident in GMP Warning Letters for FY 2014" - Pink Sheet, 26 Feb, 2015.)

From the editors of the Gold Sheet