NASA Spacewalk Mishap Investigation Board Report
“While I am concerned about ensuring this particular incident does not happen again, I am especially concerned about cultural factors that may have contributed to the event. In our exuberance to get the job done, we may have allowed ourselves to accept the commonly accepted causes for small anomalies. We have a responsibility not to move on from any abnormal situation until we understand it fully or have suitable mitigations to prevent it happening again. Our work both in-house and with our industry and commercial partners should entail diligence in assessing risk and commitment to ensuring mission safety.”
– News Conference Presentation – 2/26/14 (120 Kb PDF)
– Full report (11.2 Mb PDF)
“In summary, the causes for this mishap evolved from (1) inorganic materials causing blockage of the drum holes in the EMU water separator resulting in water spilling into the vent loop; (2) the NASA team’s lack of knowledge regarding this particular failure mode; and (3) misdiagnosis of this suit failure when it initially occurred on EVA 22.”
NASA Updates Media on Spacewalk Safety Investigation
“NASA will host a teleconference at 2 p.m. EST today to discuss the findings of an investigation into the July 2013 spacewalk at the International Space Station when water built up in an astronaut’s spacesuit helmet. Soon after the incident, NASA created a Mishap Investigation Board to identify factors that may have contributed to the incident and recommend changes that could be implemented to prevent a similar situation from occurring again. This safety investigation ran concurrently with an engineering investigation into the equipment failure.”
It will be interesting to find how far up the chain the prior leak was discussed. Also, it is surprising Luca would be willing to use the same suit again after the first leak.
Bob Clark
Considering there are multiple U.S. EVAs scheduled in 2015, in support of relocating the PMM and upgrading station docking ports thgey’re now aware how vunerable spacewalkers are than before Parmitano’s close call. Then again the EMUs are more than three decades old and should have been replaced by now.
This is always a fascinating discussion. On the one hand, some complain about sloppy engineering, overbearing management, lack of attention, “normalization deviance” and other things that suggest NASA should be more careful, more vigilant, more conservative.
On the other hand we have the “Safety is not an Option” school of thought that we “have to expect losses in a big operation.” They would have us change the most likely cause of the problem and go on. After Challenger, these were the types who said “just fly on warmer days and you will be fine.” They missed the profound learning that came from both that accident and Columbia.
We can’t have it both ways. We have to decide what is an appropriate level of care and rigor (a pretty high bar) and then let people go do their jobs. When problems occur, we need to react responsibly and fix what we think is wrong before trying again, carefully.
These EMUs have done great service for NASA and the folks who oversee their operations are experts and professionals. In this case something new happened and they were surprised, but they got the crew back in safely and a few months later they had enough confidence to conduct the pump replacement EVAs.
I for one think NASA has done a good job balancing great care for safety with getting the job done. The real lesson is for the newcomers, especially those who think they know how to design systems that do not have these kinds of subtle failure modes. They need to understand how NASA has done so well and try to learn from this incident.
This is yet more evidence that the majority of serious failures in space are the result of unanticipated failure modes. This isn’t surprising; if we understood a failure mode we would eliminate it in the design. But it’s significant because NASA safety is built on the probabalistic risk assessment, or PRA, and the PRA assumes failure modes and failure rates are constant and predictable, which is a completely unwarranted assumption for most launch vehicle and spacecraft hazards. In reality the PRA is at most a WAG. Better safety strategies for such hazards involve simplification of design, life cycle testing and periodic inspection. For launch vehicles, this means the majority of launches of any type should be unmanned, to identify failure modes before the are likely to occur on manned launches. The NASA strategy of attempting to
eliminate single point failures by redundancy is often inappropriate because redundancy is applied only to prevent anticipated failures, and because it invites common-mode deterministic failures, as seen in Challenger. Redundancy inevitably adds to complexity, mass, and
cost.
I think you’re confusing “risks”. The risk that NASA managers seem to reflexively avoid is being the person who makes the costly decision. “Go fever” isn’t a willingness to take risks, it’s a fear of being the one who says “No go”.
Challenger wasn’t the willingness to risk launch, it was a fear of being the one who says “scrub”. Columbia wasn’t a willingness to risk a reentry, nor was it really a fear of the risk of a rescue operation (although I suspect that played a subconscious role), it was a fear of being the one who says “Okay, do an (untrained, non-SAFER) EVA to assess damage” and it turns out there’s little damage. (Or worse, causing more damage from the EVA, or losing an astronaut, or…)
As such, there’s a willingness by managers to accept whatever partial analysis and advice allows them to make the least personally risky decision, even though that defers greater risk to later in the mission, or to later missions, and to dismiss those warnings of problems.
You are mistaken. Obviously people don’t like to take personal risks, but few are foolish enough to think that ignoring a problem now will excuse them from bad events later.
An essential core through all of this is deciding what risks are credible and what are not. Challenger was a bad example because of the way the roles and responsibilities were distributed across the centers. And most of Columbia was a discussion about tile damage, not leading edge damage. There was extensive history of tile damage that had been very benign. There had been virtually zero history of leading edge damage, and there was a strong belief that there was no mechanism to get tile to the leading edge. Thus there was no expectation that a life threatening damage event could possibly have happened. They were wrong, obviously, but the argument was logical.
This is not an abstract argument for me. I chaired over 300 mission management team meetings for ISS including 36 American EVAs. The process of deciding what risks to consider and how deeply is a constant one. The public only hears about a tiny fraction of the anomalies that could be indicators of potential new failure modes. If NASA chased every one they would not get anything done. So they make decisions on what ones are possible/credible and what are likely. They then work on them on the basis of that priority. The vast majority of the time they are right. Unfortunately, this industry is not very tolerant of error.
By the way, the lack of SAFER was no issue. When the Shuttle was not docked to a station it could chase down a detached crewmember.