NASA Has No Contingency Plan For Uninterrupted Human Access To Space
GAO: NASA Commercial Crew Program: Plan Needed to Ensure Uninterrupted Access to the International Space Station, GAO
“Further delays are likely as the Commercial Crew Program’s schedule risk analysis shows that the certification milestone is likely to slip. The analysis identifies a range for each contractor, with an earliest and latest possible completion date, as well as an average. The average certification date was December 2019 for Boeing and January 2020 for SpaceX, according to the program’s April 2018 analysis. Since the Space Shuttle was retired in 2011, the United States has been relying on Russia to carry astronauts to and from the International Space Station (ISS). Additional delays could result in a gap in U.S. access to the space station as NASA has contracted for seats on the Russian Soyuz spacecraft only through November 2019.
NASA is considering potential options, but it does not have a contingency plan for ensuring uninterrupted U.S. access. NASA’s certification process addresses the safety of the contractors’ crew transportation systems through several mechanisms, but there are factors that complicate the process. One of these factors is the loss of crew metric that was put in place to capture the probability of death or permanent disability to an astronaut. NASA has not identified a consistent approach for how to assess loss of crew. As a result, officials across NASA have multiple ways of assessing the metric that may yield different results.
Consequently, the risk tolerance level that NASA is accepting with loss of crew varies based upon which entity is presenting the results of its assessment. Federal internal controls state that management should define risk tolerances so they are clear and measurable. Without a consistent approach for assessing the metric, the agency as a whole may not clearly capture or document its risk tolerance with respect to loss of crew.”
Where was GAO when the Shuttle was shut down as what was then a plan to eliminate US paricipation in ISS? NASA has dealt with crew access problems following the Columbia loss, and has been dependent on Russia since the Shuttle program was terminated. Now they are finally going to have redundant access via SpaceX and Boeing. And GAO is upset bcause the Souyz contract might run out first and wants a “contingency plan”? How about reduciing micromanagement of Commercial Crew and letting them kick the tires and light the fires?
True. And going back even further, where were they when the OSP program was dropped which would have delivered two options for human crew to ISS a decade if it wasn’t replace by Dr. Griffin’s Project Constellation fantasy.
We’re still paying for Dr. Griffin’s “vision” with SLS being somewhat similar to Ares V.
Yes, and don’t forget that the Orion was to launch on the Ares I. The first contract for it was in 2006 under Administrator Griffin
The question really is what is the purpose of “clearly capturing or documenting (NASA’s) risk tolerance with respect to the loss of crew”? Is it to prevent such a loss from happening again in the same manner? Or is it to “cover the six” of whatever agency gave its approval for the mission (NASA, FAA, et al)?
I recall an article in Aviation Week BEFORE the first Shuttle flight that quoted NASA as saying “a booster failure was not survivable”. NASA did not apparently consider that a booster failure might be contributed to (there were o-ring leaks detected on two flights prior to Challenger) by a manager at MSFC who told the contractor manager to “stop thinking like an engineer and start thinking like a manager”, when the latter recommended not launching due to temperature conditions outside the design envelope. All Shuttle flights following Challenger were cancelled until the SRB joints were redesigned, it is true, but that might have been done anyway, if Challenger hadn’t launched.
What happens to a HSF program following the loss of a crew should depend on the probable cause, just as happens in aircraft accidents. The loss of TWA Flight 800 did NOT result in the grounding of a single 747 while the crash was investigated, in spite of the loss of several hundred passengers, who were NOT aware of the risks involved in flying. And it took nearly a decade before fuel tank inerting was completed in all 747’s.
Loss of a crew may be caused by undetected design flaws, safety equipment failures, or (as in the case of Spaceship I) human error combined with safety interlocks. What effect such a loss will have should depend on the cause and the cure. Has anybody looked at the potential for loss of a single crew person, say, if the water system in their space suit drowns one? EVA’s were temporarily suspended until the problem could be analyzed and corrected.
But some common sense is going to have to prevail or we might just as well send our astronauts to their rooms and let somebody else take all the risks!
Ad Luna! Ad Ares! Ad Astra!
To be fair, didn’t NASA pad the last date of Soyuz access in order to accommodate slippage? And are we now slipping beyond even those contingency dates?
That is my recollection as well.
In terms of crew risk and the certification process, I think the GAO is telling us something we knew or suspected. “NASA has not identified a consistent approach for how to assess loss of crew…”
That’s the requirement for a 1 in 270 risk of a loss of crew accident. It has been pointed out that, without significant flight experience, it’s impossible to say whether or not a vehicle satisfies this requirement. Now the GAO is saying “officials across NASA have multiple ways of assessing the metric that may yield different results.” In other words, different managers within NASA have different ideas about this and what it takes to certify Dragon 2 or CST-100 Starliner. And that ambiguity is slowing down the certification process.
The normal practice, as the GAO points out, is for someone (ideally at NASA, but certainly with NASA approval) to take the 1:270 requirement, and work out a list of things required to satisfy it. That list should be clear, specific and measurable. For example, “perform an in-flight abort test at max Q with the crew compartment subject to acceleration less than 90 m/s^2 (peak) and 40 m/s^2 (average over all 15 second windows.)” We could argue about what should be on the list, or if checking off all the boxes would actually assure the desired 1:270 risk requirement. Those sorts of level 2 requirements are the official way of saying “if you can do X, Y and Z, then we’ll believe you satisfy the higher level requirement.” The people building the hardware know what they have to do, and there aren’t ambiguities about whether or not the requirement has been satisfied. Otherwise, you end up designing to a moving target.
“Otherwise, you end up designing to a moving target.” And moving targets are the enemy of fixed lump sum contracts. Surely the contractors (SpaceX and Boeing) would be putting their hands out for money for client-driven delays and/or rework?
That depends on the language in the contract. I do not know the details of those contracts. With cost plus contracts, vague requirements are something the contractor might not mind. If the requirements change, they can just say, “sure, whatever you want, but that’s going to mean sending us more money.”
For firm, fixed price contracts, it’s a different matter. The contractor has to deliver what the contract specified. If they weren’t careful about the terms of the contract, that could be a problem. If the terms are vague, the customer (NASA) could say, “No, we actually want X plus a few other, related things, not just X” If the contract simply said, “do something like X” the contractor could be stuck doing extra work at their own expenses.
That’s why you really want the contractional requirements to be specific and measurable. And that is something the GAO is criticizing about commercial crew. NASA has not described the safety requirement in specific, measurable terms.
Usually the contractor’s lawyers are smarter than that, so fixed cost doesn’t mean the customer can change the requirements to whatever they want. Instead, it means the contractor and the government have to renegotiate the contract with the new requirements.
But what happens when the customer won’t listen to the contractor, when the latter recommends a change that could prevent a serious problem, possiblly because the customer doesn’t want to spend money? Or, because the customer doesn’t really know what it wants?
In one specific instance, a contractor warned the customer that the vibration frequency of the firing of a separation charge was within the range of the detection frequency of a subsequent event of ordnance. The customer did not want to make any changes. By chance, nothing bad happened on the first three flights. On the fourth flight, the secondary ordnance fired when the separation charges fired, resulting in premature detachment of parachute deck fittings, resulting in the loss of the high-value, otherwise reusable payloads! Fortuantely, no crew safety was involved. Only then did the customer come in with an engineering change proposal to add a baroswitch in series with the second firing circuit. Of course it cost additional money.
On that same program, with a 1980’s value of around $25M, we went through over 200 change orders during the life of the contract!
Other problems occurred on other programs when the customer and the contractor forget to be in agreement as to what measurement system, English or metric was being used. That resulted in loss of a Mars probe!
Perhaps the main safety requirement that NASA should specify should be, “Get the crew up and down in one piece! Contractors please specify how you propose to do that!” Then they can negotiate the specifics, both on the physical side and the monetary side. Maybe the contractor should be required to fly on x-number of flights…like parachute riggers are (or used to be) required to do!
We need to get to a place where NASA doesn’t certify spacecraft for its astronauts. They have little to nothing to do with certifying aircraft yet their employees fly on them quite frequently with NASA footing the bill. Why should NASA be doing the same for spacecraft? It’s not the 1960s anymore. NASA is really not the leader in this field anymore. The last time they designed and flew for the first time a crewed spacecraft, it was the 1970s with first flight in 1981. Many of those experts from the space shuttle era are not with NASA anymore.
Right now, the FAA is certifying each launch of commercial (but not government) launch vehicles. In addition, SpaceX had to be licensed by the National Oceanic and Atmospheric Administration in order to, and because, the Falcon9 second stage cameras were catching images of Earth! FAA certifies experimental aircraft, including Spaceship I and II. The same sort of thing should be done with manned launch vehicles and spacecraft…except I’d hate to see the bureaucracy corrupt the FAA. If NASA doesn’t want their astronauts to fly unless THEY certify a vehicle, then let them sit on the ground! Maybe those astronauts would resign and go to work for SpaceX, Boeing, Blue Origin, et al. Those companies would not want their space “equipment” to fail catastrophically and lose a crew anymore than a commercial airline does. Bad for business. Of course, there will always be unforeseen problems…getting hit by a large meteor, for example. If that should happen, then you turn the incident over to the NTSB, and let them come up with the “probable cause”, (Hopefully, they would proceed a little faster than current aircraft incident investigations…about a year!) Interestingly, NASA didn’t ground their astronauts from flying T-38’s when several fatal accidents occurred. (Pretty hard to insist on armoring the canopy of a plane against a goose strike!)
Of course, I don’t know if relieving NASA of certification responsibilities would take an act of Congress, or just an executive order by the president.
A technical correction to my hasty note of last night: FAA does not “certify” commercial spacecraft, they license them. A very different process that focuses on different things (in particular minimizing risk of damage to third parties.)
My understanding is that NASA only “certifies” the CCDev vehicles as being suitable for NASA employees to fly on. (Though it seems clear a faction at NASA would gladly have this apply to all US commercial flights if they could.)
But that NASA faction is able to make this process unduly onerous for reasons of protecting NASA turf because, well, NASA is funding CCDev.
In theory Congress should be keeping them focused on advancing the national interest in affordable US space transportation. But for the moment the majority in Congress is not paying attention – just going along with a minority regional faction whose local pork interests are aligned with the NASA HSF development bureaucracy’s turf-protection interest.
And thank you for correcting my technical mistake on that point! Suppose the FAA were to issue “type” licenses or whatever they are called for launch vehicles and manned spacecraft, just like they do for a new airliner? Would they then have to have pilot astronauts (not mission specialists or passengers) pass a test (probably written and in a simulator) to “type rate” these pilots? Hmmm? Not sure what kind of laws would have to be passed and by whom. And that would take “certification” away from the bureaucrats at NASA…unless, of course, as I stated above, NASA wouldn’t let their astronauts fly without NASA’s certification. Wonder how that would sit with the military service from which some “NASA” astronauts are on loan to NASA?
“Kick the tires, light the fires, first one in [space] is the leader!”
What he said. NASA doesn’t certify commercial air transports before NASA employees are allowed to book passage. Why should they be allowed to insist on certifying commercial space transports? That too is now the FAA’s job.
Elastic requirements are definitely something a good lawyer would want to keep out of a contract. (Or put in, depending on which party he was representing…) But some things slip through.
In this case, the commercial crew contracts would have definitely said something about safety and certification for human use. But at the time, NASA didn’t have a certification process and (according the the GAO) still doesn’t have one in a formal, measurable, sense. Saying you’ll follow a NASA-developed process before NASA has developed it’s smart, but it looks like that’s what they did.
This is disturbing, and I’m sure is frustrating to a company like SpaceX. Like was said…Boeing is used to it. 😉
As I understand it…NASA’s role in Elon’s plan goes something like this…in roughly this order…
1) Provide access to their knowledge base on the technical fundamentals.
2) Provide authoritative acceptance of the vehicle.
3) Provide a captive customer test bed for the design and technology to accumulate a flight history for the vehicle and and push it and the company through an improvement and experience arc.
“Help fund development and pre-flight testing” fits in there somewhere, but is not as high on the list as the public perceives that it is.
The first two of these three items are dependent on NASA having their technical house in order so that they can serve as a knowledgeable authority.
Boeing will be Boeing, but SpaceX has a goal that they intend to reach that, frankly, has only a passing resemblance to supporting an old LEO research station. They have to be wondering by now if their isn’t a better way to get to Mars than on NASA’s back. There isn’t of course, but they have to be wondering.
Because failure rate cannot be accurately predicted without flight experience and is not even constant. A typical launch system has a high failure rate on the initial launch, and the failure rate decreases progressively as long as the design is not “set in stone” like the Shuttle and is modified to eliminate failure modes as they become apparent.. To specify failure rate to three decimals in the design contract when it cannot be determined with any accuracy until the rocket has flown 6-10 times does not make it so. Sort of like telling the contractor to build the Webb for less than the cost of the Hubble.
The problem is that the NASA risk determination process is based on the premise that all the possible failure modes of an untested sysem, together with thier probabilities, can be detrmined solely by analysis. Equally critical, the NASA failure analysis method assumes that failres are randomly distributed events with constant failure rates.
Unfortunately these assumptions are wrong and have been proven wrong on multiple occasions. As Chang pointed out in his seminal paper “Space launch vehicle reliability” the majority of major contingencies are the result of unanticipated failure modes, or of failure modes for which the estimation (i.e. WAG) of the failure rate was off by orders of magnitude.
Equally important, the NASA risk method assumes that failures are random events. In reality the majority of major historical launch vehicle failures are deterministic rather than random. Random failures in mechanical systems require stochastic processes such as crack propogation, corrosion, and abraision. Launch vehicles may have a working lifetime measured in minutes, and there is no time for stochastic processes to occur. If a failure is deterministic, the answer is adequate testing to establish the failure mode and a design change to eliminate it, not redundancy and coningency plans to make us believe we can mitigate it when it occurs.
Planetary probes are susceptible to both kinds of failures, but in different subsystems. Random failures predominate in electronic systems and in mechanical systems such as rover wheels that may operate for years, while planetary landing systems operate for brief periods and failures are deterministic. In either case reliability can best be better assessed by life cycle testing than by analysis, which can only discover what you already know.
NASA is also convinced the answer to reliability is redundancy. As I understand it in the planetary community some people are finally beginning to realize that for systems susceptible to deterministic failures the most reliable system may be one with extensive testing but no redundancy at all.
“The problem is that the NASA risk determination process is based on the premise that all the possible failure modes of an untested sysem, together with thier probabilities, can be detrmined solely by analysis.”
I believe it is a more fundamental issue. The problem is some on congress want an SLS launch first….
“planetary landing systems operate for brief periods and failures are deterministic”
OK. I know what ‘deterministic’ means but can’t relate it to this context. Perhaps the scientific world uses the word in a peculiar or limited way?
The Ariane 501 failure is a good example of a deterministic failure mode. The guidance system had a software problem. If, within the first 40 seconds after launch, the vehicle traveled more than a certain horizontal distance, an error would occur. That error would send confusing data to the main computer, in a way that would assure a launch failure. No sort of random chance was involved, so the failure of the first Ariane 5 launch was completely “deterministic.”
The good thing about these failure modes is that they can be fixed once they are identified. Once fixed, the probability of the same thing happening again is zero. (Yes, I know, the probability of the fix having a different problem is non-zero…) In theory, they can also be identified and fixed in advance, before an accident occurs.
But in practice, there are almost an unlimited number of possible failure modes, and some things are likely to slip through. That’s one of the reasons why reliability generally increases with flight experience. Experience is the most comprehensive way to identify these sorts of problems.
Didn’t one of the early attempts to send a probe to the moon fail because a “+” was substituted with a “-” by accident in the guidance program?
The Russians lost a Proton because the INS gyros were installed upside down. When it lifted off, it thought it was upside down and tried to correct itself. Boom!
I don’t know about those failures, but they aren’t uncommon.
Sign errors in guidance or attitude control software are a known problem which never seems to go away. The one I remember best was a spacecraft which launched, opened its solar arrays and then, faithful to the sign error in its software, pointed the solar arrays directly _away_ from the Sun.
The Galileo atmospheric probe and the Genesis sample return capsule both used an accelerometer to trigger their parachute opening sequence. Both were wired in backwards, so the signal never went out. Fortunately for Galileo, the probe also had a timer as a backup, so we only lost fifteen seconds or so of data. Genesis didn’t, so it crashed. They managed to recover enough of the samples to claim a successful mission. But that’s generally considered a very minimal or partial success. I’m unhappy that the Galileo problem wasn’t widely described; if it had been, the Genesis team might have seen it as a “lessons learned” cautionary tale and been more careful.
Or maybe take the absolute value of acceleration. More serously, as these are one-off spacecraft designs, the primary mechanism for acquiring actual failure mode data, flight expereince, is not available.
nonrandom. Classic example Challenger. SRB had primary O-ring, and if it failed, a backup O-ring. This would have been OK if the failure was random, because it would have been unlikely for both to fail at the same time. However because the failure ocurred because of unanticipated conditions that made the probability of failure almost 100%, both the low temperature and an unexpected wind shear, both of which affected both O-rings, so both primary and secondary failed simultaneously. The answer NASA has chosen is to require redundancy but make the prime and backup systems different. This is often inappropriate because it complicates the system and may add new unanticipated failure modes.
I have a contingency plan for uninterrupted US human space access.
RIF the small army of “Certification” bureaucrats who after years and many many MANY millions still can’t usefully define “safe enough”. Test-fly both Commercial Crew ships ASAP. Then as soon as either works without obvious immediate loss-of-crew issue, put people on board and start flying them to Station and back.
All out of patience? Moi? Why yes, I am. I was publicly predicting something like this never-quite-there Xeno’s Certification result back when they first started pushing to ditch the COTS-style Space Act Agreements and bring Commercial Crew back under the FARS.
On consideration, I’ll even concede that RIFing the lot of ’em might be seen as overly harsh. I’d happily settle for NODing them instead.
To paraphrase the late G. Harry Stine, get those bureaucrats away from that rocket and FLY it.
The NASA certification function has SLSLaunchDate as a variable.
Any timing connection between SLS and CCDev is a lost cause at this point.
Certain clueless Congress folk, and whatever of their little minions are still scampering around in NASA, need to let it go. ISS is not financially viable with SLS/Orion supporting it and the crewed SLS timeline is too far down the road to provide ANY useful crew transport options for the remaining life of the ISS anyway.
Also, even if everything I just said weren’t true, SLS is still doomed to obsolescence, people will still fly on Dragon, and Mars will never be colonized by Orion.
This is happening for many more reasons than just keeping CCDev from making SLS/Orion look pointless.
A big part of it is bureaucratic turf defense. COTS Commercial Cargo bypassed a major part of the permanent HSF bureaucracy. (A big BIG part of why COTS succeeded so quickly and cheaply.) CCDev’s death of a thousand paper-cuts is the HSF “Safety” bureaucrats’ way of reestablishing control of the spacecraft development process.
As for forcing SLS/Orion dependence on Station likely killing Station, that’s a feature not a bug to these people. Killing Station frees up more billions for the HSF spacecraft development blob to absorb while going through their dimly remembered bring-back-Apollo Cargo-Cult human spacecraft development rituals (that haven’t actually produced any usable new spacecraft since 1980.)
Well ya, and it’s silly. Both the Atlas and Falcon have a much deeper launch history than SLS will ever have, and they do the most dangerous part. For Dragon and the CST-100, I should think that COTs and CCDev were actually supposed to streamline the health and safety paperwork side to some degree and instead just customize more modern industry quality control practices to cover spaceflight. That needs to happen in order for industry to learn to do this stuff competitively and profitably.
Also, the selection side of COTS and CCDev are essentially NASA Spinoff programs. While Spinoffs may at times disrupt markets, they were never at the pointy end of NASA’s spaceflight stick. COTs and CCDev were designed and intended from the start to disrupt the Spacecraft launch and development markets and that sharp stick pokes Congress and the military industrial establishment (two points of the traditional NASA procurement triumvirate) right in the eye.
That has to annoy folks and I think THAT is at the heart of most of these delays. It is how Congress asserts displeasure over a program…find ways to slow it down so that overhead costs have time to catch up to it and eat it.
I have a simuler idea, allow them to fly their spacecraft with employee crews as many times as they want pay them for those flights and treat those as certification flights
Both SpaceX and Boeing originally proposed test-flying their ships initially with company test-pilots. NASA put a quick stop to that. After all, if company test pilots fly before NASA astronauts, then what’s the point of NASA astronauts? People might start asking whether maybe things have changed since 1962…
In the Murray/Cox “Apollo” book they mention that the Houston flight controllers had a way to make the astronauts mad. They would say that they could take any reasonably fit & intelligent person off the street and train them to fly a lunar mission in about a year. They didn’t even have to have a pilot’s license.
Nice to see GAO calling NASA out on the BS way they are certifying Dragon 2 for flight.
You almost wonder if there are folks at NASA trying to turn CCP into a failure by safety requirement creep. Then they will be able to claim to Congress, we tried commercial and it didn’t work. Now give us XX billion to convert the Orion to service the ISS.
No, I don’t wonder at all.
Nothing new. NASA has not had a contingency plan siñce 2011. Remember, their plan was that Orion would be carrying crews by 2011. They are now only a decade behind.
Over in the Star Trek alternate universe next door, SpaceX has been flying crew to ISS for 2 years already , because there was no NASA Risk Aversion Clerk barricading the road to the launch pad.