SLS Plumbing Is Full Of Paraffin Wax. Oops.
Contamination found in SLS engine tubing, SpaceNews
“At a May 17 meeting of the Aerospace Safety Advisory Panel (ASAP) at NASA’s Kennedy Space Center, panel member Don McErlean said the committee had been briefed on a “late development” with the core stage, being constructed at the Michoud Assembly Facility in New Orleans. A “routine quality assurance inspection” of the core stage, he said, discovered contamination in tubing in the engine section of the core stage, which hosts the vehicle’s four RS-25 main engines and associated systems. That contamination turned out to be paraffin wax, which is used to keep the tubes from crimping while being manufactured but is supposed to be cleaned out before shipment.”
Earlier SLS postings
I wonder whether it was just a residue or whether the lines were completely filled.
Even if it’s only a little bit of residue, they’ll have to clean it up. I wonder if the “routine quality assurance inspection” happened soon enough, or if cleaning it up will require major disassembly. You really want to do those QA inspections as soon as the parts are delivered, not after you’ve put them together.
Not to mention that it would be hard to inspect the entire tubing bore. Can the fluid passages be machined, as the cooling passages are in newer engines? I have to wonder how much the wax actually helps, as it has a very low yield point. If a mandrel bending device cannot be used, a low-melting-point metal alloy might be more effective in preventing buckling.
I would guess they use paraffin because it melts at low temperature – easy to fill and empty a tube with it – and it’s relatively incompressible. Cap the tube ends before you bend it and the wax yield point shouldn’t matter.
I read somewhere that it was just residue, not tubes still full of wax. But yeah, leaving any wax residue at all in a tube that’ll carry oxygen is bad, bad, bad.
You’d think that with the amount of money being spent, they would have better QA processes.
Often more noney will just make it worst because it means more inspectors. Human nature being what it is it makes each inspector more lax because they know if they miss it the next one will see the issue. When there are fewer, or just one, they know they are all the stand between failure and success and really become obsessed with it.
I was thinking of something similar, but about the number of checks and inspections, not the number of inspectors. Faced with a very long checklist, it’s human nature not to pay less attention to each item. That’s especially true when the inspector doesn’t understand the reason for each and every item on the list.
One example, although it’s a bit obscure, is the Apollo 13 accident. According to one history, the original, damaged part which caused the explosion was a result of pre-flight tests. Someone was instructed to perform a test involving running the part for a specified length of time, while watching some diagnostics. One was the temperature, and he had instructions to stop the test if the temperature went over a certain limit for a certain amount of time. The measured temperature when up to that limit and stopped. Because that was the maximum temperature the instrument could measure. The person conducting the test didn’t know that, simply saw that the temperature didn’t go above the limit, and kept going. Which resulted in the damage to the insulation, which resulted in the explosion.
Adding more checks and tests every time something goes wrong is a common reaction. But it isn’t a miracle cure.
Reading the article more carefully, I noticed an interesting detail in terminology. The EM-1 launch schedule is described as “call[ing] for launch of EM-1 in the latter half of December 2019, but again with several months of schedule risk…” The term “schedule risk” is repeated on several occasions. I’m used to seeing a launch date and the schedule _margin_. As in, “we’re planning for March, 2020 but we should be ready in Dec. 2019. If something goes wrong and we slip by a few months, we’re still on track.” This use of the term “schedule risk” seems to be more like saying, “we really want to go in Dec. 2019, but we may not be ready.” That terminology implies something very different about how people think about schedule risk.
Sounds like another 2 to 4 year delay. Maybe they’ll fly in 2022? 2023?
Really ? Contamination ? Anyone who has ever been near a space craft fluid system knows these are a dime a dozen. What’s next? Scratches in sealing surfaces? ZZZ
I wonder if there will be a full up test on the pad of the SLS with the rocket locked down. Just in case there are any problems like this.
Wonder how much of the pad would be left if it blows up. Probably a year or more to repair it.
If Congress allows this beast to stay alive, and they don’t get every minute particle of the wax, it will be “interesting” to see what happens when LOX hits the particle! Why, oh why, doesn’t NASA turn this over to someone who remembers how to build big rockets?
I agree. LOX and paraffin do not mix. At least not well.
Actually they mix quite “enthusiastically”.
And for anyone who doesn’t get the joke, LOX in contact with pretty much any organic material results in a contact explosive.
Because the jobs wouldn’t be in Huntsville, the key mission goal of the SLS.
True! But present-day “rocket scientists” (I have never been one…I’m a retired ‘acid-on-the-hand’ rocket ENGINEER!), could move, as many of us once did.
Member of the Exalted Order of Aerospace Gypsies of the 20th Century! 😉 Or what year did you work for [insert name of the “fallen flag” aerospace companies]?
Oh, great! A softball!
“why, doesn’t NASA turn this over to someone who remembers how to build big rockets?”
Because nobody “remembers”, at least in the sense that I think you mean. If you were 30 in 1968, you are now 80 years old.
But: an SF story I read recently begins with the observation of FTL travel by Earth-bound astronomers. With this simple observation we knew that FTL was possible. That’s a big ‘find’, as they say.
The SLS engineers similarly started. But they had a lot more than the knowledge that it was possible to build the kind of booster that was necessary. For one thing they had a demonstration model over in the Visitor Center; the basic science was complete. They didn’t really need anyone who remembered. The S5 designers lacked even the knowledge that such a thing was possible.
Hawthorne, same thing, though with arguably less historical data.
With so many advantages memory wasn’t even needed.
Knowing something is possible certainly helps, and helps quite a bit. But there are also lots of things you learn the hard way, and that usually involves mistakes you have to correct once you discover the problem. Having people around who remember those things also helps quite a bit. Or having those people write down as many of the details as the can remember before they retire. Unfortunately, those sort of lessons learned documents tend to be proprietary information. So they only help some people…
Perhaps, instead of “remember” I should have said “has the experience to avoid a bunch of mistakes and to get the job done efficiently. In 1989, Martin-Marietta laid off 3500 engineers. They hired quite a few bright, young engineers straight out of college. While, in time, these people would learn what pitfalls to avoid, management left no one around to mentor them. The result was a series of costly mistakes, including the crash of a Mars probe because no one thought to ask JPL what units were being used, English or metric! I won’t waste the bandwidth with other examples at other times and places. But experience can tell you what to watch out for, even if you are dealing with new, cutting edge technology, and especially if you are doing things that HAVE been done before!
I have noticed that with age just about everything I do- professionally, or personally- is a LOT easier. Very difficult projects? Not so much. And, it is easier to make money (I’ve been self-employed by whole life, just about).
On the other hand, I also find that the “give a sh*t” attitude is similarly gaining speed…
NASA isn’t the only incompetent space organisation – look at the history of the Russian ‘Nauka’ science module for the ISS. Sometimes organisational cultures become so dysfunctional that they really need to be put out of their misery.
More like pruned than taken out back and shot. Remember NASA is still pivotal in funding spaceX and other innovative commercial entities and in performing pure science business doesn’t care for.
By ‘funding’ do you mean ‘buying services at market rates?’
Of course.
Ok, ok; my comment wasn’t meant to be snarky! Though looking back it’s hard to tell.
I responded as I did because there are folks hereabouts who will consistently posit that SX is sustained via NASA ‘handouts’, and that indeed the entire history of tech development in Hawthorne is the product of NASA largess.
These claims really irritate me, that’s all, engendering as well a kind of hyper-vigilance that force apologies.
It is certainly the case that NASA is supporting more than SX. Among the often neglected benefits of ISS is the provision of a place to *go*, requiring a supporting industry. It is critically important.
While it is always fun to find opportunities to throw barbs at SLS, this is a stretch. The article says a QA inspection did what it was supposed to do. Sure, it would be better to have avoided the contamination, but this appears to be a routine find.
OTOH, the “routine find” attitude breeds something else: complacency. We’ve seen it in SLS, JWST and a laundry list of other vampire projects.
For a mass-produced assembly, yes. For bespoke rocket hardware that is quite literally magnitudes more expensive than any conceivable commercial alternative, it is a display of comical ineptitude to allow such a basic problem to wind up in a final assembly.
For SLS, it’s akin to driver’s side door handle falling off in your hand moments after taking delivery of a Lamborghini Veneno.
These sort of random setback are almost inevitable… which is why its important that you minimize the technical and schedule risk that you can avoid.
Had we gone with a smaller x3 rs-25, x4 srb segment vehicle we would have avoided all the challenges that have arisen strengthening the core to take 5 seg SRB’s, strengthening the launch tower to take the launch stack, selling off the service module to europe cause we needed the money for those acursed 5seg SRB’s, and it could still basically perform the same mission of delivering Orion and ICPS to LEO (which would in turn deliver Orion through TLI so that Orion’s service module can capture into DRO. Seriously block 1 is an overkill waste who would need 95tons to LEO betfore 2030!?)
Let this be a lesson in not making the good the enemy of the best not that the cargo cultists in nasa ever learn this lesson they didn’t learn it after IPP failed, they didn’t learn it after constellation failed, and they won’t learn it after SLS fails…
“those acursed 5seg SRB’s”
Or, heck! Coulda eschewed those dangerous outboard riggers in favor of designing a new and modern rocket motor.
[Which will be needed at some point; I do not know if there is any sort of extant agreement or even RFP to design and build a new motor when the Shuttle engines are all splashed].
You have a point but back in 2004-5 you wouldn’t be getting anything else ready in time for the shuttle retirement other than reconfiguring the shuttle components a different way and making it a little safer with a launch escape tower.
But those 5seg’s really are cursed. 5seg SRB concept had been floating as a shuttle upgrade for ages, but was never undertaken because it would mean a major structural redesign of the ET naturally that isn’t going to change if you try to implement 5seg’s on SLS instead of STS. The structural stress the larger boosters caused meant using thicker materials than they were used to resulting in the core stages welding problems and associated delays we’ve experienced.
Its also because of the 5seg SRB’s that the Ares programs were projected to take so long they were a major pacing item to develop and prove before you could build much of the launch vehicle. If you line up the Ares rockets realistically projected time table to SLS you’ll see they are basically the same rocket, and right on the projected schedule and budget :/.
Any smaller vehicle design, could not have met the NASA requirement. “SLS SHALL be larger, carry more, a larger payload, than Atlas V or Delta IV” otherwise, it fails to create and retain jobs at the right NASA centers and contractors.
Jupiter j-130 (the one with 1.5 stages, 4 srb segments, 3 core engines, and no upperstage) is at least 60tons to LEO. Stick a DCSS/ICPS on top and thats 25-30tons to TLI, and it had growth options for even more by adding a larger clean sheet upperstage and a 4th core engine no way Atlas or Delta comes close to that.
No Ares was a scam to intentionally put nasa in a pickle of being over schedule and over budget to try to force a larger budget, and SLS is no different.
how does “.. it was leaving residue in the tubes” translate to “full of paraffin wax”?
You’re right about things going wrong on just about any spacecraft or rocket. But I can’t say this is the process working exactly as it should. The contractor should have cleaned the parts (I assume from the coverage, that was specified in their contract.) The inspection should have happened on delivery, and it sounds like they were into assembly before anyone checked. They did catch it, and they are fixing it, so it wouldn’t be fair to call the process a failure. But I wouldn’t call it exactly right and a complete success.
In point of fact, NASA (or their contractors) don’t let people knock on the door and sit in on reviews either. The difference is having to report to certain advisory committees and panels. For minor mistakes, they’d get fixed before there was a need to report them.
I don’t call the advisory committee or panels a “review” because it’s done at a much higher level that most reviews of NASA projects. You do not hear about every problem at that level.
As far as BFR is concerned, as long as Mr. Musk owns controlling interest and unless his customers insist, I don’t expect to see any similar reporting. Blue Origin is even less transparent than SpaceX.
When it comes to image and what would happen if NASA tried something several times and failed several times before they got it right? Well, NASA does try to project an image of infallibility. They should expect PR problems when that is proven to be incorrect. In contrast, SpaceX (and Blue Origin) only have to answer to the people paying the bills. That’s mostly Mr. Musk in the case of SpaceX, but he has investors and customers to worry about. In the case of Blue Origin, that’s all Mr. Bezos.