Pieces Are Falling Off Of James Webb Space Telescope
JWST suffers new problem during spacecraft testing, Space News
“In a presentation at a meeting of the National Academies’ Space Studies Board here May 3, Greg Robinson, the JWST program director at NASA Headquarters, said some “screws and washers” appear to have come off the spacecraft during recent environmental testing at a Northrop Grumman facility in Southern California. Technicians found the items after the spacecraft element of JWST, which includes the bus and sunshield but not its optics and instruments, was moved last weekend from one chamber for acoustics tests to another to prepare for vibration testing. “Right now we believe that all of this hardware – we’re talking screws and washers here – come from the sunshield cover,” he said. “We’re looking at what this really means and what is the recovery plan.” The problem, he said, was only a couple of days old, and he had few additional details about the problem. “It’s not terrible news, but it’s not good news, either,” he said.”
Keith’s note: “It’s not terrible news?” Really, Northrop Grumman? The spacecraft was designed such that every part was included for a reason, yes? If the parts are falling out during routine ground handling that means something went wrong. After how many years of delays and billions in cost over runs, Northrop Grumman can’t even keep bolts properly tightened on the spacecraft?
Man, this is so messed up…
On one hand this does look bad. On the other hand, that’s why you do this sort of testing on the ground. On these “it has to work right the first time” types of programs, it’s better to find these issues now than after you launch the thing. This isn’t Hubble. There are no servicing missions planned to fix problems.
The folks from Hawthorne and Kent might be able to do a service or retrieval mission in the future. Of course it is better if the JWST functions properly to begin with.
If the real James Webb was at NASA he would have cancelled this disaster long ago.
The first question is, did these actually come off of the spacecraft (parts of the spacecraft that came loose) or are they debris left over from the build. Both aren’t good but one is worse than the other.
Exactly. My guess is these were left behind during integration.
They were not installed and torqued, but forgotten, somewhere the tech placed them while assembling the SC. These were undiscovered FOD (trash, debris) that shook loose during vibration. Very sloppy! Did they not count for each fastener that went into the integration procedure? How else do they end up with extra parts floating in the spacecraft? Sheesh.
I’m sure that there are duplicate or triplicate signatures by different inspectors saying that yes every screw and every washer was counted. People get bogged down in processes like this and end up signing off without doing a thorough check (even unconsciously). Doubly true if they know that there are other inspectors that have to sign off on the very same work.
There are tricks to avoid things like that, but I’m not sure how they work at the screw and washer level. Of larger, “remove before launch” items (instrument covers, spacers and clips to keep folded solar arrays from slapping together during transport, etc.) they paint them red and have also have a board with cutouts for the parts. The board is painted red and the cutouts are painted green. When removed, every part goes in the appropriate cutout, and if everything is accounted for, the board is a single, solid color.
From what I remember they had the same problem with Shuttle and they never could solve it no matter how hard they tried. They would keep logs and have people double-check, etc. yet whenever they reached orbit and opened the cargo bay doors stuff would still float out.
Was this an IQEA design? I always have parts left over after building book shelves or end tables from IQEA. Seriously though, good grief! But better to have found this while still on the ground.
I remember seeing some stuff like that floating away the first time the Space Shuttle opened the payload doors.
One small piece of debiris, from whatever sources hits the optics or jams in the deployment mechanism and game over. $ 8 B for this – so much else that could have been done.
Optics are more robust than you might think. A disgruntled (or insane) employee at McDonald Observatory once emptied a 9mm pistol into the primary mirror of their 2.7 meter telescope. Since the holes were large compared to the wavelength, they just polished the edges and continued to use the mirror (now with a few square centimeters less collecting area.) Of course, that’s not a recommended practice.
At the same time, it’s a little scary, or at least a little odd, how many parts can fall off without causing a problem for a spacecraft. I’ve seen flight instruments experience at least four different soft short circuits while continuing to operate as if nothing had happened. For a missing bolt, well, often someone decided on a mechanical design with more bolts than necessary. It’s risk reduction in case one of the bolts failed or a nut worked loose. (“They do not preach that God will wake them, just before the nut works loose.”) That sort of design philosophy means things can go wrong without catastrophic consequences. Again, that’s not a recommended practice.
But I wonder about how many things slip through and we never find out because (1) the spacecraft works anyway and (2) we never get to inspect it after launch.
Most serious contingencies are the result of unanticipated failure modes, so it is worthwhile to know what happened and correct the vulnerability in future designs. Nowadays onboard cameras might allow some degree of inflight inspection at reasonable cost, and a more evolutionary design strategy with more frequent missions based on evolving designs might reduce risk and consequently testing costs.
Don’t forget the 150 ton gorilla that will soon be entering the room.
By the time JWST is finally launched the BFR will probably be available or close to entering service. I am sure they could pay for SpaceX to go fix it, or even return it to Earth if needed. It’s dimension and weight are well within the BFRs ability.
The BFR also raises the possibility of a future Hubble servicing mission, or even returning it to Earth for a refurbishing and then an orbit return. Or just place it in the Smithsonian and use BFR to deploy a bigger and better Hubble 2.0.
That is the great thing about the BFR, when it enters service it changes everything. You want a radio observatory on the Lunar Far Side? No problem, just call Elon Musk to build it. Want to add a segmented 30 meter telescope to the Lunar Observatory, no problem, the BFR will transfer the components and workers needed to build it.
I agree. The original concept for the ISS (back in the Seventies, I am that old) was that large and complex payloads like this could be deployed and checked out at the ISS in an unpressurized “hangar module”, then ferried to their operating locations by what was then envisioned to be a reusable space tug.
It was a different time! The original concept for SIRTF (nee Spitzer) was stuck in development hell starting in the 70s. The idea then was a serviceable mission, where “servicing” didn’t mean flying to it, it meant bring the spacecraft back to the space station for cryogen refill. The artist’s conception had the bus-sized spacecraft dwarfed by what looked like Babylon 5, with something like a loading crane extending and pulling it into a hangar bay.
What a different view we had of space in those days! Like you, I saw a huge open hanger just for that purpose. And a true Space Station fitted with a galley and common room (nee Alien) where the crew would sit around and grumble.
Then, of course, we learned just how pricey space would come to be.
And now we have learned that those prices are subject to huge revisions- and downward! It’s a great time to be alive.
So true. We haven’t even begun to imagine the missions BFR will enable.
We have a “stick” mentality.
Yes. The Cassini-Huygens spacecraft had a mass of less than 6 tons. Now imagine a similar spacecraft to Neptune (I believe that is the planet fcrary is interested in) weighting in at 140 tons (allowing for crew on the BFR to deploy and check it out).
It is deployed at EM L1, is fully checked out by the crew on the BFR, and then gently sent on its way with an ion or plasma drive accelerating it at 1/10 G.
Imagine it having huge antennas for high data transmission rates. Large solar panels for the ion drive that will take it directly to Neptune and then will help power it at its destination along with a few RTGs and maybe a reactor. It will have a dozen of so landers to land on the Moons (maybe a Triton rover?). Maybe a half dozen or so small satellites (only a ton or so) to put in separate orbits for gathering data (maybe one in the Sun-Neptune L1 and another in the Sun-Neptune L2 to study its impact on the solar wind, etc…), a few probes to launch into its atmosphere. Let’s see, how much of that 140 tons do we have left… Maybe room for a few more landers?
And if 140 tons is not enough you could add another 150 tons (second BFR) for only $15 million or so.
Yes, that is the paradigm change waiting for planetary science when they don’t have weight limits or have to unfold their craft after a stressful launch and hope everything works.
It will be like going from shipping via a mule train to a Union Pacific freight train on the old western frontier. 🙂
That isn’t exactly how I’d do it, and Uranus and its moons are pretty interesting as well. But those are details. Quite a few planetary scientists are very focused on flagship missions. I’ve been wondering how you can have a flagship without a fleet. What you’re describing is at least a fair-sized task force.
Yes and one of the advantages of the BFR if it lives up to specs is that you get to do all of the above. But as you noted it would take a change in mindset in how spacecraft are designed.
One example is the choice of Plutonium for RTGs which is based on its power density because of the weight limits with ELVs. When the weight limits are relaxed you have other isotopes to use, ones gathering dust at nuclear waste sites. Sure they are a couple times heavier per kw, but they are cheaper and you have lots of the isotopes to use. You have lots of options when you have 140 tons to work with.
If I had that kind of mass to play with, I’d probably give up on RTGs completely. At that point, an actual reactor starts making sense. And with that mass, maybe even a slow rather than a fast one (in the sense of neutron energy) and/or one using some other than NaK as a coolant.
Highly toxic and hypergolic fuels would also be on my list of things to get rid of. Honestly, there is a long list of things we’re currently doing the hard way because the easy ways take too much mass.
Yes, and just use the RTGs on the landers and satellites it would deploy. You also might want a pair of reactors just in case one malfunctions. But yes, when you have no strict limitations on mass, and low G acceleration, it really changes how you design your systems.
BTW thinking more about it you would probably have around 290 tons or so to work with. The reason is that I expect, following post Challenger guidelines, you would want the propulsion unit delivered in an uncrewed BFR. No big deal, just an extra $10 million, so your launch costs go from $10 million to $20 million.
So you would have the full 140 tons for your spacecraft, plus the portion of 150 tons in the second BFR not being used for the propulsion unit. Of course you would want enough fuel for the ion/plasma engines so that you wouldn’t need to worry about gravity assists and maybe even think about skipping a Hohmann Transfer orbit.
I’m not sure if I trust some of my colleagues to take advantage of that. Consider a scientist who would like to fly a 5 kg, $5 million dollar instrument. If someone said there was a really cheap launch vehicle out there, and the spacecraft could allocate 20 kg, I’d be inclined to dump all the time and expense of high optimization and miniaturization, and fly an equally capable, 20 kg and $1 million dollar instrument (all numbers are made up as an illustration…) But I know people who would insist on a much more capable, 20 kg and $20 million dollar instrument.
If BFR works as planned (and I’m still waiting to see), NASA science would be in a very different regime. With cheap, heavy lift, the cost of building spacecraft according to current practices and expectations would be the bottle neck. The idea of flying a spacecraft, having it fail, fixing the problem and reflying, is something many people involved in planetary science have trouble accepting. But that’s also something cheap, heavy lift would make practical.
I’ve asked that question more than once. This isn’t just for the benefit of future missions. Sometimes the deployments don’t work or don’t seem to work.
One of the booms on one of the MMS spacecraft jammed at about 95% of full extension. That didn’t affect the measurements significantly, other than requiring some changes to data processing on the ground. But it delayed deployment of the other spacecraft’s booms and quite a bit of time and effort diagnosing. In another case a friend was involved in, an antenna did deploy but the switch which should have signaled full deployment and latching in place failed. To figure out what happened they had to check things like the instrument and spacecraft attitude control telemetry. (Did it wobble when the spacecraft turned. Did the spacecraft’s center of mass shift in the predicted way? Etc.)
When things don’t go as planned, it would be really nice just to have a cheap, smartphone-quality camera to see what happened. In the case of deployments shortly after launch, it doesn’t even have to be a camera rated to operate in space for more than a week or so. And, if it works, it would look really cool. The public relations value would be tremendous.
When I ask, the sort of answers I get don’t really satisfy me. If you design it to the same standards as the rest of the spacecraft it would be too expensive to be justified. If you don’t, why are you flying something which might not work. And if it doesn’t work, that could add risk to the mission. And even studying the possible risk costs too much for something which shouldn’t be necessary in the first place. Etc.
This would end up being another example, like the $5m 5kg instrument being turned into a $20m 20kg version instead of a $1m 20kg version.
The temptation would be too great to add more and more requirements to the camera, until it competes with instruments for funding, and then to cancel it when something else goes over-budget.
I’m sure some people would be tempted to do that. That’s what project managers and PIs are for. They should be dictatorial enough to just say “no”, stomp ideas like that out like a bug, and keep things under control. It should be possible to add a few hundred grams for a camera to verify and diagnose deployments, without letting it grow into a subsystem-level monster.
But look at your own experience. “Ideas like that” are SOP.
It “shouldn’t”, but it is.
Which is one of many places where there is room for improvement. Accepting poor practices, and saying that’s the way things work, simply makes poor practices permanent.
But the optics aren’t the issue, it’s the sunshade. That sunshade has to work, or the science mission is lost. It must deploy, it can’t tear, or thermally short. JWST is littered with single point failures. I am astounded at how many moving parts this thing has. I have been on missions where we argued literally for years over whether even one moving part was an acceptable risk.
Well, even one moving part is probably an exaggeration. Propulsion systems need valves, many types of instruments have small, moving mirrors or covers which open after launch.
But, yes, big, deployable structures are real concern and JWST has lots of them. In comparison, Europa Clipper has five (two solar arrays, two radar antennas and a magnetometer boom), most of which are multi-jointed. By the standards of planetary missions, that’s a very large number and something I understand the project is being very careful with.
It might be bad luck to ask, but I wonder how perfectly JWST’s sunshade has to be deployed. Infrared telescopes have operated, at a with degraded data, at off-spec temperatures. Even if the JWST deployments aren’t 100% perfect, I suspect we’d get more than 0% of the science.
You only have to look at Northrop Grumman’s own video (below) portraying the launch and deployment of this spacecraft – then you realize just how absurdly complex and almost ‘Rube Goldberg’ the whole thing is! When it launches, I’m going to be terrified for all the people who worked on this thing for so many years. It wont be over for many days, long after the thing launches and leaves Earth orbit – then there’s the deployment and checkouts. If the telescope fails; all the anti-science and flat earth dingbats will come out of the woodwork and HOWL for all their worth – making funding for expensive space telescopes precarious forever after. This thing simply HAS to work…
https://www.youtube.com/wat…
I agree but nobody listens to me. However it may work.
It should work. It is not that different than sigint birds like the Rhyolite/Aquacade/Orion/Mentor series in GSO in the way they deploy their antennas/sunscreen.
Unfortunately, that may not help. Hubble had a few problems (a thermal/pointing stability issue comes to mind) which were known from remarkably similar telescopes which pointed down. That information didn’t get conveyed because the spacecraft in question were classified projects; the HST people had to find out and solve it on their own. The same sort of thing happened with the Deep Space Two mission.
“absurdly complex”
We said something similar before Curiosity’s “seven minutes of terror”, remember?
I have a lot of confidence in these techs, engineers, and scientists.
I was one of the people who said that about Curiosity. I’m still saying one success out of one attempt could just be luck. And, for Mars 2020, they have made a fair number of changes to the entry, decent and landing systems. That includes things to increase reliability as well as things to improve landing precision.
Actually “absurdly complex” barely begins to cover all the things that have to right. The deployments alone take something like a month to complete and there are well over 200 separate deployment events that need to go correctly. That scares the bejeezus out of anyone who has actually designed and built spaceflight hardware or it should. Yes all the competent techies are working hard and doing their jobs, but in such a large and complex system, there is a lot that can go wrong.
My feelings exactly. NG Quality Control has let us down recently. They designed their own special payload separation system, because the one SpaceX would have provided, was not good enough for them. And the ZUMA DoD mission a total loss…
Fast forward to this telescope. “Absurdly Complex” I give it a 50/50 chance of working…. If it fails, it would be the biggest, most expensive screw up in space technology history. No pressure NG.
But, I do hope I am wrong, and everything happens just like the video animation.
If I am I will come back here and eat me some crow.
This is what comes of having undersized too-expensive launchers and no on-orbit assembly and checkout capability. BFR will sweep all of that away.
The only real solution is for NASA to launch Webb aboard the BFS with a SpaceX flight crew and NASA mission specialists, and man-tend the deployment (and recovery for relaunch if it fails). Refuel the BFS in
LEO to allow it to travel to the L-Point and back. Put an RMS arm and a DEXTRE clone on board the BFS with the backup option of a manned EVA or two. The whole project is so far behind schedule that waiting for BFS isn’t going to hurt – and, is it sensible to think of launching on the last Ariane 5 anyway?
Oh, it will hurt. Remember, JWST has blocked the funding lines for any other major astrophysics starts for the last decade. As long as it is on the ground, US astronomy is spinning it’s wheels, while the science continues to march on. Many astro communities are waiting for this thing to get the hell out of the way.
> Problem:
Legacy aerospace company living off fat government contracts, offered by politicians who rely on the same fat legacy aerspaoce company leadership to tell them the truth, or the cash flow spigot gets turned off for gross incompetence.
Politicians see political value in preaching the high-quality jobs the fat legacy aerospace companies have in their districts – i.e. re-election marketeering.
> Solution:
Fire Congress. Vote in term limits for all members of Congress.
Demand accountability for taxpayers by new Congressional members.
No Buck Rogers, no bucks.
Either fix the issue(s) by this deadline, or the funds are cutoff. Fire anyone who contributes to the problems – like poor quality control illustrated here in the latest JWST debacle – and make certain their replacements have one or two chances to fix the issues, or they are similarly unemployed.
Watch how fast things turn around, come in UNDER budget, and our taxpayers finally get real value for their taxation burdens.
OK, I’ll follow you down this rabbit hole, but only once. The primary argument against term limits is a simple one: we already see the folly of electing to important positions folks with no ability to understand required duties and responsibilities.
By default in this scenario the repository of institutional knowledge – at least in terms of NASA, many here on this web site understand the critical nature of this knowledge. It’s true that the same knowledge can act as a brake on change. This isn’t always a bad thing.
So you’d have newcomers coming to Congress with very little ability to govern. They would be heavily motivated to write and pass naive legislation heavily informed by political persuasion. We saw this recently in Kansas, for instance.
Bottom line: real power in Washington would devolve to “staff”. A real “shadow government” would hold the real reigns of authority.
And finally: at the height of her power, Roman Senators were not able to even stand for office until they’ served in a succession of lesser offices, each design to import some level of personal authoritative knowledge. Sure it was abused. But at peak, Rome as an incredible exemplar that lasted several centuries.
Years ago Michigan instituted term limits in our legislature and it’s been a disaster. There’s no institutional memory as to what works, what doesn’t etc., and also no accountability since they know they’ll need to move on about the time bad policies implode. Doesn’t which party is in control
I do not disagree with you – I simply cannot find a decent middle-ground alternative in the current environment.
Your points are well understood and taken – clearly my example is an extreme. But, no more extreme than the fiscal irresponsibility that is in place now in DC, which is leading to the outcomes the post is about.
And I see your point as well. When I see some of these people serving 30+ and 40+ years I wonder if there isn’t a better way.
But we keep electing them. I guess we like them.
I can’t really add to anything any of the rest of you have said, except to say…
WHAT?!?!?
Poor Northrup-Grumman. Pieces fall off an $ 8 billion spacecraft when they aren’t supposed to , and a $ 1.5 billion spysat stays attached to its SpaceX booster when it’s not supposed to ( Zuma ). But it’s nothing that some space-qualified WD-40 or Loc-Tite can’t fix…