Boeing's Starliner Mission Flops Due To A Broken Clock
Larger image
Keith’s note: Boeing’s Starliner was launched on time this morning. ULA gave it a perfect flight up to the point where the spacecraft separated. Then things started to go wrong. A planned engine burn did not happen because the spacecraft’s clock was wrong and the spacecraft thought that it was somewhere else. Boeing tried to do a burn to fix the situation but a gap between several TDRSS satellites meant that the command would have been sent too late to allow the mission to have a chance to reach ISS. Boeing says that it has no idea why the clock was wrong. The curent orbital path will bring Starliner into a position to do a landing at White Sands in 48 hours. In talking about this problem NASA and Boeing tried to spin the mission as a success even though a prime objective was to dock with the ISS. It is too soon to know if a repeat flight to accomplish the original objectives will be required or if the next flight – with a crew – will be the first time that a Starliner docks with ISS.
One thing that was rather odd today: as soon as things started to go wrong NASA went dark. No TV, no meaningful updates. They said to go visit Boeing’s website which had no information. After a couple of hours information started to emerge – not from NASA PAO or Boeing but from Jim Bridenstine’s personal Twitter account. Indeed @NASA and @BoeingSpace were mostly mute. It is certainly good that the NASA Administrator has the personal capability and intent to inform the public what is going on. But I have to say that in the 25 years I have been covering NASA I have never seen such a news blackout drop into place for a launch or landing – and that includes the loss of Columbia.
Of course this is Boeing’s mission so NASA is somewhat in the back seat in terms of PR. Boeing has none of the cameras on their flight like SpaceX does – just video images of motionless people with headsets staring at monitors. Boeing had to be ordered by Bridenstine to live webcast their recent parachute test (which also had problems that Boeing tried to ignore). And Boeing is lobbying Congress for the SLS Exploration Upper Stage which they claim will be used for Artemis III despite public statements to the contrary by the NASA Administrator. One certainly has to wonder if Boeing is going to exhibit the transparency and honesty one would expect as they continue to receive billions of taxpayer funds. Moreover NASA is asking taxpayers to foot a fast paced effort to return to the Moon. You’d think that someone at Boeing and NASA would get the bright idea that stonewalling and avoiding the media is not the way to garner public support.
Maybe it’s me but I can’t help but hear an etherial voice whisper: “Boeing 737 Max 8! Boeing 737 Max 8!”
And don’t forget Boeing being fined $4 million by the FAA for installing parts without checking for defects on 133 B737 NG. Yes, there are some problems at Boeing, ones that need more than a PR fix.
NASA administrator Jim Bridenstine said ‘a lot of things went right and this is why we test’.
He said the space agency aimed to be as transparent with the public as possible throughout the issue, informing the public and the media early on.
‘When there is something that is a challenge we will be clear and transparent about it and share information as early as possible. We have done that and will continue to do that,’ he said.
https://www.dailymail.co.uk…
I’m wondering how this is consistent with going dark as soon as a problem arose. To be fair, SpaceX has a habit of doing this, as well. When something goes wrong, their on-camera staff hem and haw and change the subject. For the $6.8 billion we put into these two craft, we deserve transparency from the start, not just when the administrator declares it.
Daily Mail links are useless unless you are a subscriber. In fact, most major UK paper’s websites are behind paywalls.
Unlike U.K. papers such as the Times and the Telegraph the Daily Mail is not behind a paywall and while not a subscriber myself, I have never had any problems accessing the paper’s website.
Regarding this link, it opened for me on the first “click”.
Must be a region thing – people in NZ and Australia can’t get through to these links. Never mind – it is what it is…
Interesting.
I’ve run into “foreign/copyright” restrictions now and then preventing me from viewing certain American TV Network videos (I’m in Canada); but I didn’t know their were regional restrictions on other websites.
The Guardian isn’t. That’s why I send them checks on Christmas.
Another fine English newspaper.
I’d probably take issue with 2 things. (1) NASA is somewhat in the back seat in terms of PR: NASA should let Boeing take the lead but NASA insisted that they would be a full party in controlling the mission-it was flown out of the NASA mission control with what NASA called a joint control team; NASA and Boeing;part of the control team is public affairs. NASA is in part to “blame” for not being able to bring their TDRS satellites up to ensure they could communicate once the spacecraft reached orbit. That should have been a back-up right from the start. And NASA is partly to blame for the drop out in public information. (2) Boeing has none of the cameras on their flight like SpaceX does: At the press conference one of the contractors, Boeing or ULA, said they were getting live video of the spacecraft during separation and that it showed the Boeing spacecraft’s thrusters firing. But I guess they felt the public did not need to see it. I do wonder though what else they were not showing us?
I don’t know the details, but if there was a gap in TDRS coverage that could have been filled, that was a big failure in contingency planning.
In other contexts, there have been incidents and accidents due to too much contingency planning. If you go overboard, there are so many flight rules and constraints that you can’t do anything without waiving a few of them. Sometimes, people get used to that and get careless about which ones they waive. I’m not saying that’s the case for this incident, just that it’s known a path to a problem.
“NASA is in part to “blame” for not being able to bring their TDRS
satellites up to ensure they could communicate once the spacecraft
reached orbit”.
–
No. NASA doesnt “bring them up”. TDRSS satellites are where they are and always “on”. If there are any coverage gaps (I dont think there are any in the present day) or weak coverage areas then those are well known by the engineers.
Not Boeing’s year. I am sure they are eager to get 2019 over and get a fresh start on 2020.
Two significant QA problems in two major Starliner tests in a row?
That looks like a sign of an organization whose QA paperwork showing all required checks & inspections have been properly performed is in immaculate order – but which didn’t rigorously insist on the actual checks being performed before every form was filled out.
That sort of organizational problem is NOT an easy fix.
This reminds me of a scene from the movie “Brazil”. At one point, the main character didn’t want to let a couple of technicians into his flat. He asked if they had a properly filled out form 4654 (or some number). One of the technicians went nuts and started to beat his head against a wall. Paperwork and documentation is useful, but at some point it becomes harmful and drives people crazy.
Your whining about information is a bit over the top; leave that to Millennials please.
Put your engineer hat back on: Something weird happened and one needs TIME to figure out what just happened.
Speculation only makes the problem worse.
There’s something NASA learned from the Apollo 1 fire. Speculating is a bad idea, but a media blackout can be worse. Saying `It does not appear to be on the correct orbit, but it is in orbit and the flight team is working on it” would not be speculating. The problem with saying nothing is that, in the absence of _some_ information, the press and the public will speculate.
Not only were they open about the Apollo 1 fire, they provided lots of data open to the general public including source code of the guidance computer (and computer people been recreating the AGC ever since). These days it really difficult to find anything techie, i.e. a schematic when first seeing it gives that “ah ha! that’s how it is done.” Of course don’t want to release classified info but totally dumbed down diagrams are a non-starter for young people with interest in space systems.
That doesn’t quite match my reading of history. NASA’s initial reaction to the Apollo 1 fire was to lock down and say nothing. For the first day or two, they said as little as possible. And virtually everyone yelled at them for it. Congress opened their own investigation of the fire, because NASA’s initial, closed-mouth response made them suspicious they’d try to cover up a mistake. It was only after that criticism that NASA opened up, in the way you describe, and to their credit, they did that promptly.
I think the Administrator himself, has kept everyone updated. I have liked him since the get-go. Bravo Zulu, JB. I agree, they cannot say any more than that. It takes time to look at the data and reproduce the anomaly. I also agree 100% there should be all kinds of cameras on this capsule and rocket. Inexcusable. We want to see the views. We are paying for this.
Wrong – Speculation doesnt “make the problem worse”. The problem has occured and is what it is and it is surely being worked. Taxpayers have a right to speculate when something they paid for didnt work. You may be thinking of the 1950s when no one questioned anything – many examples of why that was myopic. Real engineers and scientists welcome thoughtful scrutiny of their work. No one is the smartest person in the world.
for this Boeing is getting twice the funding as SpaceX?
Boeing is getting 160% as much as SpaceX. That’s to do it following a process NASA is familiar and comfortable with, and which NASA is confident will produce good results. (Do I get extra points if I can say that with a straight face?)
Boeing has none of the cameras on their flight like SpaceX does – just video images of motionless people with headsets staring at monitors.
——
I didn’t watch it live but the Boeing YouTube channel had a bunch of graphics, video of the rocket, etc. They had a couple of talking heads like SpaceX and even did an interview with one of the engineers at T-5 minutes.
I thought it was pretty good
So to review:
-Boeing failed one of its parachute deployment tests, but called it a success. Space X insists on 10 parachute test with 100% deployment.
-For the crew escape Boeing relied on one flight from the ground and everything else on paper calculations. Space X had multiple launches and will test launch the far more difficult task of launching in the air from maximum dynamic pressure which will destroy a Falcon 9.
-Boeing hides and spins its failures while Space X is open.
-Boeing wants a huge undeserved part of going back to the moon guaranteed regardless of results.
-Boeing is part of the SLS fiasco.
-Bridenstine subjected Space X to an extensive safety inspection because he did not approve of Musk smoking marijuana. Boeing skated.
-When Musk revealed the Starship after using his own money for development, Bridenstine made snarky comments about Space X slowing the commercial space program even though they were ahead of Boeing. Bridenstine said nothing about Boeing.
-Boeing is not only getting much more money for the same job than Space X, it also received a huge bonus just for not withdrawing that Space X did not get.
-Boeing will charge more for flights than Space X.
-Based on demonstrated capabilities, Space X has far more potential for going to the moon and Mars at far less cost and a much sooner time than Boeing.
What will happen at the conclusion of these tests? I hope that Space X will be declared the clear winner and picked going forward. I expect that Boeing will get 50% of the business.
Yes, SpaceX actually tests while Boeing does mostly simulations.
Another pedantic comment. In the field, it’s called analysis not simulation. That includes theoretical work on paper, while simulation implies running code on a computer. But I agree that either one is a poor substitute for actual flight tests.
…and Boeing did it for $4.2B while SpaceX did it for $2.6B.
The odds of SpaceX getting 100% of the operational missions are basically zero. Officially, Dragon 2 and Starliner have different capabilities and limitations. Once both are certified, which one will get which flights will be based on the payload manifests and the schedule for ISS support. It’s looking like Dragon 2 will be certified well before Starliner, but that won’t drop Boeing’s share of the flights to zero. Cynically, I have a gut feeling that the people who decide on the payload manifests and schedules (and their managers) have preferences. And they have plenty of wiggle room to throw more flights to their preferred vendor. But nothing as raw as zero and one hundred percent.
Boeing’s regulatory (contractor?) capture seems too strong for it not to get its full six operational flights, even if the stumbles continue. Still, it would be savory to see SpaceX get at least a couple yanked over to its manifest.
More importantly, when it comes time for Phase 2 of Commercial Crew, it would be swell to see Dream Chaser get a strong look-in.
“Sounds like an easy fix so far no?”
All right, but I am from Missouri. They should still demonstrate docking with the ISS.
Why? It worked in the simulations…
You said it before me. On another thread, about SLS, I was thinking of posting a similar comment. Concerning Boeing claims, Missouri, and some things President Truman said.
Thanks for the informative update!
They still need to get the capsule down safely. Recovery at WSMR is going to take some precision flying. I hope it’s up to it. The other option would be an ocean splashdown, if anyone is looking at a plan B. But with a QC error like this you have to wonder what other errors occurred. If I recall the parachute anomaly on the Pad Abort Test was due to a bolt being loose.
What I fear is how some are talking about putting crew on the next flight. It sounds like “go fever” is developing over the pressure to get astronauts to the ISS on an American system.
Just to be pedantic, I don’t think the parachute problem on the short test was a quality control issue and it doesn’t look like this incident was either. As I understand it, quality control is about the parts; how they are manufactured and whether they are up to the specified standards. A perfectly good bolt which wasn’t installed properly is a flaw in pre-flight preparation processes, not QC. We’ll have to wait on the current incident, but if they told the flight computer to do the wrong thing and it faithfully did so, that’s not a QC issue either. Not that those incidents don’t imply serious issues. But in defense of being pedantic, you solve the problems by identifying and correctly describing the problem. And sometimes that means getting more that a bit pedantic.
Technically, assembly and testing also falls under quality’s jurisdiction as well. In most assembly/operation procedures I’ve done, there’s always a quality department watching the technician tighten a bolt to a)make sure it was installed and b)torqued correctly.
I see your point, and I guess I was making a distinction between production and use. In operations, yes, I expect someone looking over people’s shoulders and making sure no one slips up. But I’m not used to that role being called “quality control.” I hear that term used for production. By analogy, if I went the the hardware store, bought a door knob (mine has a latch which has started sticking), took it home and installed to improperly, would you call that a quality control issue? I guess you might, but using that term would imply something wrong with the part I purchased, not my own errors as a repair man.
Yes, that would be an example of human error, Boeing’s favorite excuse when something goes wrong. So we now have at least two examples of human error on the first two Starliner tests. Not a good record.
At least the problems SpaceX had were design issues easily fixed by modifying the design, which is what testing is for. Human errors by contrast are a lot more difficult to fix and would indicate a problem with the way operations are managed by Boeing.
Although quality control started out with a focus on the design and manufacturing process in the 1920’s at Western Electric it has expanded to encompass operations as the U.S. moved from a manufacturing based economy to a service based economy. It turns out many of the same techniques apply just as easily to managing services as in manufacturing goods.
In any case Boeing has to step its game up if it’s not going to add to the total of dead astronauts that it’s Rockwell Unit accounted for in X-15, Apollo and Shuttle accidents. I think it is time NASA gets the microscope out and does a safety review of those working on Starliner.
Yes, I guess we are taking about terminology. Rather than calling it a quality control issue, it might be better to talk about where the problem occured. Was it the subcontractor who supplied the part, the contractor who assembled the parts or the people who used the fully assembled vehicle? We don’t have enough data to speculate on that, but that’s what the incident investigation team will have to look into.
QC does not end with manufacturing parts – it’s part of assembly of any manufactured item (or it should be).
Same goes for software in a system. QC is part of examining and reviewing software to ensure it meets the requirements set up for it.
Bridenstine said several times during the press conference that NASA and Boeing would be open and transparent. I get the feeling he was not at all happy about Boeing’s piss-poor presentation OR the hasty dropping of the curtain.
the upfront comment counter on many threads seems to be off – did Boeing provide it?
That’s a fine joke, but no. Disqus, the service which handles comment on this and other sites was been a bit buggy for the past week. I hope the people responsible for the affected web sites have noticed and complained. (That’s a hint to KC.)
The press conference was an exercise in spin control. The question is will NASA require Boeing to re-fly this mission.
Probably, since they didn’t get to the ISS to practice prox ops.
Many in “Old-Space” seemed aghast when SpaceX started up a YouTube channel and were far more up front than anyone before that “Space is hard” to the point they have a video montage of all of their boosters crashing leading up to the beginning of a long line of successes. There have been some catastrophic failures, but the most notable: explosion of a carbon fiber COPV due to ingress of liquid oxygen condensing on the slush oxygen tank and detonation of a check valve from the pressurized hydrazine came from failure modes many engineers were not even aware existed (i.e., Complex, quasi-intermittent technical issue, not a simple Quality Control failure). Because SpaceX has been relatively transparent during such events and been public after about what they discovered and how they were going to fix it, I think they have been able to develop a level of trust and goodwill that they are doing their best, even in the face of issues.
When Boeing develops a track record of lacking transparency about their engineering decisions and failing on the “easy” stuff, while charging more and doing less, all the time doing under the cover of over-protective congresspeople that rely on the Boeing jobs in their districts, it is easy to see why this failure is not going to get the same level of patience and understanding that SpaceX would.
The first SpaceX launch had problems with propellant slosh, a well known phenomena.
For the money ULA is getting, we deserve something resembling a contemporary media stream, and one that doesn’t disappear during an anomaly.
How could they not know this? Their minimal, missing-in-action presentation seems more consistent with that of China, or the old Soviet approach than what we’ve become used to with SpaceX.
This particular Old Space outfit is living up to the negative stereotypes in more ways than one.
“How could they not know this?”
The legacy aerospace companies often seem to have feet in hardening concrete, mostly because the entire corporate culture appears formed decades before any sort of social media availability. It’s easy to understand stand-offish behavior.
Here it is important to remember the difference between ‘reason’ and ‘excuse’.
I think you mean Boeing not ULA – there was apparently nothing wrong during the ULA Atlas V launch vehicle’s flight. Problems were on Starliner AFTER it was deployed. Deployment from Atlas V was apparently nominal and inserted into the desired spot.
A single point of failure dumps the mission ? Wow. I’m thinking there is much more than this MET gremlin.
Whatever became of triple redundancy, maybe having three MET’s running simultaneously and if two or more agree that’s a Go. For starters.
The manned mission Atlas V launch sequence is weird. Is the Lockheed Atlas V really truly man rated since it was designed to loft inanimate paylords at high G climbs , or is that another rejigged Boeing-Lockheed paper sim signed off ? Leaving the SRB’s attached empty for a full minute after they burn out is weird (e.g. the moving mass dead weight penalty alone ) . The ” flat ” orbit profile is weird. That aerodynamic skirt between 2nd stage and service module that is jettisoned seems a kludge. Not using the TWO Centaur engines to go all the way to target insertion orbit is weird.
Rosie the Astronette seems normal, though. I can only hope the remainder of the truncated mission ( ~ 36 hours as I write this ) is not full of surprises.
I agree. Why did Boeing/NASA not “ask” Centaur to take the capsule all the way to the ISS orbit? Centaur has the ability to start at least two or three times with coast periods in between. Centaur is more than capable for the job. Also, the SRB separation delay was unusual and tells me the vehicle had excess performance to carry the mass longer than it had to. May have been to avoid dropping the solids too close to the coast. Many questions….
I just read about flattened trajectories for crew flights. The Shuttle did the same trajectories. So, for crew, the booster takes the capsule close to orbit, but the capsule has to fire its engines to climb that last step to orbit. If it does not work, the capsule aborts by default. This keeps the vehicle at lower altitude longer so if anything happens on the way up, we want the capsule to reenter on the same orbit to keep the crew safer. So, I am now better informed. =)
I don’t KNOW this, but is it possible the SRB’s were left on after burnout because of aerodynamic or center-of-mass reasons? With the Starliner and the skirt, the relationship between the CM and the aerodynamic Center-of-Pressure on the vehicle might have been changed versus a usual configuration. This would probably have nothing to do with the clock failure.
From https://www.nasaspaceflight…
“Nominal Atlas V launch profiles involve the first stage – the Atlas V booster – bringing the Centaur upper stage and payload to a high altitude in just 3.5 minutes.
But for Starliner missions – especially those with a crew – that trajectory would not permit survivable aborts in the event of a launch vehicle failure.
If the Atlas V were to fly its normal profile and Starliner had to abort, the angle of reentry would be such that Starliner would either skip off the atmosphere – resulting in a Loss Of Crew and Vehicle situation – or experience G-forces during reentry that would kill the crew.
To avoid this – and permit Starliner to safely abort during all phases of launch, something mandated by NASA’s Commercial Crew Program – the Atlas V had to fly a much more shallow and lower flight trajectory.
Moreover, the Atlas V booster had to throttle its Russian-made RD-180 main engine down a lot more than it does for satellite missions to maintain no more than 3 Gs of acceleration for crew safety.
These two changes to how the Atlas V booster flew with Starliner result in a significant loss of efficiency and performance of the Atlas V and mandates the use of a Dual Engine Centaur upper stage – which needs greater thrust to overcome the shortfall in the Atlas V performance for Starliner launch trajectories.”
“These two changes to how the Atlas V booster flew with Starliner result in a significant loss of efficiency and performance of the Atlas V and mandates the use of a Dual Engine Centaur upper stage – which needs greater thrust to overcome the shortfall in the Atlas V performance for Starliner launch trajectories.”
Hence my speculation in my post above as to why they kept the SRB’s attached after they had burned out, in order to keep the mass and aerodynamic pressures matched up.
I don’t know for sure either, but…
In all other cases I know of, Delta 2 launches from Vandenberg AFB in California, the rocket carried the spent boosters several seconds past their burnout. The reason. Avoid dropping the empty cases on sensitive area of the ocean/islands. So, no need to carry the boosters for any other reason, just where they fall. Since there is excess performance, it is not a big deal to carry the dead weight a little longer.
We don’t know if the clock was redundant, if more (or some) redundancy would have helped, or if the clock itself was the problem. Without speculating on this incident, I’ll note two examples. The Ariane 501 failure involved a software error in a fully redundant guidance system. Two identical computers, a primary and a “hot” backup, were running identical software in parallel. Unsurprisingly, they both hit the same bug at the same time. In terms of a timing error which didn’t involve a bad clock, the Cassini spacecraft once lost a pretty high priority observation because of a typo in the command sequence. The sequence commanded an instrument to change modes at a specific time, and it did. But the typo was in what the specified time was and the mode change happened, if memory serves, a couple days later than it should have. That wasn’t the clock’s fault.
If they safely recover the capsule, they will be able to say the incident would have been survivable had astronauts been onboard. But, no, it’s not an easy problem to fix. For this mission, it should be reasonably easy to fix or work around. But the deeper question is _why_ such a timing error occurred, and how to make sure it doesn’t happen again. That probably won’t be an easy fix.
The timing issue suggests there are process problems with Boeing’s mission software/firmware development, verification and validation.
If I were one of the engineers involved, that’s the first place I’d look. Except for one detail. When it comes to spacecraft, the command sequence and look up tables (data arrays stored in memory) usually aren’t considered flight software. If the launch time slips and the rocket needs to know (e.g. if it uses a Sun sensor for attitude control), they can change that number in memory without formally changing the flight software. There’s still a verification and validation process, but it’s less rigorous than for flight software changes.
There were/are similar loopholes for programmed logic devices…not considered ‘software’…but as programmed-logic evolved into complex FPGAs and massive OTP logic chips, NASA has started closing the loophole with NESC-RP-09-00546 (Development, Design, Test and Evaluation – DDT&E – Process for Robustness of Space Flight Programmable Logic Devices – PLDs) along with adopting guidelines based on DO-254 (Design Assurance for Airborne Electronic Hardware).
Well, that’s just great. How truly good. I guess I missed those directives since I’ve been on the beach for a couple of years. Do you think those policies actually promote lower costs or higher reliability? I’m very doubtful of that.
The jury is still out IMO and I have serious doubts. The SLS/Orion and Starliner are the first human-rated launch vehicles\spacecraft systems to be developed by NASA and its traditional contractors since the STS…over an entire generation has passed since then…there’s really no one left who has designed these types of systems (I am leaving SpaceX and BO out of this club for now since this is all new for these companies). The existing culture is risk-averse to a fault (there’s no way to be 100% sure through simulation/analysis, and verification by testing only can take you so far). Furthermore, there’s too much allegiance to the philosophy of ‘reuse’; often, the design constraints are not well understood, because the original design and product engineers are long gone (and things were not as well documented in the old days). Look no further than the now-infamous MCAS and 737-Max flight control system that was developed for an ‘existing’ air-frame, under the requirements of DO-178 (flight software) and DO-254, with supposed FAA-oversight…and we know how that turned out.
Okay…a longer answer than planned…my point is that I agree with you! Specifications and lots of governmental oversight don’t guarantee the promotion of lower costs and higher reliability.
I heard that the clock error was because someone forgot to change the button cell battery. The mission was delayed so long that that battery life was exceeded and the clock slowed. Could this be true?
I really doubt that. Digital electronics generally don’t work that way. I’d expect a low voltage or power to shut a digital clock off, not make it run slow. Typically, they’d run as intended up to a point and then shut down. Temperature is another matter, but I wouldn’t suspect that either. Things like voltage and temperature aren’t even part per thousand effects on modern electronics. I’m not sure you could make a modern, digital clock lose more than a second per hour.
According to the relative price of seats on Starliner versus Soyuz, NASA will actually save money by having to buy more seats on Soyuz. Down side … Soyuz has less leg room and lousy food. Plus side … the little bottles of Stoli.
The Starliner was carrying presents for the ISS crew. Never trust a company which promises 2-day delivery this close to Christmas.
Better to find the timing problem on the way up than the way down.
Boeing didn’t update the software when the test flight got delayed from last Summer. The timer was still set for Daylight Savings Time.
You might be surprised. A Russian launch out of Vostochny failed because they loaded software which assumed the launch was from Baikonur. Oops. The Europeans managed to launch SES-14 into the wrong orbit because someone programmed in, if memory serves, the launch azimuth appropriate for another flight. Oops, but fortunately SES-14 had enough in-space propulsion to make up for it. And spacecraft command sequences are often “epoch relative”, as in “do this 23.3 seconds after first stage separation.” Slipping and saying “after second stage separation” would be basically the same as forgetting to use standard time instead of daylight savings time. (And, yes, everyone using UTC for most things would be more reliable, but we can’t even convince people to use metric, let alone SI metric, so switching to UTC is hopeless.)
Many of the uninformed comments here demonstrate exactly why there was no info immediately after the anomaly. The only certainty about early information is that it will be wrong. Speculation without facts that come from analysis of the telemetry is worthless. Yeah, the whole “a lot went right” party line in the press conference was just spin, but anyone who thinks they know what happened and how to fix it or who tries to second guess the team right now is naive at best. Personally I’d rather spend TLM bandwidth on meaningful measurements than on video. Chances are this was another stupid procedural mistake (which is another issue unto itself), just like the parachute test problem, but no one knows yet.
Yes, we have to be protected from basic information.
Huh?
Agreed with one exception. In addition to the telemetry data about the vehicle… hell yes. I do want all the video streams (or recorded for playback) just because in this day and age, there is no reason (unless you are trying to keep your secrets) not to show them publicly.
I, and other people I know, had that argument and lost it. I’m in a brach of space science where we depend on all sorts of spacecraft booms, masts and antennas deploying properly. And it doesn’t always work. On MAVEN, one boom did deploy properly but the contact to signal correct deployment didn’t work. On MMS, one boom only extended 99% of the designed way, but they eventually decide that was fine (it was not going to move, and the effects on the measurements were easily handled by a tiny change in calibration.) Dealing with that sort of thing isn’t fun, and having a few cheap, cell phone quality images would be really nice.
But the idea of adding something like that gets shot down. We get asked if it’s necessary. If it is, then a cheap, cell phone camera won’t do because it might fail. If it isn’t necessary, then we’re asking them to add an unnecessary part which could potential fail and jeopardize the mission. Saying “That’s what circuit breakers are for” just doesn’t seem to convince project managers.
I wish they had a cheap camera on JWST (and the origami mirror and sun shield deployment would be spectacular publicity…) But I asked someone involved (giving a lecture on what sorts of planetary astronomy JWST could do) and she said no, there wasn’t a camera. They considered it but decided not to put one on for more-or-less the above mentioned reasons. (And, apologies in advance if she reads or hears about this and I misunderstood her.)
Since others have commented on the openness and speculation issues, I’ll stick to something technical. Although I will note that Boeing said they did have imagery, but didn’t show it.
I would not prefer more telemetry channels to video (or at least a framing camera.) Not for the PR value, but because the numbers in the usual housekeeping stream measure expected things. As often as not, a problem isn’t one you expected in advance, and the telemetry isn’t exactly helpful. You end up piecing things together from things like “a temperature on the adjacent part is a bit off, the part is drawing a tiny bit more power than we’d expect and the overall bus voltage dropped but is still within spec.” Video isn’t good on the bandwidth, but it is information rich in a way which covers unexpected problems.
I’ll note that one nuclear accident was caused (partially) by limited telemetry. Something went wrong and the operators dropped in the control rods. But their only indication was that the release actuator had opened as commanded. They couldn’t see that the rods had only dropped an inch or two and then jammed. Another example is the Apollo 13 explosion. Mission control initially saw the telemetry go completely nuts, and there was some initial speculation about it being an instrumentation problem rather than something real. That ended when Mr. Lovell looked out the window and reported the spacecraft was venting something.
Exactly. Telemetry is great but having video (even audio) confirming the events is very valuable for identifying/diagnosing problems and investigating mishaps. Again, it is not that expensive…. when you end up needing it.
It isn’t expensive on a launch vehicle. I’ve been in arguments about how much housekeeping telemetry we should send back from a spacecraft orbiting Saturn. That’s expensive, and I think I’m in the minority of scientists who think it’s critical and worth time and effort to get right. It’s hard to convince people that a min, max, average voltage measurement, once a minute, is worth one fantastic image per hour. For planetary missions, that’s debatable although I’m firmly on one side of the delate. For a launch vehicle within 200 km of the Earth? I really expect a flood of telemetry and diagnostic video.
I think the big bang they heard at the time of the explosion tipped them off it was something other than an instrumentation problem way before Lovell looked out the window.
Which “they”? I was talking about the people in mission control. I’m sure the crew knew it wasn’t an instrumentation problem. But according the Lovell and Kluger history, “Lost Moon”, several people in mission control were wondering about instrumentation problems until the report of venting.
There’s plenty of bandwidth to do both.
Like it or not, SpaceX has established a public expectation for flight broadcast in the present day, to the point where failure to do so too easily creates a suspicion of lack of transparency.
I shouldn’t be hard for Boeing to do something at the same level, and I expect they will, eventually.
I’m not holding my breath. I don’t know about Boeing, but their ULA partner used to have a rule against people visiting their facilities having cameras. They stuck to that well into the days when every single cell phone had a camera. At one point, I had to specifically ask for one without a camera, and the guy at the store thought he might have a two year old model somewhere in the back room. Sometimes I think companies like that need to be dragged, kicking and screaming, into the twentieth century.
Well, color me vexed.
Were there a Pantheon of Boeing Supporters surely my name is somewhere near the bottom. But fair is fair.
How in the world is this flight a ‘flop’? Indeed how is the result of any test flight properly termed a ‘flop’? The chain of events here are very well known, and are trivial, perhaps comparable in silliness when rocket stages unhappily bump into one another, experienced by SX.
From all appearances, Boeing has delivered the contracted product (here, insert something like “Considering the cost it better be good!”). Someone screwed up the clock. Won’t happen again.
***************
It’s natural to compare the Boeing in-house launch coverage with that of SX. Boeing managed at once to over-produce a paucity of facts. Everywhere I looked I wondered how SX or the Russians handled the same situation; the entire launch tower and crew access arm come to mind as stunningly over-engineered. SX gave us young people with gargantuan levels of energy and enthusiasm jumping off the screen; Boeing reinvented ‘droll’.
Could we have more unmoving shots of computer panels, please?
It’s a flop because it could not reach the ISS. A significant portion of the requirements will not be met by this test flight. Like someone else commented, it will be interesting how much NASA lets Boeing off the hook, if at all. This Operational Flight Test will not be able to practice any of the proximity flight operations that will give NASA the confidence the capsule can be controlled in close distance from the Station. Sure, hopefully the landing will be completed without any more surprises…
If FedEx delivered your package to the wrong address, or on the wrong day, would you consider that a “success?”
And please don’t say it’s a poor analogy, because that’s exactly what these spacecraft are meant to do: delivery.
In other news, a couple in California just got a Christmas card from a complete stranger in Wales, addressed to someone they didn’t know in England. It was stuck to a card for them and from a relative in Wales. A misdelivery isn’t a good thing, but it isn’t necessarily the end of the world. At this point, Boeing really does need to hope this is relatively harmless.
(Later edit) Actually, I’ve been on four commercial aircraft flights which had to divert. That was annoying, but I think it was definitely the right thing to do. And I’m happy the vehicle was capable of it. Not getting the astronauts to ISS, at a rare and occasional level, is perfectly fine as long as they get back to White Sands safely.
It’s a poor analogy!
Unless, of course, the package consisted entirely of straw…
OK, OK, sorry. I get the point.
Here is my analogy, in the case of Starliner, the delivery truck got stuck in a ditch and had to be towed back to the shop without delivering any of the packages, including groceries (presumably) and Christmas presents for the Station crew
Flop seems fair. No it’s not a total failure by any means. Assuming a successful landing tomorrow it will have tested launch, reentry and landing, and a least a limited amount of time on orbit. However this is a far cry from what the mission was intended to demonstrate. They might have salvaged it better if they stayed up the full eight days and gone through as much flight and maneuvering testing as time and fuel allowed, simulating much of the activity that would have occurred had they made it to ISS. Also providing a longer duration test of all of the systems.
Instead they are bringing it down on the first opportunity available, giving the impression that they don’t have confidence in the spacecraft to simulate a full mission. Oh I’m sure they can give rationale why it wasn’t worth the effort to keep it in orbit longer. I’m just saying the whole thing does give quite the impression of, well a flop.
The statements about fuel use confuse me. On the one hand, I would not expect attitude control thrusters to use up that much fuel. Most of the time (well, for the robotic spacecraft I’m familiar with) that’s much less than the propellent needed for a major orbital trim maneuver. But they say that’s why they don’t have enough propellent to get to ISS. Odd. On the other hand, if they really are that short on fuel, they might not have enough to stay on orbit for a week and also confidently and safely get down.
I also had been wondering about the fuel, but am less skeptical as I think about it more. My guess is the fuel loss they experienced during the anomaly really cut into their margin pretty bad. And I expect that for landing they have actually expanded the margin due to the problem at launch, in case it occurs again. So having used up margin they had to cut something to replace it, and that something would be ISS docking, either actual or simulated.
Another though occurred to me. I’m not sure about the plumbing on a CST-100, but it isn’t unusual for spacecraft to have separate tanks for the main engines and the attitude control thrusters. Using biprop MON/hydrazine for the main engine and monoprop, high-pure hydrazine for the attitude control thrusters is pretty common. Maybe they zeroed out the attitude control thruster propellent while still having a full tank for the main engine(s).
A flop might be going too far, but it’s fair to say the mission did not accomplish all of its goals. Boeing doesn’t do things in the “fly, bang, fix, refly” way SpaceX does. Their process is to build it right from the start, and the test flight is simply to prove that they did exactly that. It’s not to learn what they didn’t get right. I strongly suspect they don’t have schedule margin for this. (No, I’ve seen that schedule in a NASA OIG report; that’s a fact, not a strong suspicion.) So this is a definite delay for operational use of Starliner.
The Commercial Crew contract actually specifies that the uncrewed test flight must include a rendezvous, docking, and undocking with the ISS. Now, NASA can probably waive that; but the fact remains that key parts of this test flight were not achieved.
But if that’s not enough, consider the concern of one veteran astronaut:
https://twitter.com/thegrab…
A question for the smart people here, please? Watching the presentation, did anyone notice a particularly large guy with a hugely magnificent beard?
I don’t remember his name as I was considering his point: that the entire capsule is made by hand (so is Aston Martin) and that no power tools are used (unlike Aston Martin).
Is this the case? If so, why? Would the same be said of SX, or the Russians?
I didn’t watch the presentation, but that’s a bit surprising. I don’t know about Dragon 2 in particular, but SpaceX does use 3-D printing when it’s appropriate. That’s definitely not by hand, and I’d be surprised if there aren’t some 3-D printed parts on Dragon. There are some advantages to doing things by hand. Some tasks are just hard to automate, especially when they involve delicate things. But some of those delicate tasks are actually better done, and more reliably done, by machines. And using no power tools at all just sounds weird.
I love the work of this guy. Scott Manley explains the Starliner mishap, including the shallow trajectory.
https://www.youtube.com/wat…
Looks like somebody forgot to wind the clock or set the wake-up alarm to the wrong time. Can’t figure out why the TDRS gap wasn’t covered. Somebody should have brought this up in pre-flight planning, just in case complete coverage was required.
As far as “now” giving SpaceX all the crewed flights is concerned, PRESUMING that Boeing can figure things out, we NEED two crewed systems available (or pay $82M or more) in case there is a problem with one of them. We wound up with that situation with Shuttle! Would I prefer to ride on a Dragon 2 or a Starliner? At this point, I’d rather fly with SpaceX, as Boeing hasn’t shown much in the engineering business lately. Hopefully, they will get their act together, soon.
I see that today’s NASA/Boeing media conference while streamed, will not be broadcast on NASA TV; according to a headline box on the website.
There is also nothing in the schedule mentioning tomorrow’s landing.
Notwithstanding the Administrator’s (much appreciated) ongoing push to improve transparency; it appears that some of his people will still have to be dragged kicking and screaming into the “daylight”!
NASA TV Press Release announcing coverage of landing:
December 22, 2019
MEDIA ADVISORY M19-143
NASA Television to Air Boeing Starliner Spacecraft Landing
NASA and Boeing will provide live coverage of the landing on Sunday, Dec. 22, of the company’s CST-100 Starliner spacecraft, on return from its Orbital Flight Test for NASA’s Commercial Crew Program.
Starliner will execute a deorbit burn at 7:23 a.m. EST to begin its return to Earth, headed for a parachute-assisted landing at 7:57 a.m. at White Sands Space Harbor in New Mexico. NASA Television and the agency’s website will provide mission coverage ahead the spacecraft reentry and landing beginning at 6:45 a.m.
NASA and Boeing will host a postlanding news conference at 10 a.m. with:
NASA Administrator Jim BridenstineJim Chilton, senior vice president of Boeing’s Space and Launch DivisionSteve Stich, deputy manager of NASA’s Commercial Crew Progra
The uncrewed Starliner spacecraft launched on a United Launch Alliance (ULA) Atlas V rocket Friday, Dec. 20, from Space Launch Complex 41 at Cape Canaveral Air Force Station in Florida.
Though Starliner did not reach the planned orbit or dock to the space station as planned, Boeing still was able to complete a number of test objectives. Teams from NASA, Boeing and ULA worked quickly to ensure the spacecraft was in a stable orbit and preserved enough fuel for multiple landing opportunities.
I was surprised by the NASA press conference to update the Starliner mission. Great questions, Keith. Boeing was recording the video and was going to play it back after reaching the Station. I guess that will have to do, but seems lazy (or cheap?) not to live stream it. But, I accept the answer, if you showed me the recordings now. I’ll wait. =)
In my experience, it’s ALWAYS the clock.
I can’t see a crewed flight next.