Hubble's Problems Fixed
NASA Hubble Update: July 16, 2021 – NASA Successfully Switches to Backup Hardware on Hubble Space Telescope
“NASA has successfully switched to backup hardware on the Hubble Space Telescope, including powering on the backup payload computer, on July 15. The switch was performed to compensate for a problem with the original payload computer that occurred on June 13 when the computer halted, suspending science data collection.”
NASA Hubble Status: July 14, 2021 – ?NASA Identifies Possible Cause of Hubble Computer Problem
“NASA has identified the possible cause of the payload computer problem that suspended Hubble Space Telescope science operations on June 13. The telescope itself and science instruments remain healthy and in a safe configuration.”
Hubble Update: July 08, 2021 – Successful Completion of Multi-Day Test
“NASA successfully completed a test of procedures that would be used to switch to backup hardware on Hubble in response to the payload computer problem. This switch could occur next week after further preparations and reviews.”
Operations Underway to Restore Payload Computer on NASA’s Hubble Space Telescope
“Over the next week or so, the team will review and update all of the operations procedures, commands and other related items necessary to perform the switch to backup hardware. They will then test their execution against a high-fidelity simulator. The team performed a similar switch in 2008, which allowed Hubble to continue normal science operations after a CU/SDF module failed. A servicing mission in 2009 then replaced the entire SI C&DH unit, including the faulty CU/SDF module, with the SI C&DH unit currently in use.”
NASA Completes Additional Tests to Diagnose Computer Problem on Hubble Space Telescope
“NASA is continuing to diagnose a problem with the payload computer on the Hubble Space Telescope after completing another set of tests on June 23 and 24. The payload computer halted on June 13 and the spacecraft stopped collecting science data. The telescope itself and its science instruments remain in good health and are currently in a safe configuration.”
Operations Underway to Restore Payload Computer on NASA’s Hubble Space Telescope 22 June 2021
“NASA continues to work to resolve a problem with the Hubble Space Telescope payload computer that halted on June 13. After performing tests on several of the computer’s memory modules, the results indicate that a different piece of computer hardware may have caused the problem, with the memory errors being only a symptom. The operations team is investigating whether the Standard Interface (STINT) hardware, which bridges communications between the computer’s Central Processing Module (CPM) and other components, or the CPM itself is responsible for the issue. The team is currently designing tests that will be run in the next few days to attempt to further isolate the problem and identify a potential solution.”
Operations Continue to Restore Payload Computer on NASA’s Hubble Space Telescope
“NASA continues to work on resolving an issue with the payload computer on the Hubble Space Telescope. The operations team will be running tests and collecting more information on the system to further isolate the problem. The science instruments will remain in a safe mode state until the issue is resolved. The telescope itself and science instruments remain in good health. The computer halted on Sunday, June 13. An attempt to restart the computer failed on Monday, June 14. Initial indications pointed to a degrading computer memory module as the source of the computer halt. When the operations team attempted to switch to a back-up memory module, however, the command to initiate the backup module failed to complete. Another attempt was conducted on both modules Thursday evening to obtain more diagnostic information while again trying to bring those memory modules online. However, those attempts were not successful.”
The Hubble Space Telescope has provided us with magnificent views and science over the years. Hopefully, some fix will be possible soon. However, without the capabilities provided by the Space Shuttle there is no current capability to repair it. Best of luck to the Hubble support team and scientists in enabling the telescope to continue to furnish us with science for a while longer.
NASA have the capability to send a repair mission to Hubble. But it will take time and money. Mostly money.
Time & Money, Ah yes, JWST comes to mind. Hubble serves (served) long and proud with achievements, but enough is enough, it is criminal that JWST is still on terra firma!
To quote Wayne Hale in 2018, “To repair Hubble you need a trained repair crew in a mid high LEO with a spacecraft that has two weeks + flight duration, lots of tool stowage, an airlock, a robotic arm. We used to have some of those but none now or on the horizon.”
I would say that as far as NASA goes the “none now or on the horizon” statement is still true.
Starship I’m sure could be configured with an airlock and a robotic arm. And obviously stowage for as many tools as you want. But it would still be a very expensive project and it seems unlikely that such a mission will occur. I’m just hoping that Hubble can be brought back for the Smithsonian. Even that would be expensive, and it would be a challenge to get a mission like that into the NASA budget since it would be of a purely sentimental nature. There would be the benefit of being able to study an object that has been in space for over thirty years, but you could accomplish the same thing by bringing back a smaller satellite.
I have not heard if SpaceX has any plans to use Starship for retrieving satellites. Seems like that could be a lucrative market. Shuttle of course had that capability but with Shuttle flights costing a billion dollars each it made no sense economically. If SpaceX does eventually get into that business with Starship then a Hubble retrieval mission would be more likely. Of course that begs the question if they can retrieve Hubble then why not refurb it then launch it again, but the cost to do that would be enormous and just seems unlikely. I think the best that we can hope for at this point is that Hubble will eventually wind up in the Smithsonian and not in the Pacific.
Hubble’s been on borrowed time for awhile so if this is it then it had a good run.
I am no longer too concerned about the inability to maintain Hubble without the Shuttle. When they shut down Shuttle they were signing Hubbles epitaph. However that there is no ability to repair or maintain the Webb telescope in an analogous manner is criminal, most especially after the budget overruns of that program. Everyone needs to pray and keep fingers crossed because a minor problem could doom Webb from the start.
There is always the question of whether it is more practical to repair or replace. The second instrument would be a fraction the cost of the first. Two of every mission used to be standard. Unfortunately there is no way to build another Webb. A simpler design that can be easily duplicated might be better. With Starship for example it may be possible to use a one-piece mirror and avoid the deployment system cost. A smaller scope, possibly with UV capability, that could be shipped to ISS on Falcon would be relatively inexpensive and a valuable asset.
A pretty big fraction of the cost of any mission like this is in I&T (integration and test). I don’t think building a second would be particularly cost effective. Also, you wouldn’t want to put a space telescope at the ISS. The local environment is pretty dirty, and there’s a reason pretty much all these space telescopes now go to L2: to get away from the Earth. The Earth creates all sorts of problems in terms of difficulties in scheduling and in dealing with heat load. A huge breakthrough with the Spitzer design was the Earth-trailing orbit.
How about this: Forget more “space” telescopes. Get to the Moon and establish both optical and radio telescopes at whatever optimum places there are. Opticals might work best from one or both poles so that they could be slewed in azimuth to view whatever portion of the sky is desirable except looking directly at the sun. Radio telescopes would be best placed on the far side of the moon (and perhaps an optical as well). The radio scopes would be shielded from Earth generated electromagnetic interference. Cost? Depends on whether they would need to be crew-tended or simply relaying the date to Earth.
At least for optical/infrared telescopes, the moon has all the same problems as earth orbit, as well as some extras, like the infamous levitating dust clouds. Radio telescopes might be more workable, but it’s not clear to me that the expense would be worth it over terrestrial telescopes.
The farside of the Moon does provide visibility in lower RF frequencies that are blocked by the ionosphere on Earth, as well as some shielding from Earth based interference. Of course, with the increasing popularity of missions going to the Moon and environs, the possibility of interference from those spacecraft is a concern. The farside of the Moon may not be a radio quiet zone very long.
This has been my chief complaint with JWST since the SLS was announced. Heck, since the Ares V was announced. Its folding mirror, at least in this format, is a technological dead end whose complexity is driven by the payload shroud limitations of mid-2000s launchers. A follow up telescope will be able to take advantage of the greater volume provided by Starship, SLS, or even a modified form of the Falcon Heavy (of which I think is debuting in the next year or so unless I’m mistaken?)
When I see the JWST, I see NASA’s version of the Gerald R Ford class aircraft carrier: technologically ambitious and valuable in the long term, but there probably should have been a half step between them to see how associated technology matures. With the Ford class, the US Navy would be far less strapped for deployable carriers today (and in much better fleet maintenance shape) if it bought one more Nimitz after the George H.W. Bush. With JWST, perhaps using the other NRO donated telescope (the one not used for WFIRST) as a more direct Hubble 2.0, would have been such a worthy midstep.
I do wonder though, would a quick and dirty servicing mission using Crew Dragon, with its trunk for tool and replacement parts, just to fix this one error, be out of the question? Someone else mentioned a robotic arm, but I don’t think that’s needed anymore since the last servicing mission put the Soft Capture and Rendezvous System on it.
With reliable and inexpensive access to space via Crew Dragon finally here, in my view we shouldn’t be against far more frequent and routine servicing trips to national assets like Hubble that aren’t approaching the major overhaul scale of prior servicing missions. I would think you wouldn’t need the Shuttle’s payload capacity, if you we just sent small servicing trips more frequently to fix problems as they cropped up.
Crew Dragon doesn’t have an airlock, or any of the facilities required to support EVA work.
You could use a modified Cygnus as the airlock, cargo carrier and outbound service module. After you docked the Crew Dragon and Cygnus in LEO.
But this Hubble servicing concept is obsolete with the Starship on the horizon.
I was under the impression that if they brought actual spacesuits along, they could open the hatch and depressurize the capsule, and then depressurize it once done.
It isn’t clear if astronauts in an EVA suit could exit a Dragon. They’d have to go out the IDSS port, and that’s only an inch or two wider than NASA’s EVA suits. There are safety issues involved with that. We don’t want to repeat Alexei Leonov experience on Voskhod 2, where he got stuck trying to get back into the spacecraft.
Repairs to HST would, based on past servicing missions, almost certainly require a robotic arm, which Dragon lacks. And I’m not sure what depressurizing and repressurizing a Dragon (probably more than once) would do to its in-orbit lifetime. That’s certainly not something it was designed for and it would probably involve venting, rather than storing, the air.
What about using the hatch instead? I’m really thinking about this in basically Gemini terms.
This sounds more complicated than a quick and dirty repair job in any event. Which is a shame. Because a Crew Dragon that can support EVASs does open the door to for frequent servicing for national asset-level satellites, as opposed to Shuttle-style major overhauls. I have this idea in my head of a one service trip a year.
But if Crew Dragon cannot do it, then that’s a damn shame. Is Starliner any different, or do we not know? Orion, I know, can be depressurized and depressurized. But sending Orion on an SLS to the HST seems comical. In principle could do a Delta IV Heavy, but that’s really just a fantasy too considering how that line is winding down.
Doesn’t mean it couldn’t be fitted with one at the docking ring on it’s nose.
JWST is going to be in an Earth-Sun L2 halo orbit, for a number of good reasons. At the time JWST was being designed, there were no spacecraft which could get astronauts to that orbit (there still aren’t) and none even vaguely on the horizon. So they didn’t waste mass on designing it for servicing. That’s also why they didn’t consider very large launch vehicles (SLS or Starship) when they designed the sunshade. They didn’t exist and weren’t even concepts when JWST was designed. The design was fixed a very long time ago, and based on assumptions which made sense around 2000 or 2005.
While I have friends at STScI who seem to think otherwise, the simple reality is that eventually HST will die (just like everything else). It’s been 12 years since SM-4, and I’m astounded it’s still going. I would have bet on the attitude control system failing by now. That last set of gyros must have really been something.
The previous longest duration was 7 years due to the Columbia accident delaying the final servicing mission four years, from the originally planned 2005 to 2009.
Three of the six gyros installed in 2009 have failed, the last one in 2018. Hubble can operate with one gyro with some targeting limitations. Interestingly during the final servicing mission in 2009 they could not get one of the three gyro pairs to seat properly, so they installed a backup pair that had been retrieved during the third servicing mission in 1999 and which had been restored on the ground. I don’t know if the three gyros that have failed since 2009 include these two refurbs. But it could be because I have read a couple of articles that indicate that the three gyros that have failed were of an older design. That would be consistent with refurbished units, although I don’t know how to reconcile “two refurb gyros” with “three gyros of older design”. But I guess what I am hoping is that at least two and maybe all three of the remaining gyros are of a design that is less likely to fail than the ones that have.
NASA and SpaceX probably have been working on a plan for servicing Hubble using Starship for a while now. It’s the logical thing to do. With regular servicing, Hubble can operate indefinitely.
Your speculation is based on…..what?
His imagination.
I have been thinking along the same lines.
Starship brings HST back to Earth. Once back on the ground HST is given an overhaul and Starship take it back up. Not sure how cost effective that would be but it’s sure fun to speculate about.
I think there would be a lot of uncertainly about possible damage that the telescope might experience during reentry and landing, especially in a spacecraft that the telescope was not designed to fly in. I’m not sure if it was ever even considered to bring Hubble back for repair with the Shuttle. Even if it was discussed they obviously concluded that it was safer for the telescope to do the repairs on orbit. Although I suppose the amount of downtime for the telescope was also part of that decision, one week of downtime on orbit compared to say six months at a minimum if they brought it back to Earth. But I still think the maximum safety of the telescope was likely the primary factor, of course also making sure that the procedures would be safe for the astronauts also.
It was initially planed for HST to be brought back by the shuttle but the idea was scrapped because of concerns the shuttle to do it safely.
It wouldn’t surprise me if some at NASA have had that thought. But funding to actually study this seems unlikely.
Some years ago , Lockheed Martin and the Spooks gifted NASA a surplus spy sattelite that was in fact the same vintage and chassis that the Hubble was built on …a KH Keyhole series I seem to recall ? It was said at the time the suplus sat , while stripped of the the sensitive optical surveillance gear did have a perfectly fine properly shaped and polished 2 meter mirror and focussing assembly and all the support systems. The leftover spysat worth a couple billion bucks was accepted by NASA, then promptly put in storage IIRC where it remains to this day. NASA did not have the surplus money tor efurb it for a follow-on Hubble or run an operational program once in orbit.
Think of the possibilities, though . A hardened milspec Hubble chassis with all new deep sky sensors, a modern computer and OS , and a helluva lot more memory and storage and AI . Maybe even modified to be serviceable on orbit by robotic spacecraft . It could even be uprated to perform in a much higher orbit or even GEO to diminish or avoid the 12 X per day descents into Earth eclipse shadow and all the intense thermal shocks from those.
We have a Hubble ver. 2.0 in the garage , if we would only fund it , launch it, and use it. I’m pretty sure the American public would get 150 % behind such a smart move. Even the most antip-saceflight or frugal Senator would be challenged to vote against such a pragmatic proposal because of Hubble’s existing popularity among all the voters. ( Hubble ain’t partisan—everybody of all ages loves it , across the demographic board ! Name a better return on space program dollars. You can’t ) We’ve grown to love our Hubble Space Telescope dearly . Son of Hubble would be a fantastic evolution . As Mr. Spock would say , logical. NASA would look positively golden for doing this. ( Estimate cost $ 2-3 billion , a fraction of one SLS throwaway launch )
..so do it !
It’s more complicated than that. NASA was offered two partial spacecraft (including mirrors, but missing a number of other systems, if memory serves.) One is going to become the Nancy Grace Roman Space Telescope, formerly WFIRST. The optics aren’t quite the same as Hubble, although the mirror is about the same size. That does make it suitable for full sky surveys, which is what Roman is all about. But adapting it turned out to be very expensive, and was a major source of cost growth from what was originally estimated for WFIRST. The other spare was considered by NASA (I liked the idea of a UV telescope). But given the budget demands of JWST and WFIRST and the priorities in the Astrophysics Decadal Survey, they couldn’t find a good and affordable use for it.
There was an entire request for proposals issued by NASA for using the “national assets transferred from another agency” called “SALSO” (Studies for Applications for Large Space Optics) back in 2015. About 30 of us were further invited to present white papers in Huntsville; they ran the gamut from next generation UV telescopes to interplanetary optical data relays. It never went anywhere, there was no actual budget. None of the proposals realistically came in below a couple billion. As mentioned, it was two sets of optics (primary and side-looking secondary), along with some assorted support structures for one of the mirrors. No spacecraft. Having the optics drive the design is not a good way to work – it’s like you needing a car, so someone gave you four wheels and a hood from a Ferrari, and then wondered why you were so ungrateful about it.
Well, maybe the engine and nothing else. But point taken.
It’s not in storage… the optical telescope donated became the Nancy Grace Roman Space Telescope. It’s a big deal. It’ll fly at L2 just like Webb.
I have a bad feeling about this… something in my gut makes me think this is the one. We’ve all kept our eyes on the gyros since the mid 2000s. Imagine if in the end, it’s because a computer error.
Your concern seems misplaced. There are 4 identical units of this antiquated payload computer in the Hubble. NASA will just have to replaced the failed one with one of the 3 working ones.
Antiquated as in using ring core memory with a maximum capacity of 64kb at 18 bits. There is a Wikipedia entry for the NSSC-1. Hopefully NASA have someone who can programmed this antiquated device.
Actually there is only one backup for the payload computer. You are probably thinking of the four memory modules in each computer. They initially thought it was one of the memory modules that failed, but pretty early on in the investigation they realized that this was not the case. A couple of days ago they turned on the backup payload computer for the first time but they still see similar errors. In fact they are able to use different components from each of the two payload computers in various combinations, which they have been doing also in an attempt to locate the problem. So far unsuccessfully, so they are now considering other possibilities.
They seem to not be rushing the troubleshooting process since the overall health of the telescope is good, taking things slowly and one step at a time, and I am sure doing a lot of analyzing and strategizing in between tests. So I agree that the fact that they have not yet isolated the problem in and of itself is not a cause for alarm. However I’m not sure if every piece of the power and communication hardware has redundancy, but one would hope so. And you also have to hope that backup components that have not been powered up in over a decade are still working. So I think it’s not out of place to be concerned, but probably no need to panic at this point.
Does this remind anyone else of “Space Cowboys” ?
Why don’t they just go up there and fix it? Oh yeah, we don’t do that anymore.
Seems like there is light at the end of the tunnel.
The monkey in me would like to know what the problem was/is.
Follow the link to the article. It describes the problem and the fix.
A brief summary:
The Power Control Unit (PCU) isn’t supplying the required 5V. To fix it they are going to switch to the backup PCU. If it works it will take several day for HST to return to full operations. They had a similar issue back in 2008 and switched to the backup PCU until a servicing mission in 2009 applied a permanent fix.
It may not be 5V supply. It could be the voltage monitor that is detecting a voltage fault.
Their theory being that either the PCU is putting out voltage that is out of range, or that the voltage monitor that is associated with the active PCU may have failed in the inhibit state, thus falsely reporting a voltage fault.
They seem pretty convinced that the problem is somewhere on that side of the Science Instrument Command and Data Handling unit. They were apparently able to verify that the Payload Computer itself, as well as the memory modules and other items on that side of the SI C&DH are not the problem, the process of elimination apparently pointing to either the PCU on that side or its associated voltage monitor circuit. Thus they are hoping that switching to the backup side of the SI C&DH unit will solve the problem.
The article says they think that the problem might be hardware related to the Power Control Unit. The PCU supplies and regulates power to the Payload Computer. They will be switching to the backup side of the Science Instrument Command and Data Handling unit which contains a backup PCU. What they have been doing for the past couple of weeks is running through simulations of the procedures to switch to the backup side on a ground based test computer to make sure that they have all of the procedures correct.
Woo hoo!!
but it is now probably zero-fault tolerant….enjoy it while it lasts!
Just have to last long enough for the folks from Hawthorne to arrive with the retirement ride to the NASM.
Why not just send a Shuttle up to…oh, wait a minute… 🙁