Protecting NASA's Earth Science Data From Deletion
Diehard Coders Just Rescued NASA’s Earth Science Data, Wired
“Like similar groups across the country – in more than 20 cities – they believe that the Trump administration might want to disappear this data down a memory hole. So these hackers, scientists, and students are collecting it to save outside government servers. But now they’re going even further. Groups like DataRefuge and the Environmental Data and Governance Initiative, which organized the Berkeley hackathon to collect data from NASA’s earth sciences programs and the Department of Energy, are doing more than archiving. Diehard coders are building robust systems to monitor ongoing changes to government websites. And they’re keeping track of what’s already been removed – because yes, the pruning has already begun.”
Climate Data Preservation Efforts Mount as Trump Takes Office, Technology Review
“Earlier federal data hackathons include the “Guerrilla Archiving” event at the University of Toronto last month, the Internet Archive’s Gov Data Hackathon in San Francisco at the beginning of January, and the DataRescue Philly event at the University of Pennsylvania last week. Much of the collected data is being stored in the servers of the End of Term Web Archive, a collaborative effort to preserve government websites at the conclusion of presidential terms.”
Rogue Scientists Race to Save Climate Data from Trump, Wired
“The group was split in two. One half was setting web crawlers upon NOAA web pages that could be easily copied and sent to the Internet Archive. The other was working their way through the harder-to-crack data sets–the ones that fuel pages like the EPA’s incredibly detailed interactive map of greenhouse gas emissions, zoomable down to each high-emitting factory and power plant. “In that case, you have to find a back door,” said Michelle Murphy, a technoscience scholar at the University of Toronto.”
Heroes, all of them. I hope they back it up to enough sites so that it can’t be lost.
I think folks have some serious hysteria about this !! I doubt any data will be deleted regardless of what happens.
A week ago, I didn’t think the “nuclear football” would ever be used for Facebook selfies.
I think the publicity has helped reduce the risk, but better safe than sorry. The problem is not simply intentional deletion but loss of access due to the loss of maintenance funding.
A test case will be the EPA. Trump and the Republicans desperately want to totally shut down the EPA, and I think they may largely do that. If not, then why are EPA employees being specifically told to look out for executive orders explicitly tailored to the EPA? And why would trump issue executive orders explicitly targeting the EPA if not to all but shut it down?
http://www.reuters.com/arti…
I am a little worried about what they are archiving. Based on the article, sounds like they are focusing on the web sites and metadata, rather than the data itself. But that could be just how they are starting or how the article was written. Certainly, that’s what any automatic grab would collect first.
I also hope they get all of the associated documentation and ancillary information needed to use the data. Some of the links to that aren’t obvious and might not be followed by a web bot.
The spreadsheet referenced in the Technology Review article list at most some Tbyte of data. The actual archives of earth science data at NASA are in the multiple Pbyte range.
This is a well-meaning effort to preserve certain reduced datasets and Websites, but not the entire NASA earth science archive, which as noted below could begin to degrade after a few years of neglect (if maintenance and sys admin contracts are not renewed).
So far I have seen no evidence of issues with data archiving, and distribution. My concern is funding for future missions and for research. This is where the real damage can happen.
G.