This is not a NASA Website. You might learn something. It's YOUR space agency. Get involved. Take it back. Make it work - for YOU.
Exploration

NASA Pays For Decision Making Advice On A Decision It Already Made

By Keith Cowing
NASA Watch
August 28, 2015
Filed under ,
NASA Pays For Decision Making Advice On A Decision It Already Made

Innovative Study Supports Asteroid Initiative, Journey To Mars
“NASA employed ECAST to engage in a “participatory technology assessment,” an engagement model that seeks to improve the outcomes of science and technology decision-making through dialog with informed citizens. Participatory technology assessment involves engaging a group of non-experts who are representative of the general population but who unlike political, academic, and industry stakeholders who are often under represented in technology-related policymaking. … During meetings in Phoenix and Boston in November, 2014, participants voiced their thoughts and preferences about asteroids, planetary defense and space exploration.”
Informing NASA’s Asteroid Initiative – A Citizen’s Forum – Full report
Keith’s note: According to the report “We at ECAST designed the forums to explore what a diverse group of lay citizens thought about complex issues when provided with unbiased information and offered the opportunity to have a respectful and open conversation about these matters with their peers. Quite different from a poll or survey, forums like the one developed for this project explore the views and values that citizens use in assessing sociotechnical issues. … ECAST undertook the recruitment of the lay citizen participants, achieving a distribution that aligned with the demographic characteristics of their respective states by taking into account gender, age, education, ethnicity, income, and employment status.”
So … how did these people from nowhere in particular get up to speed on NASA’s Asteroid Redirect Mission (ARM)? According to the report “Rather than survey people who may have little understanding of the subject, these forums provided the opportunity for participants to learn a great deal about NASA’s Asteroid Initiative. In fact, participants were provided with much the same technical information that NASA’s administrators and program managers use, but presented in short thematic background papers provided prior to the workshop and four informational videos at the start of each session.”
Ah, so they only showed the participants NASA stuff. Did the participants receive materials that were in any way critical of ARM? Seriously. The participants were being asked to weigh all aspects of ARM, asteroid defense etc. Given that Congress, the NASA Advisory Council and a significant portion of the planetary science community doubt the value of ARM and/or are totally against it one would hope that this was factored in. If the participants were not given the full spectrum of viewpoints on this topic then the entire effort was null and void at its very inception.

Don’t get me wrong, I really think its great that NASA is seeking to engage and explain things to people of a wide variety of backgrounds – especially non-technical ones, but what possible value is there in asking these lightly-briefed novices to decide whether use of a nuclear device or a kinetic energy impactor is preferable when dealing with a hazardous asteroid? They actually spent time on that. If random folks off the street can make these decisions based on a short Powerpoint briefing then why do we need NASA?
OK, so they got a bunch of non-specialists together (a year ago). Again, public engagement by NASA is good (in any form) and should be encouraged. But what was the purpose of this effort? The report says “The project had two main goals. The first was to develop and apply a participatory technology assessment that elicited nuanced information from a diverse group of citizens whose insights would not otherwise be available to decision makers. Second, through informed, structured feedback from citizens in multiple locations, the project aimed to provide public views of the Asteroid Initiative as input into NASA’s decision making process.”
It took the organizers a year to write this report? Its only 32 pages long. And what is NASA going to do with the results of this effort? NASA has already made the decisions as to how it is going to do ARM and they did so well before this event even happened – and the output arrived (in the form of this report) well after the fact. How can this effort affect decisions that have already been made?
Hmm, maybe NASA should tell more of the general public what they learned from non-NASA discussions about this multi-billion dollar mission that Congress is against – and how they only obtained the input from general public after the decisions had already been made. That sequence of events ought to be illuminating. Instead NASA posts a story at the end of the day before the Labor Day weekend (without sending it to news media) in the hope that no one sees it (I guess).
– Involving a broad expanse of taxpayers in the conduct of a space mission with global significance – priceless.
– Stacking the deck with forgone assumptions and getting the output after the real decisions have been made – pointless.
If ever NASA paid for a self-licking ice cream cone, this report is it.
Keith’s update: Mahmud Farooque from Consortium for Science, Policy & Outcomes Arizona State University just tweeted this. Apparently this is not a “final” report. I’m not sure that two tweets and the promise of TBD details is really the way to be explaining this project. After having a year to collect and present all of this stuff, one would think that it would have made sense to coordinate the release of the information instead of just throwing a skinny report lacking in specifics online late on a Friday when everyone is on – or going on – vacation.

Keith’s update: I am told that some of the initial results of these forums were provided to NASA prior to the ARM decision. Of course there is no mention of that fact in the “final report” that was posted today. I am also told that a NASAWatch post Meetings on Public Opinion on Space Closed to the Public in 2013 may have had a small role in instigating this project. If you read this report and just ignore the problems I have highlighted, there is some interesting stuff here i.e. can you use actual public input to assist in making mission decisions? To be honest, I really do not think that the NASA individuals involved in the ARM decision really cared about outside opinions (they ignore that Congress has gone on the record against ARM) since they already decided to do the mission without any prior public input. But I am just guessing.

NASA Watch founder, Explorers Club Fellow, ex-NASA, Away Teams, Journalist, Space & Astrobiology, Lapsed climber.

18 responses to “NASA Pays For Decision Making Advice On A Decision It Already Made”

  1. Daniel Woodard says:
    0
    0

    I agree with Kieth. It would make almost any lay participant in a meeting like this feel awkward at the very least to be presented with such vast concepts in such a flowery manner and yet be critical of them for mundane reasons like cost and scientific value. It remains unclear who the real decision makers are, or indeed if some of these plans (ARM comes to mind) are a response to political rather than scientific goals.

    Citizen participation in the Solar Power Satellite Conference many years ago was similar in some ways, though the organizations invited were chosen specifically because of their interest. In that case the input was not put to any particular use since the SPS concept was felt to be impractical, probably a reasonable conclusion but one which not even the professionals actually admitted during the conference.

    • Rich_Palermo says:
      0
      0

      Well said.

      1) “In fact, participants were provided with much the same technical information that NASA’s administrators and program managers use,”

      I looked at the report and there is no evidence that any real technical information was provided to the participants, only an assertion that it was. I can only hope that NASA’s administrators and PMs get more than what was shown in this report.

      2) Seems like there’s one heck of a lot of Science Policy/Society programs these days.

      3) What does that photo on p.18 signify?

    • ThomasLMatula says:
      0
      0

      I agree 100%. This study makes no sense.

  2. Michael Spencer says:
    0
    0

    What a wonderful country we have created, one where we think that anybody off the street can, with proper facts, make the most complicated decision. Is this true?

    The lay public has time and again showed maturity and thoughtful decision-making in this country. In fact, the process occurs every day and in every county courthouse, where uninformed folks take decisions, getting the call right overwhelmingly.

    The difference as Keith points out is in the way our fellow citizens learn about the case- it’s presented in a very structured way by opposing sides, both of whom are bound by detailed rules of evidence.

    Courts don’t always get it right- that’s beyond the point- but lay people fairly informed can make smart decisions. Take the case of Apple v. Samsung, or other extremely complicated cases with far more facts than ARM.

    And anyway ARM isn’t that complicated once you remove the policy bias.

    • Rich_Palermo says:
      0
      0

      I agree with much of what you wrote. But, having served on juries, it was made clear to me during the selection and impaneling that my role was to decide which side had the better case (ie. attorney) and that I was not to use any judgement outside those bounds.

      • ThomasLMatula says:
        0
        0

        Rich,

        Yes, and that is a basic flaw in it today especially for complex cases in business and technology. It is more a game than a search for truth, which is why the evidence standards for court cases are at a far lower level than found in science.

        It is also 180 degrees opposite from the original intent of jury trials in Medieval England where visiting judges, generally members of the nobility, depended on the knowledge and familiarity of the jurors with the accused and local conditions to make a fair decision, versus having a decision handed down by someone unfamiliar with both. It is also why the jurors were from the same social “status” as the accused, the original meaning of a “Jury of your Peers”.

        • Rich_Palermo says:
          0
          0

          That’s very interesting. Our jury system has many flaws but I would prefer it to that of the olden days. Makes me wonder about my initial reaction to this panel. It is not perfect but the distribution of pork approach isn’t serving us either.

      • Michael Spencer says:
        0
        0

        And so it must be. The assumption is that facts presented to jurors are the relevant facts and only the relevant facts.

        You might respond that some facts might be excluded. That’s true- prejudicial facts, or personal prejudices, previous defendant behavior, or other bias must be excluded.

        It’s not perfect for sure but by and large it works.

        • Rich_Palermo says:
          0
          0

          I can’t agree that it works as well as it should. I think that money can buy a verdict and that lack of money is a detriment to obtaining justice. But, that’s getting too far afield.

          Although this particular focus group may not have been everything it could have been, maybe it will help to generate a better solution to an important problem.

    • kcowing says:
      0
      0

      The “policy bias” is that Congress won’t allow it to proceed and has said so multiple times. The NASA Advisory Council has said it should not happen either. Most planetary scientists are openly against it. Shouldn’t the forum participants have known that they were being asked to give advice to a doomed project? It is called “full disclosure”.

      • Steven Rappolee says:
        0
        0

        Their is another one of these going on now(starting in september) Its called FAST with citizen participation, it in the INspires page under closed solicitations.

        I propose a ARM redirect to martian moons boulders and the New JPL ice giants study announced at OPAG could use those SEP engines and bus

        lets pick up a boulder from Phobos and use PANDORA for close proximity operations

        http://yellowdragonblog.com

        ARM derived SEP for a Saturn/Uranus mission ! idea here,

        I love the research of Dr Okutsu

        http://yellowdragonblog.com

        The idea that Cassini at end of life could be flown to Jupiter or Uranus by a Titan flyby means you could design a mission to do this.
        So there will be a mid decade review of the decadel process ARM should be discussed then(OPAG notes)

      • Michael Spencer says:
        0
        0

        You of all people expecting things at NASA to actually make sense, Keith?

      • Steven Rappolee says:
        0
        0

        OOPs I was wrong, No regular citizens on the FAST committee instead its folks who receive federal funding for planetary missions
        Most folks who agree with me would say we do need the ARM SEP just without the ARM 🙂
        soon there will be a midterm update of the decadel survey, I think they should review the ARM and send it to a more worthy target such as the Mars.Moon system.The SEP module will be able to move around the inner solar system large habitats and other prepositioned cargo for future human missions

  3. ThomasLMatula says:
    0
    0

    Keith, I agree.

    First, A quick review of their biographies and publications indicates little evidence for understanding how to do this type of focus group research, what in the business world would be considered a variation of a consumer panel. All of their backgrounds are basically in science policy research and science outreach. None appear to have any experience, academic or practical, in the focus group design and consumer panel techniques they are using. That is why they seem to be treating this as the biggest innovation since slice bread. But if they take the time to walk over to the business schools, or if that is beneath them since they are science educations, the Psychology Departments at their universities they will find this type of research has a history decades long with established procedures. Why are they reinventing the wheel, and stumbling along as they do so?

    Second, although they give the number of participants as 98 for Arizona and 88 for Boston, there is nothing to indicate how the sample was selected, key to having a representative sample. Just because it “represents” the general demographics of a region is not evidence of it being representative of it in terms of the goals of this study. Also what does representative mean? Where is the table comparing the sample to those populations? What is the statistical significance of the difference? Understanding the sample frame they used is critical to generalizing the results. Or did they even use one? Given the use of the term recruitment I expect they simply placed advertisements and selected from those applying to fill up the cells in their sample matrix for the study creating a self-selection basis.

    Third, the success of any consumer panel depends on material being used to introduce the panel to the concepts being discussed being unbiased. This is why having a third party determine if it is biased is critical. There is no evidence they did so.

    Fourth, any material used must be shown to effectively communicate the information to the members of the consumer panel. This is why pretesting of the material to determine its effectiveness is a necessary first step. There is no evidence any pretesting or preliminary work was done. It appears the researchers felt their “expertise”in science outreach was sufficient to design good quality material to “inform” those
    selected. They did indicate they did do a pretest/post-test of the participants knowledge of planetary defense and space exploration but provide no numbers of the results only that the participants “felt good about it”


    Fifth, you don’t expect to show participants a few videos and then to discuss the issues immediately afterward and get quality independent results. Keith is correct in that. Individuals need time to digest information, think about it, and review it. That is why consumer panels like this are run in a sequence of meeting, not as a one day event.

    Last, I agree with Keith that although the idea of increasing public awareness space is good, is the public really capable of selecting a defense system for planetary defense? To put it another way, do you think the public is capable of planning the invasion of a hostile nation for the military, which is the equivalent activity? I mean, really
 Sure the nuclear blast option for a deflecting the NEO was popular, after all isn’t that how Bruce Willis saved the Earth. But is that something true experts would agree with? Definitely not the ones I served with on the old ASCE NEO mitigation subcommittee.

    • Michael Spencer says:
      0
      0

      I see we have a professional in the crowd. Thanks for running down the deficiencies- the first one I noticed was also first on your list.

      I have an idea that those in the so-called hard sciences have little regard for the techniques and analysis procedures of those in the social sciences- and setting up this type of panel is certainly in the realm of the professional social scientist.

      But I strongly disagree on your final point, where you appear to state that as the complexity increases at some point decision-making is best made by ‘specialists’.

      One could argue that this sort of decision-making is precisely the function of representative government, which is a fundamental underlying principle of our republic. It follows that any citizen is capable of becoming an informed representative.

      Nowhere does it say that the process is simple, though.

  4. RocketScientist327 says:
    0
    0

    Just my opinion here but this needs to be reported but is really not news. This is elected officials covering their… posteriors. Its nuts inside the 495 right now. There are a lot of good questions being asked and sadly, no real answers.

    We are not spending money like we should. We are not spending money in planetary, HEO, or Helio… its all about the pork. The end result does not matter… only the budget funding.

    Sad indeed.

  5. Todd Martin says:
    0
    0

    I was an ardent supporter of ARM, believing it was finally a NASA mission which would be a pathfinder for commercial BEO industry (asteroid mining). That was until Deep Space Industries said thanks, but that’s not how asteroid mining will work. You mine the asteroid at its given orbit, then move the mined materials to where you need it. NOT the entire asteroid. It just doesn’t make financial sense to spend delta-V on moving future waste material.

    • brobof says:
      0
      0

      With respect: no such thing as waste material! Radiation shielding if nothing else. Eg Lunar ‘Tailings’ used in the Stanford Torus. Later the Lunacrete of D.J. Sheppard, “Concrete space colonies”, Spaceflight 21: 3-8 (January 1979).
      With respect to Deep Space Industries according to this page http://deepspaceindustries…. they seem to be doing precisely the opposite! Looks like it’s in the bag 😉

      Just how asteroid mining will work is multiple iterations of technology in the future. And, alas, multiple generations. I would hazard a guess that the larger bodies will be mined (and colonised) in situ. The bulk of the asteroids (the small ones) will be moved to the refineries. Most, if not all the activities will be automated or teleops.

      Whilst disappointed with the current target of ARM one hopes that the mission technology can then be used for a better target: 2010 TK7 springs to mind.