Skewing The Commercial Crew Program Poll Results
Marc’s note: Last week I decided to run a poll on who our readers thought would be selected for funding in the next round of NASA’s Commercial Crew Program which many expect to be announced tomorrow. The results were surprising at first. I expected, considering the wide variety of readers we have, to have a very close poll. What I didn’t expect was the blatant padding of the results for two of the companies.
Until Monday the results were headed to what I was expecting. However at some point on Monday, “block voting” began. The votes were coming from 4 IP addresses. 3 of these IP addresses came Connecticut – specifically from the area around Norwalk (where Boeing has an office). The last IP block was traced to Sierra Nevada Corporation. It should be noted that the poll was setup with cookies so that repeat voting was not allowed. SpaceX votes were distributed across the U.S. and other countries with nothing traceable to a SpaceX office.
So presented here are two poll results. As you can see there was considerable padding of the results, but once the block voting was removed it shows a much closer result.You may interpret this unscientific poll anyway you like.
I find it interesting to note that SpaceX voters apparently didn’t see the need for padding. That and the unpadded results don’t surprise me one bit. There appears to be a lot of inertia for the “wheels on a runway” approach to spaceflight.
It is remarkable how many people prefer Dream Chaser entirely based on their perception that it’s “like the Shuttle” as if aesthetics and sentimentality have any place in picking the best tools for the job.
Matt:
Look beyond functionality for a moment and consider the optics of the situation. Being ‘like the Shuttle’ has high value in the hearts and minds department! Also, Dream Chaser may be somewhat behind SpaceX in development but it may be the shortest path to success. As a recovery system, wings and runway landings are a decade’s old mature technology (thanks to the Shuttle) with far less unknowns to… know.
SpaceX has some high hurtles to jump over with propulsive landing even if their Dragon is ‘proven’ technology. I don’t doubt they’ll succeed but, again, optics will play into the picture considering the radical nature of this new method of spacecraft recovery.
I think optics will play a major role in the final decision. Because of that, I don’t see any way to positively spin Boeing winning the contract or even being second place. They are openly the least committed to the effort anyway.
tinker
Unfortunately SpaceX won’t be landing propulsively anytime soon. For the short term, their recovery will be very Soyuz-like. I think the many years of Soyuz landing experience should be analogous to the years of shuttle landing experience.
Aren’t they going to try for propulsive landing of the first stage in some of the coming launches? And I imagine when the Dragon 2 starts flying…
The DragonV2 will land like a Soyuz (parachutes, then a small propulsive blast from the thrusters right before touchdown). This will continue until SpaceX has done extensive testing for several years on their propulsive landing system. I wouldn’t expect NASA to allow fully propulsive landings in the foreseeable future. NASA is very conservative by nature.
Jeff:
Wheels and a skid… 🙂
I too would like to see NASA get their space plane back. It’s a perfect fit for the space station as far as the limited role of crew rotation and science cargo recovery goes. If they switch the launch vehicle to Falcon 9, all the better! That would leave just the second stage being the only disposable part (and I’m working on figuring that one out!). 🙂 The combination would more reusable than the Space Shuttle by far.
tinker
Switch to a Falcon 9???? 1) everyone is married to the LV they brought to CCiCap which very well may be Space-X undoing. 2) Why would anyone switch to a vehicle that just got put on indefinite hold vs very proven flight hardware? 3) It’s not like changing tires all three companies should be deep into design and development of the LV they picked.
That is hitting below the belt with regards to what flight hardware is proven or not. SpaceX has been deliberately pushing the envelope on their hardware and building test rigs that apparently found a potential flaw.
I dare you to point out how any other launch vehicle is any better other than simply pointing out some sort of silly launch rate statistic. I certainly don’t see ULA deliberately trying to find flaws in their equipment right now on the same scale….. which is what I presume you are referring to as a more reliable vehicle and not the Soyuz or Ariane 5.
This “indefinite hold” on the Falcon 9 is not going to last more than a couple of weeks. This is more akin to what happens if there is a problem found on a 747 brake system that could potentially ground a whole class of aircraft… and just as quickly fixed.
Besides, the Falcon 9 has a flight history and has put hardware into space. That should count for something too,
Oh yeah one of them was put into the wrong Orbit as the fairing blew off the Rocket 1:80 seconds into the launch from a engine anomaly. That cost $20 Mil in a insurance hit. ULA has no reason at this point to hold up launches. Roll up hit the fuse and boom 4 minutes later all is well. Space-X is not all you believers think they are.
You make it sound like SpaceX launched only that one OrbComm satellite as the primary payload and failed to deliver, forgetting that the primary payload, namely a NASA resupply spacecraft, made the trip to its destination. NASA refused to let SpaceX make the corrective burn because there was a possibility, no matter how remote, that the burn could have failed and then crashed into the ISS.
The same thing could have happened to any launch provider, and is the kind of risk you take when you sit in the #2 slot of any launch, where you are always stuck with whatever the primary customer wanted.
OrbComm doesn’t mind though, not only did they get that insurance money (I’m not sure that happened, but I won’t dispute it either), but they also launched a separate flight that was successful and has yet another flight still scheduled that has OrbComm very pleased that they used SpaceX as their launch provider.
I would interpret it that online polls are worthless.
Well, in this case, an unexpected result of this poll was to highlight how two companies without flying hardware apparently need to enhance public perceptions while a third allows its hardware to do that.
Or SpaceX employees don’t read NASA Watch. 🙂
Seriously, though, point taken.
No, the only lesson is that online polls are worthless. To read anything into a non-scientifically conducted online poll is a fool’s game.
You miss the point.
I may have overstated my point, but I completely understand what you were saying. You found a company that has a few employees who skewed your poll. What if the other companies also stuffed the ballot but asked their employees to use their phone or do so from home so the votes wouldn’t register to one or two locations?
It is Polling 101 that the results of a non-scientific poll are not a reliable or accurate estimation of the opinion of a population. It is impossible to know if the people responding to a website’s poll, even if there isn’t ballot stuffing, are representative of that website’s readers. PZ Myers and his readers have entertainingly shown this is the case probably a hundred times over the years.
The value of an online poll is, of course, audience engagement. The poll, like allowing comments, is a tool to make a website more interactive for its readers. That’s a good thing because people who think their voice is being heard will tend to stick around, which hopefully strengthens the website’s community.
the only result a true scientific poll would show is that most americans are not even aware that the Commercial Crew program exists and that a possible downselect is imminent. “Huh? Can I just pick ‘Obama killed NASA’ as one of the options?”
Senate and congress killed NASA with the the help of NASA’s management class would be a better choice.
And the vehicle that makes NASA a Joke is SLS whether it flies or not, worse if it flies.
Actually there are two points.
1. Online polls are worthless, with padding being one of the reasons why.
2. Boeing, etc., seem to feel the need to manipulate results.
Order them in your own preference based on what you deem important.
At another blog I frequent, we constantly bombard polls NOT to skew results or distort opinion but in order to prove the first point – online polls are worthless.
If purpose is to select funding- it’s worthless.
But I don’t think it was the purpose, rather purpose was to measure response of people who read NASA Watch. And so for this purpose the padding was useless for this purpose- though indicated some people wanted to alter the poll- which is somewhat interesting,
Could one person repeatedly clear the cookie from his or her browser and then vote again?
It would be time consuming, but yes.
I can have multiple cookies and poneys?
Any number of command-line tools (curl, wget) or “headless” web browsers (PhantomJS) can trivially repeat a request such as a poll vote to a web server and discard the cookies that the server sends back.
You miss the point. We tracked the IP numbers.
I have 10+ computers behind my router at home. I use dynamic IP assignment. In a span of a few days, there might be several PCs with the same IP. Given the size of Boeing, I am not sure if they use fixed IP addressing or dynamic internally but it might just be that they have that many employees who read NASAWatch and voted. [Disclaimer: I am not an IT Expert, this is just a postulated theory…]
You can also randomize ips with proxy anonymizers like tor
Yup–every time I log out of my web browser I am eligible to vote again.
But why would I?
Sources say it’s not looking good for the announcement tomorrow, probably going to have to wait another week. 🙁 Fingers crossed for Sierra Nevada and Space-X. hoping Richard (Dick) Shelby doesn’t insist on Boeings no skin in the game/PowerPoint Tiger…
Find it interesting that Boeing is still vaporware and Sierra is still in development stages (crashed their first test flight). Only SpaceX is actually flying hardware right now. Yet their behind in poll… ROFL.
Realized immediately from original poll that some deception was going on.
To be fair, SpaceX is flying an older, cargo only, version of Dragon. Dragon V2 has yet to fly on a launch vehicle to orbit.
Boeing is passing all of its milestones, if you believe NASA’s assessment of their progress. I don’t doubt they can make CST-100 work.
But what matters to me most is flying hardware. I’m a bit disappointed that a downselect to two vehicles will happen before they all fly to orbit and back with a couple of test pilots inside. To me, that’s the minimum that they need to show before NASA ought to be cutting another company out of the competition.
I’m having a hard time understanding how it is that Boeing passed all its milestones while having only a mockup of the craft, yet SpaceX will have to fly pad and launch abort tests before it passes its milestones.
Why does the bar appear to be so much lower for Boeing?
The commercial partners defined the milestones, not NASA. In each round, the companies came to NASA with proposals for what they’d accomplish in a given timeline. SpaceX, quite simply, bit off more than it could chew with its CCiCap milestones.
Of course, Boeing and SNC, if selected next round, will still have to do abort tests.
More than half of Boeing’s award was for design reviews, not actual hardware.
Simple Boeing is on a different lap from SpaceX. NASA gave them money to get to the end of the lap. If you want to ask an import question ask “Is Boeing 2 years behind SpaceX or 4 years behind?”
Powerpoint tiger is passing all of its powerpoint milestones. Color me unimpressed.
I am surprised. the Sierra Nevada (Lockheed) Dreamchaser is a perfect 2nd choice if funding is made available for multiple vendors, but with no actual space-tested hardware and an immediate commitment to Atlas V, I never saw it as the primary choice.
Heck yeah we voted! We love our ship 😉
Trying to fraudulently win a vote on a public blog?!? How sad is that? Do they think Keith and Marc are on the committee that will decide the award or something?
Would you like to know who we would pick? 🙂
Good show Marc. I like what you did there with the two polls.
That said, it should be noted that the question was specifically “who will get the funding” which asks us to predict the future, perhaps based upon what we think NASA will do. So padding aside, the results from those of us voting honestly might look different if the question was slightly different:
Who SHOULD get the funding?
I think a bunch of people who may feel that SpaceX should get the funding, kinda based on Keith’s sentiment in his recent comment – that they have hardware flying, may have voted for Boeing in a more cynical sense of what they expect from Washington.
In additional, some folks have been expressing that they are confident that SpaceX could pull it off without further funding so therefor vote for the perceived underdog and aerodynamic sweetheart SNC, who could really use the money.
Yes, I was very specific in the question I asked and purposefully didn’t ask the question you suggested. BTW IMHO SNC has money. I don’t think it’s a stretch to say their current revenue is greater than SpaceX. SpaceX may have billions in backlog contracts, but SNC is diversified and revenue coming in from several product lines.
SNC says they will continue with their own money if they don’t make the cut–but in reality why would they spend $$$$$ for a spacecraft with no home or future? I do like Dreamchaser but I have a feeling they won’t get the nod–NASA may say they are heading in a different (Capsule) direction.
I believe they’ve said that there are other possible buyers, including Japan at the EU/UK. Dreamchaser is likely to have a future, whether or not NASA decides to chip in funding. Boeing is the only contender that has flatly stated they will kill their project without continued NASA funding, to the point of warning about impending layoffs.
Not sure but I bet that Dream Chaser may fall under an ITAR regulation and foreign countries/investors may have a hard time teaming up to utilize SNC’s bird.
ITAR ruling as of May 2014,
http://bcrdc.com/aviation_w…
Bigelow said they want two domestic service providers for human access to LEO for their stations. If Boeing says they are opting out .. SNC will have a home with BIgelow and a backup for NASA.
Vlad, I disagree SNC has to put on a positive good face for NASA and their investor’s. Neither SNC or Bigelow could afford a Delta V for launch–perhaps SpaceX could adapt it and fit it on a Falcon in the future but right now it requires a Delta V.
Rarely do sub contractors continue “as is” after they’ve lost their government assistance.
Regardless of who the winners/losers are–they are all winners and were paid handsomely for their work.
Gosh, I wonder how much Boeing pays a person to sit and monitor the web and skew polls and write tripe on blogs…
Waves a hand at Boeing …. pick me pick me
It’s likely all part of the package their web marketing consultants offer in their standard contract: “We skew polls to make the politicians think your product is the popular choice!”
Be realistic, we are talking about only 600 votes (give or take). How many people at Boeing do you think read NASAWatch. I would guess it is a helluva lot more than 600. I do not seeing this as padding or skewing, just simple demographics. If I was a Boeing employee, I would vote for my employer. And the fact that the folks at SpaceX are too busy to read NASAWatch during the day, so they probably do it at home.
This is the better interpretation.
The un padded results are thus a better view of public opinion, if useful at all
The IP addresses and timing of voting used suggest a much, much, much smaller number of people voting again and again and again.
No one is paid for that. Few individuals unafiliated with SNC or Boeing could do it just for fun.
And you know it is no one affiliated with Boeing… how? considering they had the IP addresses…
Boeing/SNC has better places to push their agenda than some dinky internet “poll”. Face it, these things are meaningless.
It is possible this scam was done by employee or relative, hence IP. But it wasn’t paid by companies. Crashing internet poll is really easy thing to do. No special company support needed – they could do it on their own.
The poll is meaningless, the preceptions it creates isn’t.
Marc: I hate to tell you, but I don’t think those who skewed the vote took your poll nearly as seriously as you did, or as some other commenters here apparently are.
The poll is just that, a poll of our readers. If someone wants to game it, as is the case, then obviously to them it’s worth the effort to do so. However the service we used, PollDaddy, is a good service and provides us with some good tools which is how we were able to discover the padding and origins.
My point was that the skewers hijacked your poll to make it into a “less filling!! Tastes great!!” kind of thing. They have no idea “who will get funded”.
If it changes just one blogger/commenter who is on the fence and starts posting as the poll gamers wanted… it was worth it to them.
They want comments etc to go viral .. reposted on facebook … the cascade effect.
Looks like this poll makes visible certain aspects, kind of embarrassing that IP addresses shows who voted for who. Although as Slashdot says, “This whole thing is wildly inaccurate. Rounding errors, ballot stuffers, dynamic IPs, firewalls. If you’re using these numbers to do anything important, you’re insane.” Of course who is chosen for CCP should be based on technical merit, not by internet ballot (but politics may be the deciding factor). I find this an interesting psychological study as it was phrased “who will get” as opposed “who should get.”
Interesting and a little disappointing.
My choices were based on capability and funding so SpaceX and SNC with Boeiing last. I simply don’t believe they can change their culture sufficiently away from their oldspace habits and so far, all indications support that premise.
But never mind, the final result will soon be in.
Cheers.
Reminded me of an old Sports Night episode…or is it exclusive behind-the-scenes footage at SNC and Boeing?
http://youtu.be/vL657mH90Yo…
Parabolic Arc poll has SpaceX way out front, with Boeing and SNC tied. I still think NASA will wait for Congress to come back. Usually NASA does not do much on Monday or Friday. They also will show it scheduled far in advance. Don’t know who it will be, but here is how it works.
Contract Terms: NASA’s contract, whether with one
company or more, will include at least one crewed flight test per
company to verify the integrated rocket and spacecraft system can
launch, maneuver in orbit, and dock to the space station, as well as
validate all its systems perform as expected. Once the test program has
been successfully completed and the systems achieve NASA certification,
the contractor/s also conduct at least two, and as many as six, crewed
missions to the space station, effectively ending the nation’s reliance
on foreign providers
From a NASA blog. Perform as expected. The problems with the cargo ships are known, so they should not have them. An uncrewed test flight on the same vehicle. Should make it to ISS without an abort.
I think the 2-6 refers to the number selected. 2for3, 3for2 or 6 for 1 provider.
Just read that Dragon will land with parachutes with rocket softening until Dragonfly proves the system.
No, you can go to your browser, manually delete the cookie (not always easy to do) and then vote again. It takes a concerted effort on the part of one source to do this again and again and again so as to rack up the points on a poll hosted on one specific blog that relates to a large potential contract for one’s company.
I would imagine Boeing has people seaching the web and any and all polls found are “worked” also in comment sections they are posting pro Boeing.
If it is an actual campaign … which I would imagine it is because it is a multi billion dollar award, then any and all polls are delt with if possible, comment sections get posted to. It is about perception and creating a narrative for Boeing.
I doubt that there is some kind of vast Boeing/SNC conspiracy to pad online polls that will have no impact on whether they get the contract or not.
No vast conspiracy needed. One or two “hackers” in basement with strong feelings and loose morals is enough.
Didn’t boeing get busted for industrial espionage when documents were obtained from a competitor over a couple billion dollar contract?
You are silly if you think this is beyond what Boeing is capable of.
I wasn’t saying Boeing is as pure as the wind driven snow. I was saying that it makes no sense to mess with an online poll that has zero impact on the award of commercial contracts.
Could a couple of people who are not fans of the pro-SpaceX bent of NASAWatch skew the poll? Sure. However, IMO that does not show some kind of vast conspiracy from Boeing and SNC that is being implied.
Can you point to the post of mine where I wrote about a “vast conspiracy”?
All it takes is a low level manager, in IT, shootin’ the shit around the water cooler saying “I saw this poll on NASA watch and we are in last place, spaceX is probably spaming it, I sure wish it would reflect the truth”
And having talked to hundreds of managers… I know for a fact that kind of stuff is common in corporate culture.
This is the most meaningless, non-scientific, worthless poll imaginable. You decided to remove what you felt were unfair entries, thereby showing the inaccuracy of your poll.
I still don’t understand why US will continue to
rely on Putin’s mercy. Then there are not two or three companies, there is only one launch provider left.
Ya … Boeing would NEVER do anything underhanded when multi billion dollar contracts are on the line…..
Boeing Managers Charged in Plot to Steal Trade Secrets from Lockheed Martin
http://www.justice.gov/crim…
Boeing worker accused of stealing documents – Gerald L. Eastman allegedly stole documents that could cost Boeing up to $15 billion
http://www.networkworld.com…
Boeing Employee Charged With Stealing 320,000 Sensitive Files
http://www.informationweek….
After reading the discussion below I am struck by one major point concerning the worth of the pole.After all the discussions of who would or how would they trick the poll to rack up numbers for their prime, no one asks why really. Specifically, does anyone think this poll will be factored into NASA’s down select decision or are they just acting like kids wanting to play at a parents job!