SpaceX’s Launch Anomaly Wasn’t Only Flaw on CRS-1

SpaceX’s Dragon capsule splashed down on October 28 after a partially successful cargo resupply mission to the International Space Station. Photo Credit: SpaceX

It’s been 18 days since SpaceX’s Dragon capsules splashed from its latest mission to the International Space Station. The mission, the first in the commercial resupply contract the company holds with NASA, was a major step. SpaceX CEO Elon Musk said the recent flight “signifies the restoration of America’s ability to deliver and return critical space station cargo,” adding that “the reliability of SpaceX’s technology and the strength of our partnership with NASA provide[s] a strong foundation for future missions and achievements to come.”

But the mission wasn’t a complete success. As those who watched the launch will remember, one of the Falcon 9‘s first stage Merlin 1C engines shut down early when the computer detected a loss in pressure in the combustion chamber just 79 seconds after lift off. The failure put the Dragon in a lower than expected orbit. An Orbcomm satellite, the secondary payload on the mission, failed to reach orbit entirely and fell back to Earth less than a week later. Orbcomm declared the mission a total loss.

Now, as engineers from NASA and SpaceX continue to analyze the flight data to determine the cause of the engine loss – no smoking gun yet – it seems there were more problems on the flight than the launch anomaly.

ISS program manager Mike Suffredini, speaking to the NASA Advisory Council’s Human Exploration and Operations Committee on Wednesday, made the scope of problems on the mission clear.

There were problems while the Dragon was berthed to the ISS. One of the capsule’s three computers failed. Flight Computer-B “de-synched” from the other two due to what Suffrendini suspects was a radiation hit. Like the Falcon 9 losing an engine, the Dragon can still function with two computers so SpaceX opted to proceed with only two active units. The third was rebooted after the anomaly, but SpaceX opted no to “re-synch” it with the other two.

SpaceX CEO Elon Musk in front of the business end of a Falcon 9 at the company’s launch site at Cape Canaveral. Photo Credit: SpaceX

Radiation took other victims on the flight. One of the Dragon’s three GPS units, its Propulsion and Trunk computers, and the Ethernet switch all malfunctioned likely due to radiation. Lucky for the mission, these units were rebooted and recovered in short order.

The radiation is a serious issue; it’s an inescapable part of spaceflight. Suffredini said that SpaceX is now facing the decision of whether or not it needs to use radiation-hardened parts. In the case of the flight computers, this change will bring with it a weight penalty and a decrease in processing power. Cost will be another factor in making the switch to radation-hardened equipment; it could be prohibitively expensive.

The Dragon didn’t leave its problems in orbit. One of the Draco thrusters rockets failed. All three coolant pumps failed after the capsule splashed down. The onboard GLACIER freezer lost power at splashdown, possibly due to water interacting with the external power units. Designed to return samples from the ISS at a controlled temperature of -139ºF, the samples were much warmer at -85ºF when the freezer was recovered and turned back on.

What this larger list of failure might mean for SpaceX’s future launches isn’t clear yet; a full assessment of the mission is still underway. But it could have a significant impact. Understandably, NASA wants to make sure anything that comes in the vicinity of the ISS won’t be a threat to the station. As Suffredini explains it, NASA doesn’t have the “go/no-go” authority over SpaceX launches but it is a very influential customer. SpaceX’s commercial resupply contract with NASA requires the company deliver a certain amount of cargo to the ISS over 12 (now 11) launches. But if NASA loses confidence in the company for any reason, it can opt to not put its cargo on the rocket. And without the major buyer’s payload, that rocket likely won’t fly.

For the time being things are still on track with SpaceX’s next launch scheduled for March of 2013. SpaceX hasn’t made any comments on the ongoing investigation. Musk said simply on Wednesday that it will likely be a number of weeks before the company releases the results of its investigation into the anomaly.

Missions » ISS » COTS »

13 Comments

  1. It’s a little worrisome to this fan of SpaceX that an engine shutdown occurred seemly so early in the plotted program of the falcon rocket. I hope that the apparent “failures to accumulated thrust time ratio” improves.
    I wonder, since the radiation effects had to be expected with non-hardened electronics, how close or far from expectations the actual effects were.
    It seems that with the combination of the necessity of secrecy in the technologies and the lack of web sites with a bent for my preference in an almost anal level of detail of what went on (adjusted for a level of understanding only somewhat above the average) during anomalies, it’ll take months, if ever, to learn the details.
    I was pretty frustrated with the lack of a deeper technical explanation into SpaceX’s decision to abandon the parachute recovery of the first stages.
    My inquiries resulted in answers only from people as ,pretty much, ignorant as I on the subject though possibly better learned in flight engineering in general.
    Here’s hoping that things get better…all around.

    • I was always curious about the water landing of first stages too, and why it wasn’t pursued. It worked pretty well for the Shuttle SRBs, after all. Sure, it may not be as sexy as a rocket landing itself, but it must’ve provided at least some cost savings in the interim.

  2. Certainly our colleagues at SpaceX are getting a lesson in why space flight is so expensive, because of the consequences of things like radiation. Rad hardened computers are more expensive, you can cut costs by just buying COTS units. But then do you want to risk a flight to save a minor amount of money? They are offering discount flights – and will have to go through the learning curve just like ULA and other companies did decades ago.

  3. I don’t understand ‘prohibitively expensive’ in terms of rad-hardened computers. Surely these must be fairly standard for spacecraft. And the costly bit of computers is the software nowadays, not the hardware.

    • cui, the first expense increase that comes to mind is the additional weight, and that is likely the major concern. The more weight we put into space, the more fuel needed to get it there. Ounces matter. Shielding will amount to additional pounds, it could tack on tens of thousands more dollars per flight. Radiation from solar storms has increases and decreases in cycles. It hasn’t been a bother since the space program began, but the sun is going through a solar maximum now. That will persist for some time and then gradually decrease over a few years. My assumption of why shielding is in debate is: “Can we hang in there until it’s over?”

  4. It would have been out of this world if SpaceX would not encounter any problems at this stage. I am following space events since 1960 and comparing what other space players experienced before – SpaceX is not doing bad at all. Sure SpaceX also will be learning by doing & experience their set-backs & lessons. But the list of problems so far is much smaller than expected. I keep my fingers crossed for Elon to achieve what nobody achieved before – space flight at a reasonable price and missions that are affordable (even to governments). It’s about time !

  5. Let us not be overly pessimistic and worried. This was an historical first. Yes there were problems, yes, there will continue to be problems. But the biggest problem would be losing faith and giving up. Challenges help us learn and grow. The more challenges we experience the more we can learn. SpaceX is on the cutting edge of technology and space exploration. Kudos to them for even trying. Kudos to Mr. Musk for having the courage and tenacity to stick through it.

    Keep going SpaceX! I am behind you! Take the time to learn and make each successive event even better than the last. One step at a time and we will soon be a spacefaring race.

    • I don’t think anyone is saying SpaceX should shut down. They have a great record. However, pride goeth before a fall, and we need to be sure that we note their failings along with their accolades.

  6. Hi Amy,

    An excellent factual article.

    A couple of points:
    – Space X decision to use COTS (Commercial Off the Self – not Commercial Orbital Transport) electronics may now give them considerable problems. The radiation hardened components are not only heavier but have different volumes/geometries. This would require repackaging of the computers and the changes in processing power would require re-writing software. That takes time and cost money.

    – The engine out anomaly (the second such occurrence in four flights) is going to require significant analysis and this will require telemetry data. It is my understanding that Space X minimized its telemetry requirements to save money, so that may make doing the required analysis problematic.

    – The failure of the GLACIER freezer is also troubling as this “Down Mass” capability was one of the big selling points for the Dragon Cargo Vehicle. Is there yet any information as to what caused the problem? Is there yet any information whether or not the problem compromised the returned samples?

    Thanks.

    • Joe – I am not privy to the details, but the announcement said that the samples were recovered at -85 (F or C, hardly matters). When we flew freezers on Spacelab, as long as the samples stayed frozen they were usable.
      Supposedly the GLACIER problem was water intrusion – that indicates a possible problem with sealing. Where else did the water go?
      And their earlier decision to use more COTS hardware will likely cost them now! As you say, if they had to switch it would cost both time and money. And people wonder why SpaceX can underbid the “dinosaurs” such as ULA.

      • Thanks Charles,

        Good news (at least potentially) on the first point, but more troublesome on the second.

        – Not only “Where else did the water go”, but how did the water get in?
        – Was it let in the main pressured volume?
        – If so when did the leakage path open (if water can leak in certainly atmosphere can leak out)?
        – That did not appear to happen on this flight, but when and why did the leak path open?

        “And their earlier decision to use more COTS hardware will likely cost them now! As you say, if they had to switch it would cost both time and money. And people wonder why SpaceX can underbid the “dinosaurs” such as ULA.”

        Agreed.

  7. Excellent article!

    My own take … it’s still early days for SpaceX. The mission, overall, was a success, but there’s still stuff to learn and some changes to make. None of this should be a surprise.

    What’d be nice is a comparison of SpaceX’s experience with lessons learned on Mercury and Gemini. Were the early US space launches without flaws? How hard was it to determine problems and make fixes way back then and what are the implications for changing things today?

One Ping

  1. Pingback:

Kepler Completes Prime Mission

All Change for Station Crew as Ford Prepares to Take Command