Summer of Discontent: 25 Years Since the Shuttle Hydrogen Leaks (Part 2)

In one of the most historic shuttle-era photographs ever taken, this image shows the STS-38/Atlantis stack (at right) returning to the Vehicle Assembly Building (VAB) for repairs on 9 August 1990. In doing so, it passed the STS-35/Columbia stack (at left), which was returning to the launch pad after several weeks of extensive repair work on its hydrogen disconnect hardware. Photo Credit: NASA, via Joachim Becker/SpaceFacts.de

In one of the most historic shuttle-era photographs ever taken, this image shows the STS-38/Atlantis stack (at right) returning to the Vehicle Assembly Building (VAB) for repairs on 9 August 1990. In doing so, it passed the STS-35/Columbia stack (at left), which was returning to the launch pad after several weeks of extensive repair work on its hydrogen disconnect hardware. Photo Credit: NASA, via Joachim Becker/SpaceFacts.de

Twenty-five years ago, at the beginning of August 1990, no shuttle had speared for the heavens in more than three months, since the STS-31 Hubble Space Telescope (HST) deployment mission. However, NASA’s hopes to stage as many as nine missions by its three surviving orbiters, Columbia, Discovery, and Atlantis—and, in so doing, equal 1985 for the maximum number of U.S. piloted launches in a single calendar year—proved short-lived. As outlined in yesterday’s AmericaSpace history article, Columbia’s long-awaited STS-35 mission, carrying the ASTRO-1 research payload, was originally targeted for a 16 May liftoff, but was repeatedly postponed, due to a maddening series of hydrogen leaks from the orbiter’s 17-inch-diameter (43 cm) disconnect hardware. Situated in the shuttle’s belly, a pair of massive disconnects served to deliver propellant from the External Tank (ET) to the three liquid-fueled main engines, and their criticality to the safety of flight was profound. Said STS-35 Commander Vance Brand of those agonizing delays: “It just was driving the engineers at the Cape crazy, trying to find the source of the leak.”

At length, on 9 August 1990, Columbia returned to Pad 39A—passing her sister ship, Atlantis, herself affected by a hydrogen leakage issue, on the crawlerway—it seemed that the STS-35 problems had been rectified. “Incorrect torqueing of bolts around the flange interface between the tank and the orbiter caused the Atlantis mishap,” Flight International told its readers on 8 August. “The Columbia leak was caused by a faulty seal in the drive mechanism used to close the flapper valve in the disconnect.” Meanwhile, STS-35 was rescheduled for launch on 1 September, but misfortune was far from finished with the unlucky flight and her seven-man crew: Commander Brand, Pilot Guy Gardner, Mission Specialists Jeff Hoffman, Mike Lounge, and Bob Parker, and Payload Specialists Sam Durrance and Ron Parise.

Two days before launch, an avionics box on the ASTRO-1 payload failed and required replacement, necessitating a short additional delay. Then, on the evening of 5 September, technicians began to load liquid oxygen and hydrogen aboard the ET … only the discover hydrogen gas again leaking into the shuttle’s aft compartment. The maximum allowable rate was 660 parts per million, but the actual rate was in excess of 6,500 ppm. After more than three months of repairs, tanking tests, and more repairs, it now became apparent that two discrete leaks had evolved. Data indicated that the original leak from the disconnect apparatus had now gone, but a new leak had emerged, somewhere deep within Columbia’s aft main engine compartment.

Pictured in September 1990, the problem-plagued STS-35 stack is shown in the foreground, on Pad 39A, with the STS-41 stack in the background on Pad 39B. Photo Credit: NASA

Pictured in September 1990, the problem-plagued STS-35 stack is shown in the foreground, on Pad 39A, with the STS-41 stack in the background on Pad 39B. Photo Credit: NASA

Initial diagnostics placed it close to the orbiter’s recirculation pump package inset or the manifold. Time to resolve the problem was short, for the time-critical STS-41 mission aboard Shuttle Discovery to loft the Ulysses solar probe was targeted for a short “launch window” between 5-23 October 1990. So urgent was the need to get the long-delayed Ulysses launched that three veteran astronauts—Commander Loren Shriver, Pilot Jim Wetherbee, and Mission Specialist Dave Hilmers—had already been assigned to backup the prime crew of Commander Dick Richards, Pilot Bob Cabana, and Mission Specialists Bruce Melnick, Bill Shepherd, and Tom Akers. Throughout the summer of 1990, Shriver, Wetherbee, and Hilmers trained in tandem with the STS-41 crew, in case illness should preclude anyone from flying. The decision had been taken following the illness of STS-36 Commander John “J.O.” Creighton, whose severe cold in February 1990 had led his mission to be delayed by several days. Any such delay to STS-41 had to be avoided at all costs.

If STS-35 did not fly soon, it would likely be postponed until after STS-41. A package of three hydrogen recirculation pumps in Columbia’s aft compartment were removed and replaced, together with a damaged Teflon seal on one of the main engines, but to no avail: In the hours preceding another launch attempt on 18 September, leaking gas was again detected, emerging at a rate of 6,700 ppm. The looming Ulysses window meant that NASA was now out of options. STS-35 would be indefinitely postponed until the hydrogen problem was resolved. Space Shuttle Program Manager Bob Crippen—himself a former astronaut—assembled a “tiger team” to investigate the disconnects and totally “retorque” Columbia’s entire liquid hydrogen system.

Crippen assigned senior engineer Bob Schwinghamer, from the Marshall Space Flight Center (MSFC) in Huntsville, Ala., to lead the investigation. In his NASA oral history, Schwinghamer remembered then-NASA Deputy Administrator J.R. Thompson telling him, without a hint of humor, that he had a one-way ticket to Florida; he was not to return to Alabama until the hydrogen leak was solved. Schwinghamer’s team spent three months at the Kennedy Space Center (KSC), from September through December, setting up an intricate fault tree and co-ordinating a huge number of personnel, spread across several NASA centers. By the time they completed the final “tanking test” on 30 October 1990, Schwinghamer could confidently declare that Columbia was now “the soundest, leak-free orbiter at that time in the fleet.” The explosive nature of hydrogen meant that a leak of any sort could not be tolerated—even though the main propulsion system was designed to overpower leaks with a nitrogen purge—and it certainly surprised Vance Brand that an orbiter was being grounded for such a long period. It marked a change in attitude from the way in which NASA management had worked before the Challenger accident.

Commander Vance Brand leads his crew out of the Operations & Checkout (O&C) Building, just before midnight on 1/2 December 1990, bound for Pad 39B and Columbia. Photo Credit: NASA, via Joachim Becker/SpaceFacts.de

Commander Vance Brand leads his crew out of the Operations & Checkout (O&C) Building, just before midnight on 1/2 December 1990, bound for Pad 39B and Columbia. Photo Credit: NASA, via Joachim Becker/SpaceFacts.de

The September 1990 delay was clearly remembered by Jeff Hoffman, many years later. Rather than risk having their children out of school for a lengthy spell, several of the crew decided to pay, personally, for air tickets from Houston to the Cape and asked neighbours to take them to the airport. In the hours preceding the 18 September attempt, the Hoffman children—aged 11 and 15 at the time—were seated aboard the aircraft, with one engine in the process of starting up, when the Continental Airlines departures desk received a call from Hoffman himself. Another leak had emerged; there was no point in the children flying down to Florida. At the last possible moment, the engines fell quiet, the doors opened, and the disappointed children got off. Not until the first week in December would they see their father launched into space.

By this time, the effort to fix the hydrogen leaks had cost NASA $3.8 million and the agency suspected that Columbia’s problems originated from a complete disassembly of the main engines following the STS-32 mission in January 1990 to remove polishing grit from her fuel lines. When the engines were reassembled, the seals were imperfectly fitted and minute glass beads were found to have contaminated the hydrogen disconnect hardware. From the beginning of June 1990, the attention of engineers was drawn solely to the disconnect leak, which posed a more serious problem, and the seals were overlooked. “As a result,” explained Flight International in mid-November, “NASA has introduced a new processing program in which key engine components will be checked for leaks before the engines are finally assembled.”

STS-35 finally lifts off, almost five years after ASTRO-1 should have been observing Halley's Comet. Photo Credit: NASA

STS-35 finally lifts off, almost five years after ASTRO-1 should have been observing Halley’s Comet. Photo Credit: NASA

In late-November 1990, NASA announced that a new hydrogen dispersion apparatus would be added to the launch pad’s Mobile Launch Platform (MLP) for future missions, beginning with STS-39. The system would provide a nitrogen-rich air flow around the disconnect hardware, thus helping to disperse hydrogen concentrations. As for the disconnects themselves, NASA was already working on plans with the shuttle’s prime contractor, Rockwell International, to develop an upgraded system, somewhat narrower at 14 inches (35.5 cm) in diameter, and contracts worth upward of $27.6 million were awarded in February 1991. It was expected that the new disconnect hardware would prevent inadvertent closure of the flapper doors during ascent, which threatened catastrophe. As circumstances transpired, the new disconnect project was canceled by NASA in 1993, although much of its technology was employed to improve the safety and reliability of the existing hardware. Furthermore, NASA instituted more rigorous rules around the issue of hydrogen leakage, embedding them more firmly within its Launch Commit Criteria (LCC).

Having been dogged by such cruel luck for so many months, it was a profound relief when Discovery flew STS-41 in early October, following by Atlantis on STS-38 in mid-November. Discovery’s disconnect hardware had shown none of the leakage issues which had so plagued her two sisters. Yet the summer of 1990 highlighted that the entire shuttle system—orbiter, ET and twin Solid Rocket Boosters (SRBs)—remained acutely “experimental” in the minds of the men and women who processed it for flight. More than one veteran astronaut has remarked that simply getting the stack into a position whereby it could launch into space was equally as miraculous as any of the technology aboard.

By the tail end of November, STS-35 was rescheduled for an opening launch attempt on 2 December, with weather conditions anticipated to be 70 percent favorable. When Vance Brand and his crew arrived at KSC in their fleet of T-38 jets, they were in a jubilant mood. “We’re back! We’re ready!” yelled Brand. Some prophets of doom noted that STS-35 would be the 13th post-Challenger launch, but Sam Durrance was personally convinced that, after so many delays, their day had finally come. Jeff Hoffman was equally optimistic. He knew that comets were supposed to be bringers of ill-fortune and many comets had been associated with ASTRO-1: Halley being the obvious one, but a string of other, lesser known, celestial wanderers had made the rounds as STS-35 struggled to get ready to fly. By December 1990, there were no comets available to observe. “We all know comets are harbingers of bad news,” Hoffman rationalised. “This time, we have no comet. So we’re going to go.”

And go they did.

As darkness fell on the evening of 1 December, the crew breakfasted in the Operations & Checkout (O&C) Building, before heading to the suiting room to don their bright-orange Launch and Entry Suits (LES). At one stage, a fully-suited Parker could not resist but help one manager with his tie. The crew headed into the glare of camera flashes and well-wishes shortly before midnight and were transported to Pad 39B. Liftoff of Columbia on her 10th overall voyage occurred a 1:49 a.m. EST on 2 December, kicking off an ambitious mission which would experience both success and disappointment, but for Brand the triumph of resolving the “Summer of Hydrogen Leaks” was a significant milestone in the post-Challenger era. Years later, he remembered that, despite the frustration, the “very commendable effort” by Bob Schwinghamer’s team “found not one leak, but three or four small leaks that added up to one big leak.” By the time STS-35 finally rocketed into orbit, Brand said, “Columbia, with regard to hydrogen plumbing, was the tightest ship in the fleet.”

 

 

This is part of a series of history articles, which will appear each weekend, barring any major news stories. Next week’s article will focus on the 10th anniversary of the STS-114 landing, the first safe return of a space shuttle to Earth in the wake of the Columbia disaster.

 

 

Want to keep up-to-date with all things space? Be sure to “Like” AmericaSpace on Facebook and follow us on Twitter: @AmericaSpace

 

1 comment to Summer of Discontent: 25 Years Since the Shuttle Hydrogen Leaks (Part 2)