NASA’s newly-appointed associate administrator for the Human Exploration and Operations (HEO) Mission Directorate, Kathy Lueders, and Commercial Crew Program Manager Steve Stich have discussed the outcome of the High Visibility Close Call review of last December’s uncrewed test flight of the Boeing CST-100 Starliner.
Lofted atop a United Launch Alliance (ULA) Atlas V booster from Space Launch Complex (SLC)-41 at Cape Canaveral Air Force Station, Fla., the Starliner was delivered smoothly to orbit for a planned week-long mission, but suffered a number of off-nominal issues which precluded a docking at the International Space Station (ISS) and necessitated a premature return to Earth two days later. Tuesday’s media teleconference identified numerous technical and organizational root causes for December’s troubled mission.
In the aftermath of Starliner’s 22 December landing at White Sands, N.M., NASA Administrator Jim Bridenstine noted that “a lot of things went right” and many challenging test objectives were satisfactorily completed. One of the most significant was the first successful “land” landing of a human-rated crew capsule in U.S. spaceflight history; all previous capsules—Mercury, Gemini, Apollo and Crew Dragon—were and are designed to splashdown in the ocean at the end of their missions. However, Mr. Bridenstine stressed that the problems were a good example of “why we test”.
In early March, a joint NASA/Boeing Independent Review Team (IRT) completed its initial investigation, which centered upon three principal anomalies. Two of these were classified as “software coding errors”. Firstly, the Starliner incorrectly synced its Mission Elapsed Timer (MET) with the Atlas V before the onset of the terminal countdown, which led the spacecraft to presume it was at a different stage of the mission following separation from the rocket. As a result, it did not conduct the correct maneuvers at the right time. Secondly, the MET anomaly led to a review of other flight phases, which led to the detection and correction of another software issue which might have impaired the successful separation and disposal of Starliner’s service module at the end of the mission.
“Regarding the first two anomalies, the team found the two critical software defects were not detected ahead of flight despite multiple safeguards,” NASA noted in February. “Ground intervention prevented loss of vehicle in both cases. Breakdowns in the design and code phase inserted the original defects. Additionally, breakdowns in the test and verification phase failed to identify the defects preflight despite their detectability. While both errors could have led to risk of spacecraft loss, the actions of the NASA-Boeing team were able to correct the issues and return the Starliner spacecraft safely to Earth.”
The third issue concern revolved around an unexpected loss of Space-to-Ground Communication, where an intermittent forward-link issue impeded flight controllers’ ability to command and control Starliner. This was subsequently traced to radio-frequency interference with the spacecraft’s communications system. Had a human crew been aboard the Starliner, this could have adversely affected Mission Control’s ability to achieve reliable voice communication with the astronauts.
In the aftermath of these findings, it was noted that “several technical and organization issues related to Boeing’s work” had been identified and a self-review by NASA uncovered “several areas where the agency can improve its level of participation and involvement into company’s processes”. An Organizational Safety Assessment (OSA) was executed to evaluate Boeing workplace culture, which included employee interviews and sampling across a broad personnel cross-section in order to achieve “a comprehensive safety assessment”.
As a result, NASA declared the December anomaly as a “High Visibility Close Call”, one of 24 such procedural requirements thus designated since 2004. A particularly significant example of an earlier HVCC occurred back in July 2013, when Italian astronaut Luca Parmitano suffered water intrusion into his helmet during a spacewalk.
More than 60 corrective actions were identified after the NASA/Boeing IRT investigation, categorized into four key areas. Boeing will review and correct the coding for the MET and service module disposal burns, strengthen its review processes, enhance the fidelity of its software testing and ensure overall “product integrity”.
As this process got underway, in early April Boeing reported that it would perform a second uncrewed test flight, at no added cost to the government, possibly later this fall. And work continues unabated towards both that mission and the first voyage of Starliner with astronauts aboard, anticipated to occur in the early part of next year. Just last month, Boeing conducted additional tests of the spacecraft’s parachute system over White Sands, successfully demonstrating their effectiveness under “dynamic abort conditions and a simulated failure”.
In its overview, NASA outlined a total of 80 recommendations that it is addressing with Boeing after the conclusion of the HVCC process, but cited the full list as “company sensitive and proprietary”. However, the areas to be addressed fall into five broad categories.
Twenty-one relate to testing and simulation, emphasizing the need for greater hardware and software integration testing, performance of an end-to-end “run-for-record” test before each flight, reviews of subsystem behaviors and limitations and the identification of simulation and emulation gaps. Ten pertain to assessment of all software requirements with multiple logic conditions to ensure full test coverage. Thirty-five others require modifications to change board documentation, bolstering the numbers of participants in peer reviews and test data reviews and and “increasing the involvement of subject matter experts in safety-critical areas”. Seven deal with the need for organizational changes to the NASA/Boeing safety culture, whilst another seven focus upon updates of the Starliner software coding issues which caused the MET and service module separation problems.
Speaking at yesterday’s media teleconference, Ms. Lueders underscored the fact the December’s mission as an uncrewed test flight and that “every mission like this is a learning experience”, not just in terms of the Commercial Crew Program, but also the wider objectives of HEOMD.
Mr. Stich added that over 20 interviews with NASA and Boeing personnel were conducted as part of the High Visibility Close Call investigative process and they have extensive “staffed-up in the program” to collaborate more effectively.
Although he did not reveal a launch date for the second Orbital Flight Test mission of Starliner—dubbed “OFT-2″—Mr. Stich pointed to an expectation that it may occur “as early as later this calendar year”.
Hardware for that mission, Mr. Stich said, is taking shape in the Commercial Crew and Cargo Processing Facility (C3PF) at the Kennedy Space Center (KSC), with testing of the Starliner software upgrades considered the major “pacing item” at the present time.
“NASA and Boeing have completed a tremendous amount of work reviewing the issues experienced during the uncrewed flight test of Starliner,” said Steve Jurczyk, associate administrator at NASA. “Ultimately, everything we’ve found will help us improve as we move forward in the development and testing of Starliner, and in our future work with commercial industry as a whole.”
“I can’t stress enough how committed the Boeing team has been throughout this process,” added Phil McAlister, director of commercial spaceflight development at NASA. “Boeing has worked collaboratively with NASA to perform these detailed assessments. To be clear, we have a lot more work ahead, but these significant steps help us move forward on the path toward resuming our flight tests.”
Aside from its troubles, December’s test flight achieved a number of remarkable successes, including the first “land” landing of a human-rated capsule in U.S. spaceflight history. Additionally, it marked the first successful launch of a human-rated Atlas V and thoroughly checked out Starliner’s propulsion systems, communications systems, navigation systems and the NASA Docking System (NDS) that it will utilize to dock at the ISS.
“As vital as it is to understand the technical causes that resulted in the flight test not fulfilling all of its planned objectives, it’s equally as important to understand how those causes connect to organizational factors that could be contributors,” said Mr. Jurczyk. “That’s why NASA also decided to perform a high visibility close call review that looked at our combined teams.”
.
.
FOLLOW AmericaSpace on Facebook and Twitter!
.
.
