NASA and Terminating Contracts

I was surprised to learn that in 2010 NASA had 16,343 contracts in place for procurement of goods and services, of these only 28 were terminated. That’s less than 0.2% of their contracts.

Either NASA is very good at finding efficient vendors and accurately forecasting requirements and contingencies within the contract, or they aren’t being decisive enough in adapting to project changes or dealing with cases of extreme vendor underperformance.

How Good Is NASA At Project Management?

Source: NASA Ares Project Office

NASA too experiences schedule and cost overruns. Of the 10 NASA projects that have been in implementation phase for several years, those 10 projects have experienced cost overruns of 18.7% and launch delays of 15 months. In 2005 Congress required NASA to provide cost and schedule baselines, so no long term data is available. NASA’s projects are consistently one of a kind and pioneering, therefore uncertainty is likely to be higher than for other sorts of projects.

These cost and schedule overruns are largely due to the following factors:

External dependencies

The primary external dependencies that cause problem for NASA are weather issues causing launch delay and issues with partners on projects. NASA projects with partners experienced longer delays of 18 months relative to 11 months for those projects without partners.

Technological feasibility

As the Government Accountability Office assessment states: “Commitments were made to deliver capability without knowing whether the technologies needed could really work as intended.” This is so often a cause of project failure, see my articles on the Sydney Opera House, Denver Airport Baggage System and many
others for examples of how common this cause of failure is.

Failure to achieve stable designs at Critical Design Reviews

90% stability at Critical Design Review is cited by the Government Audit Office as a goal for successful projects, which is consistent with NASA’s System Engineering Handbook. Without this, designs are not sufficiently robust to execute against. It’s clear that NASA takes Critical Design Reviews seriously, but doesn’t always achieve 90% stability. The exact value varies across projects, but appears to be in the 70-90% range for most NASA projects. Raising the stability level at theseCritical Design Reviews would reduce project risk.

Though long, the Government Audit Office report contains lots interesting of further detail and can be found here.

True Scarcity

Interview with Fred Brooks author of the Mythical Man Month in Wired written by Kevin Kelly can be found here. As Stephen Dubner points out in his Freakonomics blog, Fred makes the interesting point that the scarce resource on a project isn’t always money, time or people. For example, when building an oceanfront property the constraint is beachfront square footage, on the first NASA moon landing, the constraint was lightness. Therefore determining what your true contrainst is can help you stay focused on non-traditional projects.

Fred also subscribes to the principle, from Edwin Land, of starting with the vision and working back from that to a technically feasible design rather than aggregating a set of attainable features together to create whatever vision results, which might be easier but has less impact. He argues working back from an ambitious vision is the model that Apple follows to achieve visionary design.

He also mentions that he’s changed his thinking from doing a first design and then throwing it away and building a second version, towards a process of ongoing iteration.

The Wired article can be read online here.