Monthly Archives: April 2010

A Study In IT Project Failure

source: mayhem (via Flickr)

A data rich study into IT project failure within the European Union by Dr John McManus and Dr Trevor Wood Harper can be found here.

They cite “total reliance on methodologies” as a cause of failure. This is interesting because you might think that adopting a methodology is a positive step, but they find that a methodology is adopted as a substitute for having experienced professionals in place.

It is also interesting to examine their data on when projects failed. Of course, this is a fairly small data set, but it is interesting to note how few projects fail at the testing or implementation phase. One might think that that would be the time when the rubber meets the road, and most failures would emerge, but it seems that either organizations are savvy enough to spot problems early, or to grin and bear it and redouble efforts at the later stage of the process when problems emerge.

Percentage Chance of Failure For Each Stage of the IT Development Lifecycle

What Failed Projects Have In Common

I’ve written about a number of failed projects (see the Project Failure link at the top of this blog). There is a common thread across all of them. Lack of clear requirements. It is the lack of clarity that is the key point.

The US Census devices had “thousands” of “unreconciled” requirements, the Sydney Opera House had an attractive picture, but no hard factual underpinnings for how the design could ever work, the FBI’s virtual case file project were “ill defined and evolved as the project progressed” and during the H1N1 vaccine project the scope of work changed materially. All of the projects I’ve just described failed.

Now this is a short post, so I won’t detail strategies for project success, but it does seem that a sure fire way to fail is to have vague, changing or conflicting requirements. Interestingly, bad estimates don’t lead to massive delays, vague requirements do. Equally, poor budgeting may not cause massive cost overruns, vague requirements do.

Of course, to run a successful project you need to succeed in a host of areas. It’s not just about the triple constraint of cost, schedules and resources. You need to get the people and communications aspects right too, but to fail you only need to keep the requirements vague.

The Beetlecam and Innovation

The Beetlecam is a great example of an innovative project. Two brothers from the UK, Will and Matt Burrard-Lucas wanted a better way to photograph wildlife in Tanzania and came up with the concept of putting a camera on a mobile buggy. They got some great photos from it, which you can see here on their site.

Work vs. Activity

“Habits are at first cobwebs, then cables.” Spanish Proverb

How much does you team do that produces real results? How much is done through habit?

Reporting and analysis is one example, I’ve seen many beautiful reports, but the key question is what action will you take based on the information in this report? If the answer is “I don’t know” then the report is probably a waste of time.

Evaluate what you’re spending time on, what was initially a great idea may now be a waste of time. Other candidates for cutting beyond potentially irrelevant reports include recurring meetings, checking your email every 15 minutes and anything else which consumes your time, but doesn’t lead to results. Evaluating how you spend your time can be an enormously valuable activity.

Rescue Time is an interesting application in this area, it automatically tracks the time you spend on your computer, which of course is only part of your day, but the results can be insightful.

The Failure Of The FBI’s Virtual Case File Project

Between 2001 and 2005, the FBI’s Virtual Case File project failed. The Virtual Case File project was part of a larger initiative called Trilogy. Costs overran by 89% or just over $200M. A project that should have taken 3 years, failed after 4 years with requirements still not met. In some ways this project is similar to the London Stock Exchange’s Taurus Project, where scope creep lead to the project (at least as initially envisioned) being cancelled after massive cost overruns.  This analysis draws on a very detailed report from the Office of the Inspector General, but here we focus explicitly on the project management failings.

The FBI’s Trilogy project contained three parts:

  1. Upgrading software and hardware for FBI agents
  2. Upgrading the FBI’s communications network
  3. Significantly upgrading the FBI’s case management system (Virtual Case File) to enable better access to, and sharing of, case-related information across the FBI

The first two elements of the project were completed, not perfectly or all that impressively, but roughly as planned. However, the Virtual Case File project experienced major cost and schedule overruns and never achieved its objectives. It is widely used as an example of a failed IT project.

Vague and inappropriate requirements

As with all the cases of project failure that this blog has explored (the Sydney Opera House, the 2010 Census, the Channel Tunnel, Denver Airport and the H1N1 vaccination program) the heart of the problem is vague requirements. “Trilogy’s design requirements were ill-defined and evolving as the project progressed.”

Scope creep

It’s perhaps inappropriate to use the term scope creep for a project where scope is never clearly defined. But the political climate created by the September 11 attacks and reports into the Oklahoma City bombings in March 2002 increased pressure on the project to produce results faster (regardless of what was feasible). In addition, the Hanssen espionage case, one of the most significant in the FBI’s history as portrayed in the movie Breach, raised the bar for security requirements for the Virtual Case File system. According to one report on the project. “Trilogy’s scope grew by about 80 percent since initiation of the project.” Of course, if scope is increasing faster than work is being complete, it will be impossible for any project to finish.

Scheduling by desired outcomes, not resource based estimates

In my book, I describe this as this as the King Canute approach to project management, just because you’re a powerful executive and you want something to be complete faster than 3 years, does not mean that that is possible given the scope of work and resources allocated. However, the scheduling for the Virtual Case File project focused on what was desired, not what was possible. Two quotes from the report illustrate this:

“The FBI also developed plans to accelerate completion of Trilogy because at the time the project’s 3-year modernization timeframe was considered too long.”

“The contractor disagreed with the resulting schedules, because the schedules showed that the full implementation of the project exceeded the proposed completion date of the project.”

It is easy to see these errors with the benefit of hindsight, but in the pressure of the moment it is all to do easy to massage the timeline to make it look as if you might hit the desired outcome, on paper at least.

Cost plus contracting

The interests of the FBI and the contractors working on the project were not aligned.“The cost-plus-award-fee type of contract used for Trilogy…did not require specific completion  milestones…and..did not provide for penalties if the milestones were not met.” Arguably, this is a result of vague scope, but in addition to vague language the incentives created by cost plus pricing, made it hard for the FBI to hold their contractors accountable.

Lack of clear ownership

Tellingly, a project manager was appointed only late into the project, the FBI cycled through 5 people in the CIO role in 4 years and accountability appears generally absent due to “decision making by committees instead of knowledgeable individuals.” In addition to a lack of accountability at the contractor level, there was a need for clearer accountability within the FBI. Lack of clear accountability structures is a clear theme in failed projects as the Eurofighter project shows.

See more analysis of project failure here on this blog and at