The Failure Of The FBI’s Virtual Case File Project

Between 2001 and 2005, the FBI’s Virtual Case File project failed. The Virtual Case File project was part of a larger initiative called Trilogy. Costs overran by 89% or just over $200M. A project that should have taken 3 years, failed after 4 years with requirements still not met. In some ways this project is similar to the London Stock Exchange’s Taurus Project, where scope creep lead to the project (at least as initially envisioned) being cancelled after massive cost overruns.  This analysis draws on a very detailed report from the Office of the Inspector General, but here we focus explicitly on the project management failings.

The FBI’s Trilogy project contained three parts:

1. Upgrading software and hardware for FBI agents
2. Upgrading the FBI’s communications network
3. Significantly upgrading the FBI’s case management system (Virtual Case File) to enable better access to, and sharing of, case-related information across the FBI

The first two elements of the project were completed, not perfectly or all that impressively, but roughly as planned. However, the Virtual Case File project experienced major cost and schedule overruns and never achieved its objectives. It is widely used as an example of a failed IT project.

Vague and inappropriate requirements

As with all the cases of project failure that this blog has explored (the Sydney Opera House, the 2010 Census, the Channel Tunnel, Denver Airport and the H1N1 vaccination program) the heart of the problem is vague requirements. “Trilogy’s design requirements were ill-defined and evolving as the project progressed.”

Scope creep

It’s perhaps inappropriate to use the term scope creep for a project where scope is never clearly defined. But the political climate created by the September 11 attacks and reports into the Oklahoma City bombings in March 2002 increased pressure on the project to produce results faster (regardless of what was feasible). In addition, the Hanssen espionage case, one of the most significant in the FBI’s history as portrayed in the movie Breach, raised the bar for security requirements for the Virtual Case File system. According to one report on the project. “Trilogy’s scope grew by about 80 percent since initiation of the project.” Of course, if scope is increasing faster than work is being complete, it will be impossible for any project to finish.

Scheduling by desired outcomes, not resource based estimates

In my book, I describe this as this as the King Canute approach to project management, just because you’re a powerful executive and you want something to be complete faster than 3 years, does not mean that that is possible given the scope of work and resources allocated. However, the scheduling for the Virtual Case File project focused on what was desired, not what was possible. Two quotes from the report illustrate this:

“The FBI also developed plans to accelerate completion of Trilogy because at the time the project’s 3-year modernization timeframe was considered too long.”

“The contractor disagreed with the resulting schedules, because the schedules showed that the full implementation of the project exceeded the proposed completion date of the project.”

It is easy to see these errors with the benefit of hindsight, but in the pressure of the moment it is all to do easy to massage the timeline to make it look as if you might hit the desired outcome, on paper at least.

Cost plus contracting

The interests of the FBI and the contractors working on the project were not aligned.“The cost-plus-award-fee type of contract used for Trilogy…did not require specific completion  milestones…and..did not provide for penalties if the milestones were not met.” Arguably, this is a result of vague scope, but in addition to vague language the incentives created by cost plus pricing, made it hard for the FBI to hold their contractors accountable.

Lack of clear ownership

Tellingly, a project manager was appointed only late into the project, the FBI cycled through 5 people in the CIO role in 4 years and accountability appears generally absent due to “decision making by committees instead of knowledgeable individuals.” In addition to a lack of accountability at the contractor level, there was a need for clearer accountability within the FBI. Lack of clear accountability structures is a clear theme in failed projects as the Eurofighter project shows.

See more analysis of project failure here on this blog and at www.projectcasestudies.com.

Swine Flu Vaccines and Project Management

Earlier this year, the CDC committed to providing H1N1 vaccines on a broad scale as Time documented. However, the Wall Street Journal reported today, swine flu vaccine production has fallen far short of expectations. This is at a time when cases of swine flu are rising sharply (see graph below). It appears much of this shortfall will be recovered in the next several months. Still a shortfall remains, this post looks at why that’s happened from a project management perspective and what we can learn from it.

The government forecast 30 million doses of swine flu vaccine would be ready by the end of October 2009. With a week to go we are at 16 million, so 46% below target, and even that target was revised down last week weeks ago for 40 million doses, so really we are 60% below initial target, of course some of that deficit will be made back in the next 7 days, but not all of it.

This appears to be a classic example of project failing to deliver on time as expected, even in this case where resources are relatively abundent. That’s not surprising, a project failing to meet expectations is the most common outcome (see related post on project failure). The more interesting question is why, because over time our project management skills do not seem to be improving

The reasons for the delay are as follows:

• Inaccurate estimates – yield lower than expected from seed virus at at least 2 producers
• Regulatory risk – FDA approval for Glaxo SmithKlein’s vaccine taking longer than expected
• Negative events not captured in plan – Some production lines experience ‘brief interruptions’ as larger scale production ramps up
• Scope change – Government changed request to relatively more single dose syringes vs. multi-dose vials

Now of these the only one which is ‘legitimate’ in that it’s not under the producer’s control or ability to forecast, was the scope change from the government to single dose syringes, because that changed the scope of project. All the others events are forecastable. Of course, no estimate is perfect and things will deviate from it. But, crucially, if there are these reasons for things to be behind schedule, shouldn’t be there a similar number of factors driving things to be ahead of schedule? Basically, shouldn’t the ‘good luck’ and the ‘bad luck’ cancel out if the estimates used in the first place were fair guesses? 

Apparently, not

• Medimmune could produce more vaccine (delivered via nasal spray), but are limited by the number of sprayers they have. So they remain on target, rather than ahead of it.

Net – the majorty of firms are behind and none are ahead. It appears there was too much optimism baked into estimates. A classic problem when formulating project plans.

 Image source (Center for Disease Control Site)

Also, if the chart above worries you, note there’s a good chance, the swine flu infection numbers are somewhat overestimated based on historical data.

If these sort of project post-mortems interest you, see this related post on the delay and cost overruns of the Sydney Opera House.