Monthly Archives: October 2010

Better Forecasting

Projects fail often, most studies find failure rates above 30% depending on the exact definition of failure, and since budget overrun often cause failure, it seems obvious that better cost forecasts would reduce project failure. Bent Flyvbjerg demonstrates a sound method for improving cost forecasts here in the 2006 Project Management Journal, it’s a fairly long and academic article, so I’ll provide a brief summary.

The approach Flyvbjerg suggests is called reference class forecasting. It’s a simple and elegant solution to the problem of overconfidence in forecasting. In essence, for any project, you estimate how much it would cost using normal methods, of course this requires putting in all the effort and process you would normally invest to develop a sound cost forecast. You then find a set of comparable projects (a “reference class”), with enough projects in the group to be statistically significant, but small enough that the projects are similar to the one that your undertaking. In practise, getting such data is relatively hard unless your organization conducts rigorous post-mortems consistently, but the article cites some distributions for certain project classes. For example rail projects exceed budget by 40% on average. You then gross up your cost estimate by this number and your estimate will be much more reliable.

Reference class forecasting may seem fatalistic or too simple, but it is far more reliable than existing methods. Just as most car owners believe they are better drivers than average, so project managers have excessive confidence in their own estimates, as the literature on project failure rates shows. Flyvbjerg describes this approach as taking the “outside view” looking across projects, rather than dwelling on the “inside view”, the details of the specific project. It is more useful to compare a project with a broad set of similar projects than obsess on the details of your own project.

Of course, there are some caveats to this view, if you believe overruns result from poor forecasting, then this is an effective solution, but if overruns stem from low-balling costs in order to get a project up and running, then the reference class forecasting approach won’t solve that problem.