Project Management at the BBC

Continuing the theme of UK based projects such as Wembley and Scottish Parliament, the BBC is a state funded provider of television and radio in the UK. The UK’s National Audit Office reviewed 3 recent construction projects by the BBC, 2 of which exceed cost and budget and 1 came in below budget the full report is here.

The key findings are as follows:

• The projects did not identify benefits clearly from the outset, making it hard to assess success objectively
• The importance of financial contingencies – the reason the one project (Salford Quays) came under budget was because it had financial contingencies at just over 10% of the initial budget. The projects which went over budget had much smaller contingencies.
• Change management, all projects had material changes during their lifetime, an average of 41 per project, which generally lead increased costs and in some cases could have been avoided through more thorough planning efforts.
• There is a need to better assess skills required to execute the project relative to the organization’s existing capabilities.

A New Model for Risk/Return on Projects

In finance there is a concept called the Sharpe Ratio, which project management could learn from.

Basically a Sharpe Ratio captures return achieved in exchange for risk.

So it’s not just about how risky (delayed) a project but what what the true return is relative to that. This is not Return on Investment but Return on Risk.

Project management can’t achieve this level of accuracy because both return and risk are much harder to measure for a one off project than for an investment. For example, shares in Ford trade on the markets on a daily basis making it very easy to do complex statistical analysis. Project management is getting better in terms of Monte Carlo analysis to model schedule risk. But at a fundamental level the data project managers have to work with just is not as rich as in finance, because each project is unique.

The point is this, as I look at project failures, the most horrendously delayed projects are also often the ones that win major awards.

For example, would you rather have this on time and budget?

photo rytc (via Flickr)

Or these with delays and cost overruns, (but lots of awards)?

photo: Ian Kroll (via Flickr)

source: Anita363 (via Flickr)

A Holistic Look At The Triple Constraint

I’ve previously argued that the triple constraint is more a myth than a useful project management tool.

Roger Atkinson’s academic paper takes a different approach, the argument is the triple constraint (referred to in the paper as the Iron Triangle) is only a subset of project management criteria, and a better approach is to combine 3 additional sets of metrics to capture project management success holistically, one of the metrics Atkinson proposes is IT specific, but the others capture stakeholder benefits and organizational benefits.

It makes sense, time and budget can be the easiest things to measure but aren’t necessarily the most important. See the full article here.

Better Estimation – Looking Back To Look Forward

It should not be surprising that the best way to estimate how long something will take, is to understand how long it took last time. In many cases this data is not hard to find, even though the project might be new, many of the tasks within it are not.

So how do you do your estimates? Do you start with a blank page, or look back at your last project?

Project Failure – Scottish Parliament

source: Asif Musthafa (via Flickr)

The Scottish Parliament overran initial costs by a factor of ten and was delayed by 3 years.  It is clear that similar to the Sydney Opera House the design was not finalized before construction and estimates were not backed by a credible cost estimation process. For example, here are the 5 finalists chosen from a shortlist of 12 in May 1998. Only one of these finalists adheres to the budget, every other proposal fails to meet the brief in terms of both cost and size. It is absurd to specific minimal criteria for a brief and then fail to shortlist finalists based on those criteria. When a project starts with this level of disregard for process, it is unlikely to ever get back on course.

Source: House of Commons Briefing (for reference EMBT/RMJM was selected)

From there, the project continues to unravel, for example the architect then added 4,000 square meters (+14%) to his design area. It is perhaps not surprising that the project manager resigned just over a year into the project because there was clear tension between the sponsors’ desire to have the Parliament ready as soon as possible and the architects’ desire for a “gestation period” to really flesh out his design together with a need to be engaged in all decision making. The lead project sponsor (Donald Dewar) and architect (Enric Miralles) sadly both died during the construction period, which further complicated the project.

Detailed reports on the project can be found in the Hollyrood Enquiry and Parliamentary Briefing and this article from Max Wideman.

Quality

It is worth noting that the Scottish Parliament has won many national and international architectural awards. This is similar to many delayed projects in that they fail specularly on time and budget constraints, but the quality of what is delivered can ultimately be extremely high, even if the apparent process in creating it was not.

Conclusions

The process of selecting and managing the new Scottish Parliament had no regard for credible estimates from the outset. As a result it is unsurprising that the final result bore no relation to the initial estimates. It also appears that the sponsors of the project wanted quality above all, and in that context it is not surprising that cost and time gave way to that objective.

Johnson on How Innovation Happens

Interesting video from Steven Johnson on innovation (one of the TED talks)

Specifically, within the video it’s worth looking at the example of the baby incubator and the Sputnik to GPS innovations within the video (the links should take you to the right places within the video). Or if you want to watch the full video it’s below:

Interesting Use of GPS Sharing

Glympse is a particularly interesting app, of the hundreds of thousands of apps out there on smartphone app stores.

It enables you to share your location with a specific person for a specific period of time via emailing them a web link. Unlike other location sharing apps, this level of targeting makes it much more useful to scenarios such as meeting up or letting someone know when you’ll arrive.

It’s an interesting innovation, because Glympse aren’t offering anything radically new, just taking the technology that’s already out there and simplifying it so it becomes more useful.

Which Are The Most Popular Risk Management Techniques?

This detailed survey of Israeli project managers, looks at the tools/techniques being used to manage risk from a list of 38 tools/techniques. It goes into a lot of statistical detail, but provides a interesting view on the tools and techniques used to manage risk by project managers:

Most Popular Risk Management Techniques:

1. Simulation
2. Responsibility assignment
3. Risk impact assessment

Least popular Risk Management Techniques:

1. Graphic presentation of risk information
2. Procedure for closing risks
3. Checklists

Most likely to be used by those with good project and/or risk management practices (note the definition of “good” was self-assessed) :

1. Risk impact assessment
2. Risk classification
3. Ranking of risks

Given the recent publication of the Checklist Manifesto, it’s interesting that checklists aren’t broadly used. Generally, it seems the different between basic and more sophisticated managers is that good project managers don’t just count risks, they classify them and assess their relative impact.

See the full article here, it’s almost 10 years old, but the techniques described haven’t fundamentally changed.

Managing Risk – Part 4

photo: Ian Britton (via Flickr)

The previous 3 posts on risk management have focused on understanding risk. The next step, and the primary purpose of risk management is mitigation.

For each risk that is identified what would you do differently if it occurs?

Also, perhaps more important is what is the agreed definition of occurrence of the risk? Disagreement within a group over whether/when a risk factor has occurred can delay the response to it.

Finally, risks are unlikely to ever play out exactly as you anticipate, but the act of considering risks and their mitigation strategies will make your plans more robust. You may also want to consider Gary Klein’s pre-mortem approach.

“Plans are worthless, but planning is everything.”Dwight Eisenhower

Managing Risk – Part 3

photo: Paul Belson (via Flickr)

One of the reasons risk management goes wrong is a psychological phenomenon called anchoring. Anchoring means that once an idea is out there (such as estimate of when a project will end) it has a lot of power and influence, even if it’s not intended to.  Lots of academic research documents this across various settings and context and it’s pretty robust. Once an ‘anchor’ is set, even unintentionally, it is notoriously hard to move it.

What does this mean for risk management? Well it means that once a project plan is out there, the initial estimate will be changed more slowly that it should because of these psychological factors. If the project is was initially meant to cost $3M and then a revised estimate comes in at $4M, the estimate is more likely to move to $3.5M than all the way to $4M, because of the impact of the initial estimate, even though it’s now out of date.

Good risk management should ensure that only the latest information is factored into any estimate, effectively starting from a ‘blank slate’ each time. However, even though this sounds obvious, the impact of anchoring does mean that it’s hard to do in practice.