What could be a more effective risk management practice than wearing a seat belt when driving? Surely that sort of improvement benefits everyone? Actually, economists have shown that it’s not that simple and there are lessons for project managers from this lesson too.
If you engage in risk management, as all project managers do, then be aware of the work of the Chicago economist Sam Peltzman.
What Peltzman identified is that behavior can change substantially in response to risk management policies. For example in the case of seat belts, people tend to drive faster and have more collisions when they use seat belts because of the sense of security it gives them. This is not to say that seat belts are ineffective, it seems that traffic safety overall (especially in terms of road deaths) has improved as a result of them, but not as much as you would have expected, because of this behavioral offset.
And more omniously, though drivers are safer from seat belts, cyclists and pedestrians are more at risk with seat belts because drivers tend to drive faster and those outside the car don’t have any offsetting protection.
Similar studies have shown the same phenomena in NASCAR racing, when safety improves drivers then notch up the risks they are willing to take, offsetting some of the benefit.
photo credit: Roger Barker
So the question for the project manager is are your risk management policies changing behavior? This is not to say you shouldn’t bother with risk management, just as with seat belts the overall benefit is likely to be positive, but pay attention to how behavior is changing as a result of the changes you implement. Are people less cautious knowing there is a more robust monitoring process in place for their projects? Are team members less focused on escalating problems knowing that someone else is watching out for them?
The lessons from other areas such these unintended consequences may be more important than you may initially suspect.
Interesting talk from Noreena Hertz on the dangers of trusting experts, and the implications for how our brains work and decision making. When I watch the video I think of the estimation processes for projects and how they are likely subject to the same errors and biases.
The current Zeitgeist in a lot of business writing seems essentially to favor agile thinking. Peter Sims recent book Little Bets and Tim Harford’s book Adapt both go in this direction.
Both authors follow largely similar logic:
- The future is hard (sometimes impossible) to predict
- The best way to test something is to try it
- The more you test the more likely you’ll hit on something good
The result is a rejection of top-down planning and an endorsement of decentralized creative processes, with clear success metrics in place to pick winners and a framework to make sure you don’t lose so much in the iterative testing that the gains of a win are erased.
This is all good, the logic can’t be faulted and the process is appropriate in many situations. But crucially not all situations and that’s where I take issue with both books.
If things are as complex as they both argue they are, then isn’t a one-size-fits all answer a little too easy? The examples too, are very similar between the books. For example, the Iraqi War and defeating the insurgents shows that troops on the ground pioneered smart ideas that worked in building local trust and were then used broadly across the military. The Soviet Union showed top down planning was a non-starter, and Pixar demonstrates iteration from demo shorts of jumping lamps, to computer generated adverts to ultimately the pioneering success of Toy Story.
That’s all great, but what about the fact that the initial ‘shock and awe’ invasion of Iraq (if politically misguided) was an overnight triumph of top down planning? The Soviet Union failed, but China thus far could teach many other economies some tricks with a pretty high degree of command and control. Thirdly, whilst Pixar showed the success of incremental steps. Titanic and Avatar both show that James Cameron’s big bangs can be more successful than even Toy Story 3
Finally, didn’t Karl Popper make these very persuasive arguments about iteration and testing back in 1962
? Though, I admit his text is a little harder to read.
It’s great to see the tidal wave of support for agile processes, but let’s not forget that planning is needed in many circumstances. Indeed as Napoleon said “In preparing for battle, I have always found that plans are useless, but planning is indispensible.”
In summary, both Little Bets and Adapt are well written books, but let’s not swing the pendulum over to agile processes too far and be careful about balance in the examples we pick for support.
Exploring Requirements by Donald C. Gause and Gerald M. Weinberg is a very good basic text on the requirements process. It’s been available for some time, but only came out in paperback earlier this year. Although the ideas in the book are simple they are also very powerful and are the sort of things that seem obvious once you read them, but are not practiced as widely as they should be. The book shares ideas in an engaging, friendly tone, making use of some pretty frivolous examples that make it a more interesting read than your average project management book.
Much of the book is focused on the insight that the requirements process can be greatly improved by clearer communication and the removal of all ambiguity, and the authors use some clever examples that illustrate their points, and offer clear steps for communicating better and removing the typical barriers to collecting requirements. As with many books related to project management the book leans more towards software projects than other types of project, but this doesn’t harm the broad applicability of most concepts. The book is strongest on requirements gathering, but also has some tips for other areas like meeting management or brainstorming. If you’re feel a bit rusty on your requirements gathering skills this text is a quick way to sharpen your technique and broaden your perspective.
Nice chart based on Barry Boehm’s book on Software Economics, shows us the value of fixing problems early in projects, the costs of fixing problems at least doubles at each stage of the process, and become even larger when the results are in operation. The data is based on software projects. This is intuitive, but it’s worth remembering just how much the costs can rise.
It’s important, but hard, to separate good process from good outcomes. Often it’s assumed that any good outcome, must reflect a good process and vice versa. But in risky situations this approach could lead you to make major mistakes. Assume you win the lottery. Now, you now have a very large amount of money, but that does not change the fact that lotteries are, as Adam Smith said, “a tax on idiots” and the expected return on any lottery ticket is negative – there are much better ways to spend your money. So just because people win the lottery each week does not mean playing lottery is a good idea (unless you like losing money). So playing the lottery is a bad process, but there’s a chance you hit a good outcome. In fact, it happens every week.
I’m just using the lottery as an example to show that in many cases closer to home, we might be making the same mistake. For example, your project finished ahead of schedule, but how much of that is due to good process that can be repeated? And how much is due to luck? The answer comes down to how good your process is.
Of course, there’s a more positive side to this too, just because you didn’t get the outcome you wanted didn’t mean the process was bad. Speed skating at the Winter Olympics is a good example of this, it takes many years of dedicated training to enter the Olympics, but in a speed skating race you can easily get pushed over by a competitor, and it might be totally out of your control. It doesn’t mean you shouldn’t have won gold, but it means you didn’t win goal. Good process, bad outcome.
So what to do in situations where risk means that outcome and process aren’t totally tied together?
Two things can help:
- Repetition – over time processes and outcomes will converge where risk is present. You might get lucky on one project, but across ten it’s far less likely. Look for multiple instances of a situation before forming a judgment.
- Analysis – good process can be supported by analysis. If something went wrong or poorly look at why it happened. Luck can often be identified with logical analysis – a good process should make sense and be robust.
||Changes could make things worse
||Process improvement needed
This detailed survey of Israeli project managers, looks at the tools/techniques being used to manage risk from a list of 38 tools/techniques. It goes into a lot of statistical detail, but provides a interesting view on the tools and techniques used to manage risk by project managers:
Most Popular Risk Management Techniques:
- Responsibility assignment
- Risk impact assessment
Least popular Risk Management Techniques:
- Graphic presentation of risk information
- Procedure for closing risks
Most likely to be used by those with good project and/or risk management practices (note the definition of “good” was self-assessed) :
- Risk impact assessment
- Risk classification
- Ranking of risks
Given the recent publication of the Checklist Manifesto, it’s interesting that checklists aren’t broadly used. Generally, it seems the different between basic and more sophisticated managers is that good project managers don’t just count risks, they classify them and assess their relative impact.
See the full article here, it’s almost 10 years old, but the techniques described haven’t fundamentally changed.
photo: Ian Britton (via Flickr)
The previous 3 posts on risk management have focused on understanding risk. The next step, and the primary purpose of risk management is mitigation.
For each risk that is identified what would you do differently if it occurs?
Also, perhaps more important is what is the agreed definition of occurrence of the risk? Disagreement within a group over whether/when a risk factor has occurred can delay the response to it.
Finally, risks are unlikely to ever play out exactly as you anticipate, but the act of considering risks and their mitigation strategies will make your plans more robust. You may also want to consider Gary Klein’s pre-mortem approach.
“Plans are worthless, but planning is everything.”Dwight Eisenhower
photo: Paul Belson (via Flickr)
One of the reasons risk management goes wrong is a psychological phenomenon called anchoring. Anchoring means that once an idea is out there (such as estimate of when a project will end) it has a lot of power and influence, even if it’s not intended to. Lots of academic research documents this across various settings and context and it’s pretty robust. Once an ‘anchor’ is set, even unintentionally, it is notoriously hard to move it.
What does this mean for risk management? Well it means that once a project plan is out there, the initial estimate will be changed more slowly that it should because of these psychological factors. If the project is was initially meant to cost $3M and then a revised estimate comes in at $4M, the estimate is more likely to move to $3.5M than all the way to $4M, because of the impact of the initial estimate, even though it’s now out of date.
Good risk management should ensure that only the latest information is factored into any estimate, effectively starting from a ‘blank slate’ each time. However, even though this sounds obvious, the impact of anchoring does mean that it’s hard to do in practice.
Risk involves some form of estimation, as Flyvberg has shown, the best estimates are based on past, factual reference data, but for most projects that’s hard to obtain and so project managers are left to rely on their estimation skills. There’s an exercise on estimation here that I think everyone should do at least once. Most people are over confident in many areas.
Do you think you are a better driver than average? 93% of people put themselves in that category (of course it should be 50%). The same is true of estimation, and it’s an area that’s key to risk management, the temptation is to be excessively precise in one’s estimates, and this level of false precision understates true risk.
source: Joe Futrelle