Software projects - the sting in the tail

In an earlier blog I looked at the arguments in favour of buying in specialist software, rather than trying to build it yourself.  Of course, as someone whose business is providing software services for mortality and longevity work, I am somewhat partisan.  To balance things out, I wrote a follow-up blog on when it makes sense - even for us - to source external software components.

Let's assume that you have read both blogs and that you still want to build your own software.  You draft a project plan with cost estimates, add a contingency margin of 10-15% and submit it to management for approval.  The only thing standing between you and a shiny new system is the expected time to build it, right?

Not quite.  You will no doubt be aware that IT projects don't always run according to plan (the Horizon project at the UK Post Office is a study in horror and injustice).  Indeed, if you are of a certain age, you are probably aware of quite a few IT projects that have had major issues.  You might even feel that there is something of a pattern here, and that someone should study it.

As it happens, someone has: a Danish academic named Bent Flyvbjerg.  Together with long-term collaborators he has built a database of projects of various types, including IT projects.  And once you collect enough samples, you can start to see patterns.  It turns out that IT projects are the fifth worst in Flyvbjerg's database, with a mean cost overrun of 73% (excluding inflation).  For comparison, the mean cost overrun of building an airport is 39% (Source: Flyvbjerg & Gardner, 2023, Appendix A).  And cost overruns go hand-in-hand with time overruns, so such IT projects are delayed as well as more expensive.

Of course, that is just the mean cost overrun; there will be variability.  Actuaries in particular will relate to Flyvbjerg's focus on project tail risk.  If we define the tail as having a cost overrun of more than 50%, then 18% of IT projects land here (Source: Flyvbjerg & Gardner, 2023, Appendix A).  Finally, there is the question of the mean overrun in the tail, i.e. the conditional tail expectation (CTE).  For IT projects it is a staggering 447%, worse even than the building of nuclear storage at 427% (Source: Flyvbjerg & Gardner, 2023, Appendix A).  If your IT project lands in the tail, things could get very ugly indeed.

Flyvbjerg & Gardner (2023) make recommendations for minimising project tail risk.  I'm not going to repeat them all here, as you should buy and read their book.  In fact you should persuade your company to buy several copies, as it will - quite literally - pay for itself hundreds of times over.  However, one statement resonated with me:

What can you do about the tail?  Cut it off.

Flyvbjerg & Gardner (2023, p118)

Flyvbjerg and Gardner give examples where organizations paid money to successfully eliminate tail risk, describing it as a kind of insurance.  (Actuaries will recognise this as a variant of stop-loss insurance.)  However, paying money to cap the risk in building software can go further still:

The most radical possible solution for constructing software is not to construct it at all.

Brooks (1995, p197)

The best way to manage tail risk in IT projects is to buy in software that is already built and tested.