Deduplication and pension schemes

Deduplication is an essential part of data preparation for statistical modelling. The phenomenon of multiple policies per person is a major issue for annuity portfolios, and arises from life companies' policy-orientated view of the world. This makes perfect sense for insurers, of course, since their legal liability is the policy.

My expectation was that it would be less of an issue for pension schemes, whom I thought would naturally have a more person-orientated view of their liabilities. However, I recently analysed the mortality of a UK pension scheme with over 38,000 benefit records, of which over 1,300 were clear duplicates. I didn't reckon on the frequency with which people can return to a former employer, thus accruing two separate periods of pensionable service. Many people also meet their spouse at work, which results in having one record for their main pension benefit and another for a surviving spouse's benefit after the death of their partner.

It looks like deduplication is an essential part of data preparation for pension schemes as well.