Why mean outcomes are often meaningless

Excellent piece by John Kay, Why mean outcomes are often meaningless:

You have spent £2 on a lottery ticket. On Saturday evening you may be a millionaire. Or, more likely, not.

But meantime, the auditors arrive. They must confirm that your accounts show a true and fair view. An old-fashioned auditor might allow you to record the lottery ticket at its historic cost of £2. A modern one would want to assess its fair value.

But there is no market in second-hand lottery tickets. The auditor might allow you to treat it as a “level two” asset which can be valued by reference to the price of other traded items and use a discount to the primary market price. Or the accountant might encourage you to “mark to model”: multiply the payouts by their probabilities and compute an expected value, £1.20 say, though good models attach different values to different tickets because some numbers are more popular than others.

But no one buys a lottery ticket to trade it. The purchasers seek the thrill of winning, but expect the disappointment of losing. You cannot sensibly value an asset at £2, or £1.20, when you can be certain that on Sunday morning its value will be completely different.

  

As the infosec industry matures, there's an increasing focus on analyzing the probability of events. In fact, what matters more in risk analysis is the impact.  As John Kay says "Quantum physicists have long struggled with the problem of Schrödinger’s cat, at once dead and alive, and accountants have not been more successful in resolving the paradox."

The paradox we face in infosec is similar. There are assets valuable to attackers that are not tradeable by their owners. Then there is the liquidity of the asset. Then there is the ability to combine the asset with others to make a combined useful asset for the attacker. And so a valuable addition in risk analysis is to lay out range of these possible outcomes for impact analysis.