Richard Bejtlich has a post talking about the risks of when models and reality diverge -
Did we not just suffer a global recession exacerbated by clowns who thought they could model risk "with a high degree of accuracy"?This of course is the financial crisis in a nutshell, we've had plenty of real estate crisis in the past, but they never lead to 10% unemployment and cratering the whole financial system. This was the first time that the geniuses had built mortgage backed security models on top of real estate (key underlying assumption - house prices never go down in the US), add in some false precision and perverse incentives, stir - and voila.
Its not to say models are bad, just when they're confused with reality. We don't need to look any further than the world of college sports. College football's championship is based solely on a model where you calculate strength of schedules, margins of victory, strength of opponents, and so on, but what you get is an unholy mess that leads to someone getting hosed every year, and endless legitimate arguments. Its all model. (Plus a zillion lame bowl games - the Swiffer Sweeper bowl?)
College basketball otoh starts with a model - picking top 64 teams (pretty large sample set, probably fair to say if you are not in the top 64 then you don't deserve to be national champ), but then the model is stress tested and determines the champ by who actually wins the game (note - Kansas looks good this year). Many times (most?) the winner is not the #1 ranked team going in, is this bad? Not at all, the model gives a logical starting point but its not an end in and of itself.