The role of assessment in security is to provide some evidence that the reality of the implementation meets the security goals. You can't just rely on paper documents and standards or even reading source code alone. As the spooks like to say - "we don't break standards, we break implementations." Confirmation bias and perverse incentives are two of the biggest enemies for assessments, because they direct the assessment's bias in the wrong direction - away from reality and towards wishful thinking.
Last year, I blogged about restaurant inspections in a city where only one restaurant had been closed down in 30 years for health code violations. The article said that a "roach infested restaurant was in danger of being closed down." I think the danger was not that the restaurant would be closed but rather that someone might eat there and get a roach burger.
This kind of thinking isn't limited to restaurant inspection. The events in Europe's financial crisis are truly unprecedentd and the mix of the banks, governments, and eu governing body provides a breadth and depth that I don't think we've ever seen. Like the US crisis, the eu authorities sought to shore up confidence in the banks by performing a stress test on banks. Spoiler alert - most, including all banks of size, passed the test which concluded July 15.
The stress tests measured among other things, the Tier 1 capital ratios, and the eu wanted to see north of 10% Tier 1 capital for the banks to be deemed healthy enough to weather the inevitable storms.
86 days after receiving a clean bill of health from the eu, the French-Belgian bank Dexia collapsed. Where did the failed bank rank in the overall assessment measuring the safety of the bank's assets? Number one! They had the best Tier 1 capital ratios!
How was this number one ranking achieved, when they were less than three months from meeting their end? Well, the Tier 1 capital ratio specifically excluded things, including exposure to Greek debt!
For assessing the safety of assets, they assessed the safety of the safest assets and ignored the risky ones. Now that number one in "safety" is toast, how does that make you feel about the safety of others who "passed the test"?
An assessment is supposed to go up to the dart board and check to see if you got a bulls eye or how close you got. Having people throw darts and then going up to the board and drawing a bullseye around where the dart lands isn't helpful.
This kind of assessment is worse than useless, its harmful, its like giving people umbrellas and taking them back when it rains. being insecure is not the biggest problem, you can be insecure, know you are insecure and act accordingly. As Brian Snow said, the most dangerous stance is to assume you are secure when in fact you are not secure.
Your blog is very nice to read because you included a lot of information in it.Thanks for sharing.
Posted by: phone slots | October 24, 2011 at 06:12 AM
The problem with the stress tests was they were politicaly motivated, politicaly organised, and thus had exemptions for politicaly inspired debt.
The UK satirical magazine Private Eye was pointing out the failings in the EU State "Sovereign Debt" before even the US banking crisis, so nobody can claim it was "unknown".
In the UK we have come to the realisation that a significant number of our politicians are crooks, and have recently jailed some of them for blatant fraud.
However what is worse is the complicitness of the likes of major banks etc with other types of fairly blatant fraud (revolving door employment of politicos and civil servants who have been fundemental in aproving contracts to the companies).
Whilst "sovereign debt" is bad there is yet another state sponsored fiscal calamity waiting in the wings and this is PFI/PPI. In the UK it is already puting organisations that had it foisted on them by the politicos into chronic debt and bankruptcy after just a couple of years, and some of the contracts have 40year life times.
Again PFI/PPI like the Sovereign Debt is politicaly inspired with the sole purpose of making a government look financialy responsable by putting debt "off book" whilst in reality they are "fiddling whilst Rome burns".
And in all cases of such political motivated fraud it is the ordinary person who pays not just once but many times over.
Oh and one thing that appears to have escaped the UK politicos attention, the economy is based on people spending money, the world economy on countries spending. If all countries stop spending money then all their economies will collapse. So whilst "trimming the fat" is fiscaly sensible "starving to death" is not. As with food however it is the quality not the quantity that is important, in the US they have tried various stimulass packages but according to many they have failed because it was quantity befor quality.
Posted by: Clive Robinson | October 26, 2011 at 08:01 AM