Dan Geer posted a digest of MetriCon 1.0. It is a great read, almost as good as attending. The slides are here. I blogged some additional thoughts here [1, 2, 3]. As the track chair for Software Security metrics, I had to follow Steve Bellovin saying that we do not have system security metrics that are usable and the security metrics are infeasible. My point was that "system" is way to all encompassing for where we are at right now and we need to begin by decomposing things into smaller pieces. A number of the presentation in the software security metrics did just that with attack pattern metrics at a channel, method, and data level, and pattern-based metrics, for example.
The second point is that the word "security" is particularly harmful in the metrics space, at least for where we are at right now as an industry. We need to be more granular and focus on measuring what we can today. Let's say that security means the union of confidentiality, integrity and availability. So if you are like most enterprises, then you can't do *all* of them today (much less join them all together in some meaningful way), but does that mean you shouldn't measure what you can? Well, every enterprise I see has at least some availability metrics. Granted confidentiality and integrity may be much harder, but at least there are some starting points, for example in authN, authZ, and identity and access management systems are rich sources for potential metrics.
The conference is 1.0 for a reason, it is where the industry is right now, but I never saw any software product that was perfect in version 1.0 and we shouldn't expect security metrics to be perfect right otu of the chute either.
Comments