Qualitative and Quantitative Metrics in Iraq

Training a self sufficient Iraqi army and police is listed by pretty much all sides of the debate as an exit criteria. Until Petraeus took over training the army, the effectiveness of the US training program was apparently judged by the metric of how many Iraqis were enrolled in the army. This number has fluctuated, but it is generally in the six figures meaning that the Iraqi army and police supposedly has a larger footprint on the ground than coalition forces. What this metrics does not
contain is any ability of the forces to be effective (their shots hit targets, for example), whether they run when the first shot is fired, and how well trained and coordinated they are. Since Petraeus and Dempsey have taken over training these metrics are being put into a larger context by assessing
readiness.

What might all this mean to Infosec? When we measure the amount of coverage a security mechanism has, we should also consider the readiness of the mechanism (and its administrators) to respond and defeat an attack. The mix of a quantitative metrics inside of a qualitative maturity or surety level can yield a more three dimensional model that maps more closely to reality.

Gauging Iraqi Readiness Centers on 'Feel'
Beyond Metrics, U.S. Taps Battlefield Views to Assess
Local Troops' Strength, Progress of War
 WSJ

 "During Vietnam, he says, a key metric that he and other Marine officers used
was whether a four-man Vietnamese security team could walk through the
marketplace in a village without being ambushed. In 2004, when the Marines
turned over Fallujah to Iraqi security forces, the Marines knew the enterprise
was doomed when the head of the Fallujah brigade drove home every night to
Baghdad to sleep. "He was too scared to spend the night in Fallujah with his
men," Mr. West says.

Along those lines, Col. Richard Swengros, who oversees the development of Iraqi
police in Iraq, says one key measure of progress is the number of police
stations lost to the insurgents. The Iraqi police surrendered dozens of police
stations to the insurgents in the latter half of 2004. Despite heavy casualties
from car bombs and attacks, the police haven't surrendered any stations in
Baghdad in 2005. "The Iraqi police are standing and fighting," he says in an
email from Iraq.

Other Pentagon officials, however, point to less-heartening measures. One army
official with experience in Iraq recently suggested that a key metric worth
watching is how many Iraqi army and police colonels and lieutenant colonels are
killed each month. Several weeks ago, an Iraqi special police colonel was slain
by insurgents. "If you are an officer in that unit right now, you are watching
your back closely or trying to cut a deal with insurgents," the official says."

Qualitative readiness levels are assigned to the battalions and police forces these qualtative assessments attempt to measure staff skills, training, equipment, communication, and sustainment.

There are three readiness levels:

Level 1 is fully independent, “capable of planning and executing operations, and
sustaining itself, without coalition support.”

Level 2 units are “in the lead,”, "capable of planning, executing and sustaining
counterinsurgency operations with some coalition support."

Level 3, “fighting alongside”: "capable of conducting counterinsurgency
operations in conjunction with coalition units."
Tigerhawk

When we look at any number of infosec metrics like access control product's coverage in system, or patch levels, or scanning tool coverage, we are getting important, but insufficient data. These qualitative metrics need to put into a larger context that horizontally addresses the system properties to gauge resiliency for the system as a whole.

Comments

[Dominic White]

What does it mean to:
"horizontally addresses the system properties" and if I was sitting with a stack of metrics how could I practically achieve a horizontal guage of system resiliency?

Thanks,
Dominic

[Gunnar]

When I use the term horizontal, I mean across domain. The domains may be databases, app servers, and web servers, for example. The purpose of horizontal approach is to syntehsize concerns across disparate domains.