Lies, Damn Lies, and Metrics in Small Wars
Lies, Damn Lies, and Metrics in Small Wars
Can We Measure Progress or Failure in War?
U.S. Army Major Josh Thiel’s recent article The Statistical Irrelevance of American SIGACT Data: Iraq Surge Analysis Reveals Reality challenges the contemporary notion that you can “add more (forces) and then you win” in a protracted insurgency. Josh, an Army Special Forces officer assigned to 1st SFG, studied economics at USMA and defense analysis at NPS. His concise work illustrates what is known in econometrics as a red flag. A simple linear regression of two variables showing substantial deviation is enough to suggest that the problem is much more complicated than simply adhering to the tested independent variable.
So what? Josh’s work is the first part of extensive research being conducted at the CORE Lab in Monterey, CA to determine if we can ever accurately measure causation in war. This type of research is funded throughout the country by grants such as the Minerva Initiative, an investment endorsed by the SECDEF as a 21st century effort to promote collaboration between the military and social sciences to find better solutions in modern conflict.
Rigorous analysis and collaboration into quantifying war goes back as far as 1948. British scientist Lewis Fry Richardson sought to find patterns in conflict, and RAND’s game theorist developed the concept of Mutually Assured Destruction (MAD) that eventually translated into a foreign policy of nuclear deterrence. In the 21st century, we have yet to find a definitive model to gain a broader understanding. To date, it remains undetermined if such a model is possible.
As Josh noted, it is naí¯ve to blindly look at collected metrics from the occupying force, counterinsurgent force, or host nation in order to gleam the current situation. For example, as I noted in the Break Point, from November 2006 to February 2007, over 126 SIGACTS occurred in the small village of Zaganiyah; however, this data will not be found in the SIGACT database because there was no coalition observer there to collect the information at the time of occurrence. Instead, if you looked at the data available, it would show only a small amount of violence in the area. This finding led me to conclude,
Control, while hard to quantify, is often “you know it when you see it,” but it can be somewhat measured by the strength of the counterinsurgent security force OR the absence of violence in areas with significant government presence. The absence of violence in areas outside the reach of the government OR outside external monitor observation (press, peacekeepers) should be assumed to be under the insurgent’s control.
Several other qualitative measures must be addressed when testing or collecting data.
What is the interest of the reporting officer? There is usually a political, economic, or personal interest involved. For instance, in Iraq, Shia would often minimize Shia on Sunni violence while exaggerating Sunni on Shia violence.
How accurate is the data? While many of us call it Iraqi math, exaggeration extends to many small wars. Often, a partnering Army officer or frightened civilian would report 100 armed men attacked a patrol or a village. Upon investigation, no evidence supported this report. Over time we learned to drop a zero so 100 meant 10. How do we input this data into the SIGACT database? It remains a tough judgment call. In a perfect world, the higher command will serve as the referee, but that does not make the information necessarily accurate.
When analyzing the data, one must consider the following,
What was the coalition forces mission? What were they actually doing? Were they constrained to Advisory missions in a large forward operating base? Were they conducting primarily direct action raids? Were they living in patrol bases in contested areas?
What was the enemy’s mission? Were they actively massing to attack coalition forces? Have they temporarily laid down their arms to wait for the coalition to move elsewhere? Were they conducting selective assassinations of key leaders, acts of intimidation of the populace, or overt population control measures to pacify the populace into not supporting the host nation government?
How did the people react? What is the number of intelligence tips to coalition forces? Is the information valid? How many local leaders publically support the coalition force mission? How many religious leaders actively preach resistance in Friday prayers? How active are the local markets when coalition are present?
Again, we have merely begun to dig into studying the quantifiable aspects of small wars. There remains many more unknowns than known facts. Major Thiel’s paper should serve as a caution that there are no bumper sticker solutions, and hopefully, his work will encourage others to collaborate to help find better answers.
