Geospatial Editor: Joyce Hogan
Download The Full Article: The Statistical Irrelevance of American SIGACT Data:
Maneuver warfare at its core is a mechanistic endeavor and fits with a corresponding necessity of top-down hierarchies. Conversely, counterinsurgency is a more ambiguous environment that varies in its complexity and context; it is the chess match of war. It is different in every locale and can cover the entire spectrum of war simultaneously. Consequently, counterinsurgency is difficult to put on a bumper sticker, to trademark as a catch phrase, or sell to a population and their representatives. In 2006 the United States (U.S.) public's perception of success or failure of the Iraqi counterinsurgency strategy was concentrated around the concept of massing combat power in time and space, often called the "The Surge." The term, "The Surge," condensed a new counterinsurgency strategy into a simple and quantifiable slogan for the sound bite culture surrounding current affairs in the modern world. Unfortunately, counterinsurgency is more complex than "add more and then you win."
Download The Full Article: The Statistical Irrelevance of American SIGACT Data:
Major Joshua Thiel is a United States Army Special Forces Officer and a recent graduate of the Naval Postgraduate School with a Masters of Science in Defense Analysis and of American Military University with a Masters of Arts in Low Intensity Conflict. His undergraduate degree in Economics is from the United States Military Academy at West Point. He has deployed to Iraq, Thailand, Malaysia, and Papua New Guinea serving in both the Infantry and Special Forces. He continues his association with the Defense Analysis Department at Naval Postgraduate School by consulting with the CORE Lab as a Research Associate. He is currently serving in 1st Special Forces Group (Airborne).
About the Author(s)
I would be more appropriate to write a completely separate article describing the mistakes made in "The Statistical Irrelevance of American SIGACT Data", but they are so large that a timely response is required to help prevent readers from repeating them.
The first two mistakes appear in the first sentence. Maneuver warfare is neither mechanistic nor top-down. In Marine Corps doctrine, it relies on the concepts of commander's intent, main effort, and critical vulnerability to allow decision making at the lowest possible levels. General Charles Krulak described this best in two words: "Strategic Corporal."
The author also fails to define correlation properly. In statistics, and the manner used in this article, correlation is the strength of a linear relationship between two variables. It does not "measure" cause and effect. It is regrettable that the word appears 8 times in the text before it is ill defined. The "correlation" of .14 that the author reports is, based on rough calculations, likely the R-squared statistic, which is commonly used and misused for convienence (it is sometimes difficult to communicate the meaning of a negative correlation). It is entirely possible for two variables to have a low linear relationship, but be highly correlated by a non-linear relationship (i.e. the numbers 1, 4, 9, 16 are the squares of 1, 2, 3, 4).
The "study" states that it "proved that these intangible factors affect security more than the number of deployed coalition battalions." There is no "proof" of this. What the graphics provided do suggest is that SIGACTs per battalion decrease over time (which could be evidence of surge success). To "prove" that one variable (intangible factors) had a greater effect than another (number of battalions) you would require some form of hypothesis test, or preferably an empirical case study of both variables.
The region in Iraq where the article's thesis is supported, an increase of battalions decreases SIGACTs, is mentioned but then ignored. The city of Basra (mentioned as Maysan province in the article) had a large increase in security forces in 2008 as Maliki moved the Iraqi Army, with British, US Army, and US Marines in support, to squash the Sadr faction in the city. Accordingly, the province goes from a decrease in battalions in Map One with a 49% increase in SIGACTs, to an increase in battalions in Map Tow with a 74% decrease in SIGACTs. The article's comment on Maysan reflects an ignorance of operational events in Iraq in 2008.
Lastly, the entire use of SIGACTs for this comparison is at best a bad indicator of security and at worst a bait and switch. Independent reports of the dead and wounded would be a much better indicator, or perhaps economic indicators that flourish in secure environments. As described, we have no idea what constitutes a significant kinetic event, which may indeed speak to their irrelevance.
This article was successful and proved a point, though I feel the author made drifted from his strong position when he began relating the finding to Afghanistan. I actually felt the authors other SWJ article on COIN ratios was a stronger article that ultimately supports the same thesis: "how employed over how many are employed."
These types of articles I find amusing. It's like we can't shake the legacy of McNamara and "Systems Analysis." I wonder if the author has his Lean Six Sigma black belt yet. I'm going to have to talk to some of my buddies teaching over at NPS.
Do you folks really think we defeated the enemy in Iraq? I would say not, we essentially pulled out of the cities (where the enemy is) and went into the countryside after kicking their butts for a while during this surge thing, which we did not sustain. We simply did like Roberto Duran and said, "no mas." Did we think the insurgents were going to come out of the cities and chase us down in the countryside? Of course not. Leaving the battlefield and then pumping money into the enemy's country doesn't mean we defeated the enemy. They are still there. Go back into the cities and see. After all it was for the most part an urban insurgency.
Anbar Awakening - This had nothing to do with Petraeus. This started under the initiative of a few captains and light colonels. Those with stars who had careers to manage told these guys they were on their own. Of course when it started working, the Courtney Massengales readily took credit.
As for the term "COIN Jedi," I think it is much too complimentary to add the "Jedi" part. I would use the term "COINdista" as per Ralph Peters. COINdista sounds more like "Fashionista" which is what COIN is all about - being in fashion with the intellectual class at the local wine & cheese tasting.
You may not realize it Dale, but the hearts and minds matrix dominates. It directly shapes current operations in Afghanistan; on commander's mission slides from company all the way up is the never ending proclamation that the "population are the center of gravity" or the "prize" or that the goal as General Carter said recently is to win "the argument in the minds of the people." Dude that is pure hearts and minds stuff. I could go on endlessly, but why. You may see a cold logic to it all of a clear understanding that the two are linked and inseparable, but within the American army hearts and minds coin has become operationalized. Bing West characterized it correctly when he said that the purpose of the American military in Vietnam was to kill the enemy in order to protect the population but now in Afghanistan the purpose of the US Army is to "persuade the population to turn away from the Taliban."
There should be a strategic choice between the two Dale, really there should. There also should be the choice to blend both. But the matrix of pop centric coin has as its overall super structure that of state building and the provision of services and security which if done correctly, as the theory goes, will win the allegiance of the local population over to the side of the government (General Carter's statement verifies this to be the case). You may see some clearer logic to it all Dale, but not many others share your clear view of things. And until our doctrine is decoupled from this outmoded state building approach, we are doomed in the future to deal with other problems of instability in the world in the same way.
Your argument that there is no link between the Anbar Awakening and what happened in the south of Baghdad is silly. Come on, are you saying that the people and groups (shia or sunni) had no sense of what was happening in the west or in Baghdad? Come on, this makes no sense. They certainly knew what was going on and drew on it to shape their own actions. In fact I personally saw the beginnings of its spread in Ameriyah, Baghdad in fall of 06.
Lastly, fine, just like I acknowledged in my review of your book on the 3ID there is something to the increase of numbers of troops south of Baghdad and how that increase interacted with the other conditions on the ground. But to take that simple, tactical point and then turn it into an argument that the Surge writ large drastically changed the situation in Iraq is what i argue against.
You are making the same mistake that the Krepinevichs, the Nagls, the Sorleys have made with Vietnam. You are mistakenly elevating either actual or perceived tactical success into strategic progress. So what if the 3ID was tactically successful south of Baghdad? Strategically the surge was a failure. You have fallen into the trap of the promise of a better tactical war in Iraq saving strategy and policy, and I imagine you are now making the same mistake in Afghanistan.
Yeah sure anytime bro, anytime at all I am happy to further debate and discuss. But you do not hold the key to the vault of knowledge on Iraq, or Vietnam, and your quest for a usable past in both of those wars is only contributing to the mire of stalemate we are in Afghanistan now.
Ironically, the title of the piece sums up the problem--it reflects what should have been its conclusion.
SIGACT data during much of this period were drawn solely from direct observations by US (and other Coalition) troops. Thus, its data are not independent of changes in troop strength and are very likely to correlate with it: More observers simply result in more observations.
It's like the old question of whether an increase in drug seizures at the border indicates a greater percentage is being interdicted out of a steady or decling flow of drugs, or could actually be a declining percentage of an increasing amount. In other words, just because seizures are up it doesn't necessarily mean fewer drugs are making it across the border.
I don't have a suggestion on how to statistically accomplish it, but unless MAJ Thiel controls for the very likely possibility that more troops in the country means a greater percentage of violence is observed and reported regardless of the underlying "real" incidence of violence, the conclusion that "other critical variables and policies" were large factors cannot be supported--especially since they were not tested.
The solution might be in the data that MAJ Thiel notes on page 8 as being beyond the scope of his research: Iraqi reported data. For years, this was available but deliberately excluded from MNF-I reporting under the theory that it was less reliable than reports from Coalition Forces.
Also problematic is that most of the data set remains unnecessarily classified, making it difficult for researchers whose work is limited to open sources to do the sort of detailed parsing of the data that MAJ Thiel and some of the other commentators have suggested, e.g., a finer division of the x axis into months, weeks, or days; and sub-categories of SIGACT such as attacks on Coalition Forces, attacks on Iraqi Security Forces, and attacks on civilians.
Gian: I disagree with pretty much everything you say about the surge in Iraq, and the comparison between Westmoreland-Abrams in Vietnam and Casey-Petraeus in Iraq is way off base. If you read my position on Iraq as saying that Petraeus was the savior of Iraq then you missed my point completely. I do say that the numbers were important--they paved the way for all else that followed.
As for Anbar and the awakening, it is completely different from the Sons of Iraq south of Baghdad, and there was little continuity or evolution between the first and the second. Anbar had all Sunnis, they lived in only a handful of towns, etc. Most important to the issue though is that in Anbar, the prerequisite was that US forces had brought about enough security to allow the Sunnis to reject al Qaeda and "reconcile." This did not simply spill over into other areas of Iraq, as you seem to believe. South of Baghdad, where populations were mixed between Sunnis and Shiites, there was ZERO reconciliation with the Sunnis until security was more firmly established along the Tigris and Euphrates Rivers (April and May 2007 along the Euphrates and August-September on the Tigris). It is very clear that the SOI emerge ONLY when they felt safe enough to do so, and this was directly linked to permanent presence of US troops (there were virtually no Iraqi army units there, which was part of the problem in MND-C). There is a direct tie-in between security and the presence of US forces and the increase in SOI south of Baghdad. I do not see how you can make the simplistic argument that the Anbar Awakening is the direct ancestor of all else, nor do I see how you can ignore the importance of American security forces. I'm no hearts and minds guy, as you know, but your insistence on linking the troop surge with HAM is baffling to me. The numbers of troops don't herald a new way of fighting (that's the Koolaid put out there by analysts--both civilian and military) and your position has only soldified the mistaken notion that there is a strategic choice between population security (not herats and minds) and kinetic operations. This is also where you misread and mis-charachterize the Vietnam case.
Bottom line: the Amercans were indeed the most important factor in the surge south of Baghdad--and that stemmed from increased numbers, not brilliant COIN strategy. And, yes, I do believe in my own research on the 3d ID in Iraq--and in my work on Vietnam as well--and I'd be happy to debate you anytime on either subject.
Another point I would like to make about the 'market of force' in Iraq. I remember being on several contracts in Iraq where our guard force of Iraqis were filled with folks from Hit, Fallujah, Ramadi, etc. All the 'bad areas', and at the time period where there was active fighting in those cities. These folks were all of fighting age, and I know there were several in our guard force that probably fought for the insurgents at one point in the cities.
What I wanted to mention here is that they were taken off of the battle field in those cities, and inserted into our guard force, and paid to fight and possibly die for our company--and not al qaeda or the insurgency. Our company worked for the coalition, and was contracted to do a service for the coalition. So by association, these folks were now on 'our side'.
That is a radical concept folks. One of my contracts, we had hundreds of fighting age Iraqis, and these were the guys that could not or would not join the Iraqi Army or Police. I think we were paying each contracted Iraqi about 500 dollars a month, if I am not mistaken. Although that price went up and down, dependent on the owner of the security company/sheik they worked for. So there is one area of study, that is entirely missing from any surge studies.
Another thing that happens when you employee and keep these young guys under your watch. You can talk with them and win them over. I would make a point of talking with our guard force, and get a feel for what they were all about. Some of these guys you could tell wanted to kill you, but wouldn't because they cared to much about the job. Others were more on the fence or were totally supportive of the coalition. And guess what, you had their ear all day long on post. lol You can win them over with kindness, with food, with you name it. The point here, is that just by the fact that they were in our company, and not the company of the insurgency or al qaeda, that we could dispel myths about what it is that we were. We were not just foreigners with guns, we were their co-workers, and we helped to fill the information void on one another. Plus these kids and guards would come to really enjoy hanging out with the American (or Brits, or whatever expats) and look at us as their cash cow.
That last part is important as well. A young man in Iraq, like many young men in the world, strive to make a better life and get married. They also want to be attractive to women so they can 'fiki fiki'. (if you don't know what that is, ask around) Well in order to be attractive, they need a home, and they need to be bread winners. Being employed increase self esteem, and an Iraqi's value to the local population's women. So for these guys who worked our contracts, a steady paycheck to fund their dreams of homes, businesses, and a family, was very important. Stuff to think about....
Interesting paper. I would have liked to have seen more numbers on the Awakening. Meaning, how much money did we spend on buying the loyalty of these private security groups? Or buying equipment and weapons for them? The reason why I say that is because Iraqis don't do anything for free, and their sheiks don't do anything for free. So perhaps a run down of the tactical use of funds for standing up these private militias should require some study, or at least some mention in papers like this. Someone should have the numbers on money spent?
Also, there was no mention about how much Al Qaeda was paying in their market of force. They paid bomb makers, and hole diggers, and snipers, etc., and Iraqis lined up to do this work. Was there a point where they were actually sick of Al Qaeda and the insurgency, or did they simple go over to the side with the highest chance of success and who had the best gear, and most importantly--the best pay?
I say this, because we are seeing a market of force in Afghanistan right now. The local populations are making their bets. Do they work for the coalition as a soldier, or do they pick the Taliban's side, because when the coalition leaves, these will be the folks in charge out in the hinterlands? Also, who pays better--the Taliban or the government/coalition/ISAF?
I guess my point is, is that could learn a lot by looking at each Afghan or Iraqi as a company or contractor, and try to determine what market forces dictate their actions? Just a thought....
...moreover Dale, if one studies the strategic planning documents of General Casey from 2006 and early 2007 one too sees continuity rather than discontinuity with Petraeus and the Surge. To be sure Casey was not calling for five additional brigades, but he did believe that a few more were in order. He also understood that the population needed to be protected, that the violence from the civil war needed to be reduced significantly before the ISF could take the lead, and that the US military would need to remain in force in Iraq well into 2008. How is any of that fundamentally different (except for a few more brigades) than what Petraeus did.
Casey himself was aware of the Anbar Awakening and its spread as I personally on numerous occasions took him into Ameriyah were he consuled with many of the leading sunni Imams there.
Your kind of argument that placed the Surge as the fulcrum point of change in Iraq tends to assume that if the Surge didnt happen, things would have come to abrupt end.
And that Dale, is not supported by the facts.
But the origins of the sons of Iraq, the Anbar Awakening and its spread started before the Surge even started, Dale.
And if you read my post carefully you would have seen that what I called into question is the notion that the answer to the lowering of violence was some kind of operational shift by the Surge units which therefore becomes the main cause of the lowering of violence. It is that notion that I question.
You have agreed with me that with regard to Vietnam the operational framework remains the same throughout the war: search and destroy. There was no tectonic shift when Abrams rides onto the scene.
I argue that one sees the same thing in Iraq; yes to be sure the increase with the Surge brigades has an effect, but there is more continuity than discontinuity than from what came before.
Perhaps you want to believe your own history of the 3ID too much, Dale; that things turned or didnt turn in Iraq based on what the US military did or didnt do. Your own US-centric history also downplays the role of the Iraqis themselves and the other more conditions on the ground that were the real drivers of change.
Gian's beliefs about the surge in Iraq are simplistic and not supported by the facts. In particular, his contention that the rise of the Sons of Iraq was inevitable with or without the surge is wrong.
But to Joshua Thiel's article. A couple points need to be made. First, breaking the data down by province is misleading. For example, south of Baghdad in MND-C--where I had the most experience--it was only the areas nearest Baghdad which had surge troops and therefore the most vilolence. So it's not surprising that Babil, Karbala and Wasit might show no correlation between violence and US troop increases (and your definition of SIGACT is too limiting. Confining it to incidents between insurgents and coalition forces, without including violence against civilians, is an artifical distintion which certainly distorts the data), but the reality is that there was a causal link between troop numbers south of baghdad and an increase in security. As Thiel pointed out, there was "lag time," with increaded violence across the board as US forces built up, but then it subsided (yes, there are many other variable besides numbers here). I think it is interesting that while Thiel's figures here might show little correlation between numbers and security, the overall regional data for casualties and violence do show such a cause and effect.
In MND-C that cause and effect was obvious. Before the surge there were two under-strength brigades south of Baghdad and by the summer of 2007 there were four brigades, including two heavy BCTs from the 3d Infantry Division. This made an obvious difference--and it did so rapidly.
In the end, numbers do matter in COIN, and there is too much time spent on looking for a strategic choice in the kinetic approach and the hearts and minds approach. There is no choice--both are important. Petraeus was (and is) well aware of this.
I sense a bit of testiness in your above post, with the implication that those who criticized your article deserved your derogatory label "Coin Jedi" but those who didnt criticize it but only used your data are "constructive."
I also think your target audience statement is a bit narrow minded and really almost a cop-out. It is like saying hey some of you people who know a tad bit about Coin dont say anything critical to me because i wrote it for the uninformed. Come on dude, you can do better than that. Look, your article was not objective and a simple, neutral representation of data but highly subjective and interpretive, which was one of the things that made it interesting and provocative as well. I get the sense that some of you folks out at Monterey think that you have cornered the market on objectivity and the truth and those who criticize should be banished to coin "jedi land."
Personally I have found the best way to learn from writing published works are the criticisms that take apart and lay bare the problems with an argument and interpretation. So in a way, one learns more from destruction than construction.
Thanks for pointing out the misperception of the word "mechanistic." I should have endnoted that sentence. The word "mechanistic" relates to organizational design theory, specifically, Henry Mintzberg's organizational forms: simple, adhocracy, machine, professional, and missionary. Mechanistic and top-down were not meant as an insult or belittling of the complexities of maneuver warfare. You can find Mintzberg's information online or check out his book. At that point, you may agree that maneuver warfare of large organizations reflects the machine organizational design. The formations of WWI reflect a simple organizational structure, though massive. Lastly, insurgencies tend to take on adhocracy as an organization type, until perhaps / if they progress to a Maoist phase three war of movement at which point they evolve towards machine type. AQ has evolved to a missionary type organization as they have been isolated in recent years. I hope this clears up that statement and reclaims some "credibility."
Reference to all.
I understand the two variable simplicity of this article is disappointing to many of you who are COIN Jedi. However, please note the intent and target audience for this piece; the target audience is the average citizen, novice COIN researcher, and politicians who have latched on the concept of "surge to win."
I have received emails from two proactive readers who want access to my data so they can build upon it for future improved analysis. Hats off to them for their constructive approach.
It is good that investigators are doing retrospective analyses and not simply abandoning potentially useful data. On the other hand, we need to do better in collecting and analyzing statistics directly measuring the desired outcome (principally, indicators of civilian normalcy) as well as more specific indicators of mission inputs (e.g., number and quality of "tea" ceremonies, level of patrol activity and troop movement activity, activity levels of Iraqi forces). A study correlating force deployment levels and SIGACT is simply too far removed from both the key dependent variables (mission objectives) and the key inputs.
I respectfully suggest that when using a data source for analysis, one should talk to someone who knows something about the data source. It's not as if there aren't plenty of folks who know something about the history of the SIGACT repositories, including folks who work for NPS, who might be able to give a number of very plausible reasons why the total number of SIGACTS might go up during that particular time period, even if the number of battalions went down.
"Maneuver warfare at its core is a mechanistic endeavor and fits with a corresponding necessity of top-down hierarchies."
If you believe that, you don't understand maneuver warfare. Attrition warfare mechanistic? Yes. Maneuver warfare mechanistic? Hardly.
This opening sentence undermines the author's credibility from the get go...
This study performs generic quantitative analysis of troop numbers to only one facet of one line of operation. What was the makeup of the sigacts per province and how many were related to sectarian violence (protecting the population) vs. attacks against our forces? How much did the increase in troops impact our ability to consistently engage more of the population in influence operations? I agree that other factors influenced outcomes in Iraq over this period, having spent a year deployed there right smack dab in the middle of the period in question, but what good is shifting credit to other changes in doctrine, if the conditions are not set to execute that doctrine? I suppose this argument would hold up in court citing proximate causes, but I was always taught that there are multiple facets to successful counterinsurgency (one being the proper ratio of forces), and that if one of these facets is nil, they are all zero. Overall a very interesting premise, and hopefully the author is able to conduct future research to address those objections cited in the paper, and those brought to the table by others. It might take me a little while to digest this without more specific analysis.
Intagible means incapable of being realized or defined. Leaders continue to mention this word but in the next breath insist and driving subordinates to produce 'scratch and sniff' analysis for very complex issues that do not lend well to quantitative metrics.
Col Gentile - the 'surge' certainly supported the 'other' critical conditions you cite but I agree there were many variables to the success that did not derive from Gen Petraeus' presence.
What happened to military judgement? Has the American political leader's need for the military to portray everything as pollyanna replaced our ability for objective assessment? I am not advocating that we simplify the problem but I think we should embrace the problem for what it is - difficult! Sometimes intangibles are key drivers for success vs the $2bil of JIEDDO/misc contracts...but I digress.
While the argument of a small correlation between security and troop numbers seems valid, is it a really good metric to simply use troop numbers? Would it be more appropriate to use a base/area density or another metric that effectively reduces the number of confounding variables that were clearly addressed in the report?
Statistics are powerful but provide a definitive number (r-squared, r value..) that doesn't accurately represent the uncertainty in the data set or confounding influences.
Thanks for the questions
The spread of the Anbar awakening, the Shia militia stand-down, the physical seperation of Baghdad into sectarian districts. These were the critical conditions that came together to lower violence and they would have happened even if general Casey had stayed in command and asked for only a couple of additional brigades instead of five. In these area security coin wars commanding generals at the four star level are just not as important as the narrative has made them out to be.
Forgive my ignorance, but what is your BLUF here? At the end of your comment you state, "But in Iraq it was neither the increase in troops as part of the Surge (as Jason effectively argues) nor was it a decisive change in operational framework (as he incorrectly asserts) and instead the lowering of violence had to do with other more critical conditions occurring."
I read your comments to mean that neither the increase in troop strength nor a real or perceived change in operational framework were responsible for the lowering of violence.
An earlier comment of yours, "Since success in these wars and conflicts are simply a matter of getting the right number of troops on the ground with the right tactics and with the savior general, then they can be won again and again."; has me confused. I think I understand that second quote to be your paraphrasing of others' take on this situation. But...is that the case, or is that your analysis?
In any event, what's your conclusion regarding the proximal cause of the lower violence in Iraq?
Look forward to your response, your comments are always worth paying attention to.
Jason said this at the end of the piece:
"...in Afghanistan in 2011, will the victor once again write the history by touting the Afghanistan troop surge of 2010-2011 rather than the decisive operational changes."
What evidence, I mean hard evidence (and beyond what officers who were part of the Surge recall)that there was a "decisive operational change."? How much "decisive" operational change can there be in an area security mission where combat forces are dispersed widely and operate in a decentralized manner? This operational framework was in place in Iraq from spring of 2003 on. The answer is that there was not a decisive change in the operational framework. Oh to be sure there were some tweaks made here and there, a few more outposts here and there, but by and large it remained the same.
Unfortunately a narrative has been constructed that posits that a savior General named Petraeus came on board, reinvented his field army operationally and combined with an increase of troops was the primary cause of the lowering of violence. This is a chimera.
Yet folks, especially us in the Army who have spilled blood in these places, want to believe that what happens or doesnt happen is because of us and what we do or dont do, or because of savior generals riding onto the scene.
Yet the foreign policy elite (and many military leaders) in this country love this narrative and want it to stick because it places emphasis and criticism on the mechanics of doing these wars of intervention and state building and away from the strategy and policy that put them into place. Since success in these wars and conflicts are simply a matter of getting the right number of troops on the ground with the right tactics and with the savior general, then they can be won again and again.
As senior Army generals in Afghanistan argue "the right inputs are finally in place," so too are we already seeing calls in certain quarters for bog in Libya.
But in Iraq it was neither the increase in troops as part of the Surge (as Jason effectively argues) nor was it a decisive change in operational framework (as he incorrectly asserts) and instead the lowering of violence had to do with other more critical conditions occurring.