Comments
BK Price,
You wrote, "Because many of these "sources" are anonymous and maybe fairly new to the scene, there is no effective way of evaluating how well they actually speak for the population. This is not to say it shouldn't be done but that these sources must be cultivated and evaluated just like any controlled sources to ensure that what we are seeing is an accurate depiction."
I really doubt the intell community is going to cultivate "credible" sources that speak for the larger populace. It is best to approach this as a census (half science, half art). There is obviously a need a for vetted sources, but not all forms of HUMINT require vetted sources. The social media can be a useful indicator, and some cases more depending on widely used it is.
You wrote,
"As one analyst demonstrated, he had accurately predicted the fall of Tunisia based on the rising unemployment, the youth bluge, etc. He was only off by a decade."
This isn't analysis anymore than analysts telling us that someday North Korea will collapse due to economic challenges and poor governance. Maybe, maybe not. It is hardly any different than stating the stock market will go up or go down, if you keep clinging to the same story you'll eventually be right, but it won't help anyone make money, and analysis that states a Nation "may" implode that is off by decade if of little use to policy makers.
You added,
"I would point that classified intelligence sources are generally focused on no kidding threats."
I am not implying that the intell community shouldn't continue to focus on "real" threats, but they also need to focus on opportunities. The current situation in the Arab world could be an opportunity (or it may in fact be a real threat) that if predicted (and I admit I don't know if that is possible) would have allowed policy makers to move quicker one way or the other.
Focusing on red forces (not the only real threat to our national interests) is insufficient, and it has always has been insufficient. In the realm of irregular warfare understanding the populace and popular movements are critical to our interests, just as critical as the tactical intell that faciitates a NEO.
In short I don't disagree with what you wrote, but I think you only addressed half the challenge. The part of the challenge we're actually pretty good at already. The other half is still an unfulfilled requirement.
Bill M., I believe Intellwar's point was to highlight the fact that in these communities the vast majority of the population does NOT have access to Twitter, Facebook, or other social media tools. Therefore, monitoring such information gives you an excellent insight into the ideas of a select few individuals who may or may not be representative of the whole. Because many of these "sources" are anonymous and maybe fairly new to the scene, there is no effective way of evaluating how well they actually speak for the population. This is not to say it shouldn't be done but that these sources must be cultivated and evaluated just like any controlled sources to ensure that what we are seeing is an accurate depiction.
Secondly, the seeds of these revolts have been recognized for quite some time by the intelligence community. As one analyst demonstrated, he had accurately predicted the fall of Tunisia based on the rising unemployment, the youth bluge, etc. He was only off by a decade. The problem was the lack of a spark...and it is very difficult to predict which immolation or unjust imprisonment is going to set off the revolution.
The problem is, intel says, "revolution is coming" and the decision maker asks, "when?" Intel shrugs and says, "could be any day now or it could be years down the road, but trust me, its coming." So the decision maker goes back to focusing on other things without further consideration. If we knew Egypt and Tunisia were going to fall ten years ago, why were we so unprepared from a policy perspective to deal with these eventualities? These plans should have been developed, the talking points established, and policy implications mapped out when intel first said "beware."
Yes, once the spark happened, social media provided much faster and more up-to-date reporting than traditional intel sources. But I have yet to see any indication that social media anticipated the spark.
Lastly, I would point that classified intelligence sources are generally focused on no kidding threats. The fact that a revolution occurred in Tunisia or Libya is not a direct threat to the US. So if the IC was late to identifying the exact date and time of a revolution starting, I'm not sure what the negative implications are from that. The only really valid argument would be the case of getting the NEO accomplished safely and effectively. But those plans sit on the shelf at the ready waiting for the Ambassador to ask for them. And they are already chock full of the intel needed to support them.
OS can be useful but there is a LOT of noise to be sifted through. And in the end, it is no better at "predicting" events than classified information. Just better fodder for Monday morning quarterbacks and CNN anchors.
Intellwar,
In the same post you state OSINT focuses on the greatest noise level (guess what, so does the majority of any nation despite statements to the contrary), and note there is a difference between a person creating noise like Sean Penn, and the clammor by thousands of people (and they only represent the tip of the iceberg).
Then it appears that you imply the population doesn't understand itself? Since listening to social media is in fact listening to the population, what sources do you suggest that we actually listen to understand the populace? It appears all the so called trusted sources in the know were completely in the dark on these uprisings, but after the fact they developed all type of explanations.
This appears to be a classic case of the intelligence community attempting to justify their current processes, which in the military are still very much focused on enemy order of battle (still required, but we need to understand more than that). The old paradigm is that open source information augments classified information, but perhaps the new paradigm should be that classified information augments open source for irregular warfare?
The seeds of these revolts were there for us to see long before the spark was set if we actually listened to the people who were talking in the open instead of listening through a soda straw to one particular terrorist cell, or watching a military exercise, etc.
Like it or not, creative destruction is coming, and the intelligence community will have to change.
One of the assumptions regarding this new emphasis on OSINT is that the sources with the greatest noise level...the ones with access and the willingness to transmit...represent the views of the greater population. That may have been the case in Egypt and Tunisia...but the support for Gadhafi by large segments of the Libyan population came as a surprise to many. Syria may prove to be the same.
OSINT tools are great for research...but they shouldn't replace the need for sound understanding of the operating environment based on sources that really know the population.
docseneca - Formidable indeed. The shear volume, or perhaps better put, the glut of information that is available for mining, and translating, must be overwhelming.
Although I have been away from the tactical for some time, suffering from that malady that eventually strikes Soldiers, old age, I am aware that at the national level, we use open source information in connection with information from classified sources, which can make it tricky to measure how much of it contributes to the end product called intelligence.
I suspect your statement that "analytical capabilities that rest outside of, but still support, intel channels" would be a good idea at any level.
Dave and Tyrtaios, I believe you both touch on another fundamental problem endemic at operational and higher levels: we are desperately short of analysts in many commands. open source research has paid proven dividends in my experience, but the review and analysis often rested on our tactical level analysts or, more commonly, our own.The volume of information OSINT adds to an analyst's workload is formidable, especially in higher command -2 shops where manpower isn't there to address the classified workload. I recall hearing last year in one Joint HQ that their 2 shop existed mostly to support the daily CUB (and as such, produced analysis that was only power point deep). I heard the NPR article this week and agree that this is another potential tool. The nifty thing about OSINT is that it can be culled and employed at all operational levels, but for its use to become widely beneficial we ought to develop (and come to count on) analytical capabilities that rest outside of, but still support, intel channels.
Tyrtaios: you identify a fundamental problem. Most of the information we pay for may be found open source or at least through analysis of open source. What really needs to be secret is not necessarily the raw information but the source and most importantly the analysis that is tied to our decision making (and not even in all cases does that analysis need to be secret). And how often have we all seen media analysis that includes the same information that is secret. We should not forget that journalists tend to be the best collectors expert at elicitation. The only difference is they cannot be tasked so we have to pay attention to what they write and report.
It would seem open source information, in the 21st century of global information sharing, would be instrumental in assisting to find out what is known, and thereby not only assist in focusing clandestine collection sources for further development, but would also give the analyst the ability to know what is the real secret they may possess?
In the end it may center around resource allocation and the mindset that only secret information has the answer to base a decision on.
I recall awhile back reading that one former Director of Central Intelligence said, "I only have enough money to pay for secrets."
Useful for understanding and at least partially assessing the resistance potential (or at least contributing to the assessment) within an area and if the strategic option of unconventional warfare is appropriate (activities to enable a resistance or insurgency to coerce, disrupt, or overthrow a government or occupying power through or with an underground, auxiliary, and guerrilla force in a denied area - hate to beat a dead horse but events around the world continue to demonstrate the relevance of UW and the need to understand and be able to execute this critical strategic option).
But Reid Sawyer is exactly right regarding our bias for classified information. Our reliance on classified information often makes use not see the forest for the trees. Open source information is equally if not more important (in some cases) than classified information.
And strongly concur with Bill M on CTC!!
I read recently that the greatest innovations in hindsight often seem to be quite obvious and we wonder why we didn't think of them, but only in hindsight. The Combating Terrorism Center at West Point in my opinion has been on the cutting edge of irregular warfare value added ideas for the past few years.