The linked Rand paper is worth the read. Great doctrinal historical run down.
I have to take slight exception to the wording of this paragraph (page 28) below. It is not that we (in Army Special Forces) consider Foreign Internal Defense (FID) to be a core mission, it is by law a core Special Operations Force (SOF) mission but by doctrine all services are required to provide forces trained and ready to conduct FID. It is not an exclusive mission to any one service or force.
Note that the variation in observed COIN practice in Iraq likely stems from differences in military organizations. Army Special Forces have always considered FID to be part of their core mission and have developed patterns of thinking appropriate to that mission. These patterns are often highly at odds with the larger Army. The Marine Corps falls somewhere between the Special Forces and the larger Army in terms of patterns of thinking. Of course, the Special Forces are not expected to win a high-intensity conflict alone and, despite the occasional exuberant Marine rhetoric, neither is the Marine Corps.
And I would also a caveat to the above. Just as Special Forces and the Marine Corps are not expected to win high-intensity conflicts alone, Special Forces an/ or the Marine Corps, and General Purpose Forces in general do not "win" FID or COIN alone. It takes an integrated whole of government effort not to win the FID/COIN fight but to support friends, partners, and allies in their fight against an insurgency (or lawlessness, or terrorism or ungoverned spaces and sanctuaries, whatever internal or transnational threat exists the country or region). But again, I hate to beat a dead horse, one of the fundamental problems I think we have with FID/COIN is that we approach it from the perspective of the US winning the FID/COIN fight. While our past doctrine has alluded to it and you can read it in the attached paper when it discusses the paramount requirement for the local government to win (or win back) the support of the population, we tend to pay lip service to that fact and a consistent theme in all our operations in post WWII has been about the US winning in FID/COIN. For example we place great emphasis about "living with the people." While I absolutely believe that is critically important (see CIDG, CAP, and CORDS discussions in the paper on pages 14-15) , it is not so much about us (as in big US) living with the indigenous people - it should be us living with the indigenous security forces - but with us in the background advising those indigenous security forces to live among their people. We have to get out of the mindset that we have to be the victors in Iraq and Afghanistan. We can only be the external support to help the Iraqi and Afghan people be the victors by helping those governments and their security forces to be successful. In the long run, it is counterproductive to try to win these fights ourselves, because we cannot. The only COIN fight we can win is one directed against the United States.
And one last editorial comment. We have to get over the "we-they" in our own military and our own government. What we have learned in the era of jointness is that no one service or one capability (e.g., SOF vs GPF) can be successful in any campaign. It takes the proper application of the right joint capabilities and resources to be successful. And the we-they problem is not just within the military -- we have a "we-they" problem within our whole of government particularly as evidenced when we say "the military and the interagency." We seem to forget that DOD is part of the interagency when we tend to talk like it is something separate. We need to get over our parochialism and get on with accomplishing the mission and getting the job done as a joint and interagency team.