FPRI Post on Treating America’s al Qaeda Addiction

This week I published Part 2 of my series on “Smarter Counterterrorism” at FPRI.  The second post, “Treating America’s al Qaeda Addiction,” discusses America’s fixation on al Qaeda – how we got there and what we can do to alleviate this addiction.  The discussion focuses on the role of Americans writ large, the media, the counterterrorism industry and politicians in sustaining the focus on al Qaeda.  Here is a sample from the post and to read the entire article click here.  Here’s my take on the current state of the counterterrorism industry:

“This system progressed fine until the drawdowns in Iraq and now in Afghanistan.  As the big theaters closed, this forced analysts to chase the next big threat, rapidly research a new al Qaeda affiliate and region, reassert their relevance and publish prose on al Qaeda’s next rise – all done in an effort to protect our nation from terrorism and our own livelihoods in the process. (Remember, I am a member of this industry.)  The reports routinely prescribe one of three patent solutions for defeating al Qaeda: 1) the only way to defeat al Qaeda is to completely wipe the planet of al Qaeda’s ideology 2) we must win the hearts and minds of every disenfranchised community from Africa to South Asia or 3) both of these things.  In all three cases, a multi-billion dollar campaign of undetermined length, under-researched methods with fuzzy long-run objectives is required – completely infeasible, utterly unsustainable and not appropriately scoped for the more narrow and severe threat of ‘Old Guard’ al Qaeda.

The net result of this system has been a splurge of terrorism and counterterrorism punditry by analysts increasingly removed from the frontlines with al Qaeda, relying on less and less journalist reporting and primary documents, framing thinking based on notions of al Qaeda circa 2001 rather than 2011 and trying to piece together a global al Qaeda strategy from a noisy jihadi social media landscape.  Each report, if sufficiently scary, presents another opportunity for funded research or a speaking engagement.  Who wants to read a complicated report on the rise of the next serious threat presented by Lashkar-Fill-in-the-Blank or Ansar-Fill-in-the-Blank unless its “tied”, “connected” or “linked” to al Qaeda – and “al Qaeda” means whatever you need it to be.  The counterterrorism punditry isn’t doing anything devious or deliberate. They are not members of the top 1% nor trying to lead their country astray.  Most are passionate about their profession, genuinely well intentioned and highly competitive with one another.  Anyone that’s ever sat in a meeting of terrorism and counterterrorism analysts and academics knows its really a passive aggressive game to see who’s smartest – the equivalent of the TV Show “Survivor” for people that don’t like to go outside, where everyone protects or bluffs about their sources and builds alliances to protect their food (I mean funding). The outcome is al Qaeda threat conflation, an endless game of Back-to-Bin Laden or Zawahiri informed by limited sourcing and perpetuated by competition over relevancy.

The worst part of today’s CT punditry is over the long-run it’s a self-fulfilling prophecy: by over-classifying things as al Qaeda, we hunt for more al Qaeda, and we find more al Qaeda.  We end up over pursuing, making more mistakes, spreading ourselves thin and in fact creating more al Qaeda than we eliminate.  Today’s al Qaeda and the jihadi militants swirling around them are too diffuse, scattered amongst too many cultures and countries and evolving too quickly for any one counterterrorism pundit or TV talking head to maintain a persistent understanding.”

 

Bakos on Libya Attack – Good Read

Today, the Huffington Post published an article by @nadabakos on the recent attack on the U.S. Consulate in Benghazi, Libya.  Nada’s article, “Attack in Libya Represents Subtle But Meaningful Shift In Threat to American Interests“, provides a needed counterbalance to the hopped up politically charged and flawed al Qaeda analysis proliferating the media after the death of a U.S. Ambassador.

I enjoy Nada’s post and it mirrors in many places my own thoughts (here and here) on how we assess and move forward in a post-al Qaeda world.  In fact, it was a Twitter discussion with Nada that inspired the title of my article from July “What if there is no al Qaeda? Preparing for Future Terrorism.”  Here’s Nada’s conclusion and I encourage all seeking an al Qaeda explanation to all current and future violence in the Middle East and North Africa to take notes.

Going forward, the U.S. needs to embrace a new calculus for assessing and responding to these loosely affiliated networks and militias, and watch to make sure that they do not coalesce into a successor to the threat posed by al Qaeda at its zenith. The tactics used in Benghazi resemble those used by al Qaeda, but, smaller in scope and scale, and mainly threaten our interests and assets overseas. Our diplomatic presence in other countries has always served us well when it’s open and engaging, but, like any other deployment of U.S. national power, incurs a certain degree of necessary risk. Withdrawing from the world is every bit as implausible as treating every militia as if it is al Qaeda.

Pundits Seeking Al Qaeda Connection To Libya Violence

The distressing news of the U.S Consulate attack in Libya and the death of U.S. Ambassador J. Christopher Stevens continues to confound pundits and news outlets in the U.S. desperately searching for a global al Qaeda conspiracy to explain the recent wave of violence.

The latest NPR story, “U.S., Libyan Versions of Consulate Attack Diverge”, seems to have appropriately balanced the debate.  My estimate of the attack goes as follows:

Here are my reasons for why I don’t believe this is a global al Qaeda plot nor a sign of a “rising al Qaeda”. Instead, I feel the attack in Libya represents the problems with a weak Libya security environment, the availability of soft American targets and the emergence of a new threat environment the U.S. has not properly assessed.  If this were a real al Qaeda plot typical of past events, I would have expected:

  • …a very public media announcement from al Qaeda coinciding with the attack.  If really planned far in advance, I’d expect all jihadi media outlets would have received a prepared announcement of considerable scale timed for release shortly after the attack.  The videos and announcements I’ve seen thus far and the alleged reprisal for the death of Abu Yahya al-Libi all seem haphazardly put together at the last minute trying to exploit the unexpected success of a meeting engagement.  Preparing and distributing these messages take weeks in preparation.  I imagine there will be AQ propaganda in the coming weeks taking credit for this.  If Zawahiri publishes a video in two weeks taking credit for the Consulate attack, you’ll know he wasn’t even in on it – he’s just reacting.  In fact, Ansar al-Sharia in Libya is trying to distance itself from the attacks.  It doesn’t mean they are innocent, but its not very like al Qaeda.
  • …the group would have tried to take the Ambassador alive, taken the body or staged a public execution.  I’m not convinced they even knew the Ambassador was there or that he had died.  It’s possible they did, but I’m not convinced yet. Hopefully the investigation will yield more clarity on this.  The kidnapping of a U.S. Ambassador would have been far more devastating to the U.S.  Sadly, this attack suggests that had they planned a kidnapping, they might have been able to pull it off.
  • …the attack to be quite a bit more sophisticated.  The reports I’ve read make it seem fairly straight forward – a rapid attack on known locations following a diversion.  Bigger, planned AQ attacks tend to hit public targets in high profile ways exploiting the media potential of the event.  While this was an unfortunate success for the perpetrators, I think a well planned AQ attack would have actually been much more successful from AQ’s perspective and more devastating to the West.
  • …they would have filmed the attack.  AQ attacks are often filmed by AQ members for their media value and then quickly posted online.  I’m sure this attack was filmed in parts but not in a pre-planned way to exploit it for media value.

The compulsion of media and pundits to position this as an “al Qaeda is rising again” chapter in a never-ending saga of good versus evil is frustrating.  There is no al Qaeda.  To call someone al Qaeda today literally means almost nothing as we have no collective understanding in the U.S. as to what constitutes al Qaeda.

Just last month, I posted a question asking readers “Who should we call al Qaeda?” based on a hypothetical small band of jihadi militants operating in the Middle East, North Africa or South Asia calling themselves “Ansar al-something”. More than 80% said “No”. See graph below.

Senators Lieberman and Collins appear to be doubling down on their amorphous “al Qaeda is everywhere” philosophy, but I bet if you asked them what constitutes “al Qaeda” they would probably struggle to explain it:

“I have come to the opposite conclusion and agree with the president of Libya that this was a premeditated, planned attack that was associated with the anniversary of 9/11,” she said, adding that classified briefings she had seen supported her conclusion. “I just don’t think that people come to protests equipped with RPG and other heavy weapons. I think the report from the president of Libya is more likely the correct one.”

Lieberman quickly sided with her, saying, “My own inclination is to agree with Sen. Collins, as I usually do, but I will await the investigation.”

Of course, Senators Lieberman and Collins strongly prefer the “one equals many” theory of al Qaeda.

The West and particularly the U.S. is doing itself a great disservice viewing the current threat environment via an “al Qaeda only” lens.  What we are seeing is a new post-al Qaeda security environment where a host of militant groups on more than three continents purportedly follow the al Qaeda ideology but ultimately choose their own violent path forward.  An upstart militant group leader (like one in Libya) competing for funding and popular support amongst a sea of militant groups has no reason to wait for a far off al Qaeda leader (Zawahiri for example), whom they likely don’t even know nor receive any funding from, to issue orders about who to attack.

Jihadi militant group leaders have now entered the “O.Y.O.” era – On Your Own.  Militant groups are rebuilding, consolidating, finding new bases of support and new financial backers.  I again return to my stance from this past July hoping smart analysts and pundits will learn and move on from al Qaeda exploring each group as its own entity. In so doing, the U.S. should develop new policies, strategies and tactics which prevent us from over-reacting, allow us to expand our thinking on counterterrorism and adequately mitigate threats without building every attack into a global conspiracy.  The “al Qaeda Only” lens gives too much credit to al Qaeda and needlessly frightens a confused American public.

For those that perpetrated the attacks in Libya, capture them if you can, try them if possible and if this can’t be done feasibly then kill them without creating civilian casualties.  Our interests in these countries is quite limited so go directly after the threat without falling into the trap of trying to do nation-building.

I’ll close with my recap from July and re-up the results of the “Who is al Qaeda?” survey:

Counterterrorism analysts now face a similar challenge to those studying the Soviet Union in 1991—what do we do now? Analysts of al-Qaeda and its affiliates still have plenty to do. Instead of approaching al-Qaeda as central to global terrorism, counterterrorism analysts will be best served by opening the aperture to see al-Qaeda as one of many potential forms of future terrorism. Rather than seeking linkages between Zawahiri and every terrorist group, analyses should explore several questions, some old and some new, that break from al-Qaeda constructs seen in 2001 rather than 2012. Here are several areas of future terrorism analysis needing exploration:

  • Competition versus Cooperation: Absent an al-Qaeda governing body, will al-Qaeda affiliates, al-Qaeda upstart groups, and other militant Islamist groups in the Arab Spring compete or cooperate? Today, in comparison to ten years ago, more extremist groups occupy the global landscape. Effective counterterrorism analysis should identify when these terror groups compete and when they cooperate. Knowing when terror groups compete will help the West construct an environment around threat groups replicating the conditions most prone for destructive interference. In contrast, understanding when disparate terror groups cooperate will help analysts detect the emergence of larger groups able to execute global terror attacks on a routine basis.
  • Focus on national and regional forces rather than al-Qaeda’s global strategy: Al-Qaeda analysts in recent years have invested great effort attempting to forecast the group’s global strategy. Absent some form of centralized or decentralized governing body, sufficient financing and new crops of operatives, an al-Qaeda grand strategy appears nothing more than misplaced optimism for the terror group. For the few remaining core al-Qaeda leaders, survival and reconstitution likely weigh heavy on their minds. Despite global al-Qaeda’s decline, those with language skills and regional experience should concentrate their analysis on national and regional militant groups emerging throughout Africa, the Levant and South Asia examining the linkages between al-Qaeda and these new upstarts as a peripheral rather than primary factor of their emergence. In short, counterterrorism analysts’ regional expertise, cultural knowledge, and language skills will trump knowledge of al-Qaeda’s 2001 organizational chart and Bin Laden’s fatwas.
  • Follow the money; track the pace of attacks: Future extremist group growth will depend heavily on financing. Bin Laden ran the most popular terrorism operation on the planet and personally provided the seed capital to get his group going. Emerging groups in North Africa will depend on wealthy benefactors and illicit operations. Emerging groups seeking funding will generate attacks to raise their credibility, and as they grow in size, they’ll produce attacks at a quicker pace. Analysts of terrorism finance and attack trends may prove particularly valuable in detecting the next generation of global terrorism.
  • Syria – al-Qaeda’s last great hope: While most eyes have shifted to study AQAP in Yemen, Syria’s protracted civil war may breathe some life into al-Qaeda. Al-Qaeda cites lessons learned from the failure of past fighting against the Syrian regime.12 Al-Qaeda has an established operational safe haven in Western Iraq through which to funnel fighters and ally with Sunni tribesmen in sectarian battles against the Shia majority government in Baghdad. Additionally, Syria’s proximate location to Israel provides a parallel jihadi cause for which al-Qaeda can pursue an enduring agenda beyond the Assad regime. However, a Muslim Brotherhood-backed parallel resistance force might likely outpace a Syrian Al-Qaeda front. Only time and good analysis will provide clarity on a poorly understood Syrian rebel landscape.
  • The Iran wild card: For many years, rumors of Iranian involvement and maybe conflict with al-Qaeda have persisted.8 Some senior al-Qaeda leaders, most notably Saif al-Adel, have allegedly been in a strange state of house arrest or operational support in Iran. Iran has always been a sly state sponsor of terrorist groups, both Sunni and Shia. If tensions were to arise between Iran and Israel or the U.S., would Iran seek to sustain al-Qaeda as a proxy? Analysts deliberating this issue may provide invaluable insights in the near future.
  • Where are the most talented al-Qaeda veterans going? Today, analysts should seek to identify what path al-Qaeda’s most talented veterans are choosing to pursue. Al-Qaeda’s limited centralized control has likely encouraged some talented terrorists to move on to new groups. Knowing where these veterans go will be essential for anticipating future threats.
  • In between conflicts, the U.S. is prone to prepare for, train for, and want to refight the last battle (al-Qaeda 2001) rather than the next battle (al-Qaeda and other terror groups in 2012). While the battle with al-Qaeda is not entirely over, the U.S. and its allies should begin imagining how the remnants of the old al-Qaeda threat will re-emerge as a new manifestation among regional and transnational extremist upstarts. The West should work vigorously to identify what this new frontier in terrorism will look like.

The results of the “Who should we call al Qaeda?” poll:

Crowdsourcing U.S. National Security: In Progress Review

Those familiar with this blog have likely participated in some of my crowdsourcing experiments.  You may have needed to fall asleep one night and decided to read an article I co-authored on its application in national security issues.  Well, this week, the LA Times published an in-progress update on a recent IARPA study to see if groups of experts can accurately predict future national security events.

In “U.S. intelligence tests crowd-sourcing against its experts“, one finds Dr. Philip Tetlock, the Godfather of Expert Political Judgment, taking on a MITRE team to see if crowds can do better than internal U.S. government experts in predicting future national security events.

The study, known as Aggregative Contingent Estimation, is designed to see whether the 17 agencies in the U.S. intelligence community can aggregate the judgment of its thousands of analysts — rather than rely on the expertise of just a few — to issue more accurate warnings to policy makers before and during major global events.

Tetlock notes that his group of experts:

“In year one, we beat the unweighted average by about 57%, which was big,” he said. A control group, run by Mitre Corp., averages scores without giving weight to participants who tally the best results.

Mark Lowenthal, a veteran intelligence professional, disagrees with the notion of crowdsourcing in the national security forecasting space.

“I don’t believe in the wisdom of crowds,” said Mark Lowenthal, a former senior CIA and State Department analyst (and 1988“Jeopardy!” champion) who now teaches classified courses about intelligence. “Crowds produce riots. Experts produce wisdom.”

Well, thanks to those that have participated in my crowdsourcing experiments (1, 2, & 3). I have my own opinions based on your contributions.  I’ll keep my mouth shut for now, but expect something quite soon that strays from both Tetlock and Lowenthal’s positions as I’m currently analyzing the results of the 1 Year After Bin Laden survey.  Thanks to all of you for contributing, and if new to this blog and interested in seeing what crowdsourcing is all about, click on this link – “One Year After Bin Laden Crowdsourcing“.  Results will be forthcoming at this blog beginning in September.

And to see the results of the first crowdsourcing experiment, see this page with associated highlights.

Differing Perspectives – “Who should we call al Qaeda?” Part 3 – Conclusion

The “al Qaeda” name game will not go away anytime soon. (See here, here and here for previous parts to this discussion.)  Many different interests and perspectives drive the terminology that has dominated the U.S.’s principle adversary for more than a decade.  The most recent emergence of several jihadi-like groups (Ansar al Sharia, Ansar al Dine, many more in Syria, etc.) only further complicates this difficult and often political question. Depending on where you sit, calling an upstart jihadi group “al Qaeda” may have a variety of pros and cons.

Following up on the “Should we call this group al Qaeda?” survey question and Dr. Bruce Hoffman’s NPR discussion, I’ve put together a chart here showing some of the pros and cons in calling or not calling emerging jihadi groups “al Qaeda”.  The chart is by no means all encompassing and is meant strictly for generating discussion on the AQ name game.

In general, I’ve focused on three broad categories of perspectives shown in the left hand column: 1- Jihadi upstart groups and Existing AQ Affiliates with an AQ moniker, 2- al Qaeda Senior Leaders, both past (Bin Laden) and present (Zawahiri) and 3- U.S. government & CT pundits. For each, I tried to anticipate the Pros and Cons for calling or not calling an upstart jihadi group “al Qaeda”.  This isn’t exhaustive; just a cursory stab for discussion.

For upstart groups, Richard Barrett, Coordinator of the Al-Qaida-Taliban Monitoring Team at the United Nations, rightly noted that choosing the AQ name comes with a level of credibility.

“Al Qaeda has become a useful label for any group that essentially pursues local aims but wishes to exaggerate its reach and sophistication.”

Meanwhile, many successful insurgent groups with ties to al Qaeda have since backed away from the “al Qaeda” name and followed the wishes of Bin Laden seeking to re-brand al Qaeda as a group that cares about the issues of local Muslims.

So where does that leave Zawahiri?  If novice, unqualified and ideologically dubious groups continue to call themselves “al Qaeda” while more qualified affiliates leave the brand name, Zawahiri becomes decreasingly influential and tied to a hodge-podge of terror group misfits – something he complained about to Bin Laden.  (see al Qaeda Doesn’t Know Who Is In al Qaeda)

Most interesting of all is what the implications of the “al Qaeda” name game is to the West.  For the U.S., not having “al Qaeda” affiliates to pursue questions why the “War on Terror” continuesEnding the “War on Terror” would vastly limit the U.S. ability to pursue al Qaeda groups and members while also putting more pressure on the Department of State to designate foreign terrorists and foreign terrorist organizations;  an important role that is bureaucratically intensive and operationally limiting further slowing the pace of pursuit.

More ironically, the discussion of the al Qaeda brand has a significant impact on the counterterrorism punditry and the media.  How does one study something that may not really exist anymore?  Who is an expert?  Who can really understand a non-cohesive, amorphous, ill-defined terror threat and communicate the risks of such an entity to the public?

For the media, this is equally troubling, stories carrying the AQ moniker were easier to cover and grabbed wider audiences than stories describing “a jihadi group that may or may not have links to al Qaeda attacked a group of armed guards in a place that you could not pick out on a map and are not likely to care about if we don’t say al Qaeda is there.”

In conclusion, the al Qaeda name game has implications for lots of different stakeholders – terrorists, counterterrorists and the media.

Pundit Tracker: A Great New Website Tracking “Experts”

A recent NPR story turned me onto a great new website called Pundit Tracker.  I love the website and what they are trying to do.  Pundit Tracker is:

“bringing accountability to the prediction industry”

I pray that Pundit Tracker begins tracking the predictions of CT pundits – a topic of particular interest here at this blog.  Maybe I’ll help them do it.

If interested in the topic, listen to the following NPR broadcast “The Pundit Tracker” where Pundit Tracker founder Sanjay Ayer explains their purpose and how they’ll go about measuring expert predictions.

And for good fun, check out their coverage of Jim Cramer, which they seem to have designated as target #1.  (I must admit, I like Jim Cramer for his entertainment value.)

For those reading the blog from time to time, you’ll notice Sanjay Ayer mention several indicators of Tetlock’s “Hedgehogs” – blowhard overconfident pundits.

If you are really into expert analysis and its applications in government, you can really bore yourself to tears with my lengthy take here at this link.

Listen To Your Friends, Read Academic Publications, Build Your Confidence, Poll Results #11c

Finally back to my computer and ready to wrap up the analysis of confidence I started last month.  In the lead up posts, I noted two excellent articles/books on analysis: Kahneman’s analysis of forecasting leadership potential and the amazing work of Dr. Tetlock who examined confidence levels and the accuracy of predictions.  I then compared this to the confidence levels annotated by respondents to the AQ Strategy and Post UBL Poll.  The first post examined the confidence levels of different professional groups and the second post focused on comparing confidence levels of those with different academic degrees and international experience.

The last analysis of confidence examines the relationship between respondent confidence levels and their preferred information source.   Below is a chart showing the average confidence of respondents selecting each of the following types of information source.  The question specifically asked:

What source do you rely on the most for getting relevant information on terrorism/counterterrorism? (You can only pick one.) 

 

No surprise, those with the highest confidence selected academic publications as their primary source.  This makes sense for a couple of reasons. First, academics were the most confident overall and they selected academic publications at much higher rates than any other group; a likely correlation.  Second, academic journals conduct peer review and require data and research to back up assertions.  One would expect readers of these journals to have confidence in the information within them.

Overall, those with the highest confidence levels relied on personal relationships as their most trusted information source.  Essentially, trusted advisers – physical friends – have the greatest influence on respondents understanding of terrorism and counterterrorism.  This isn’t surprising but demonstrates the power of physical, social networks.  Friends make us feel more confident in our assertions.  Unfortunately, over reliance on personal relationships with regards to analysis can also lead to group think.  Social circles lacking dissenting opinions can be self-reinforcing (WMD in Iraq comes to mind.)

Social media, magazines and newspapers all clocked in at about the same confidence levels which also seems quite reasonable since they tend to cite the same sources.  In contrast, a small group of respondents selected radio as their primary source and overall had markedly lower confidence levels than any other group.  I have no idea why but maybe we don’t trust what we hear as much as what we read or what we see.

Should We End The ‘War On Terror’?

Scrolling through old NPR shows, I stumbled onto an excellent debate from around the time of the September 11 anniversary.  The debate question was “Is it time to end the War on Terror?”.  This Oxford style debate featured two sides.  (Note: Oxford style debates are conducted in collared, button down shirts with Khaki pants where as Bermuda style are conducted in long shorts.)

For ending the ‘War on Terror’ were Peter Bergen of the New America Foundation and Juliette Kayyem of Harvard University’s Kennedy school.  Bergen led the charge asserting the U.S. has achieved victory in this conflict killing the majority of AQ’s senior leadership and preventing any major AQ attack for several years.  While he noted AQ members or AQ affiliates could attack again, the wind is out of AQ and by perpetuating the ‘War on Terror’ mantra we are only further scaring the U.S. population and spending unnecessary resources to placate this fear.

Against ending the ‘War on Terror’ were former CIA/NSA director General Michael Hayden and Deputy NYPD commissioner/White House Dep. Homeland Security Advisor Richard Falkenwrath.  Both Hayden and Falkenwrath focused on the legal implications of ending the ‘War on Terror’.  As they importantly noted, declaring victory and ending the war would also end the legal authorities which allow the U.S. to pursue al Qaeda and its affiliate anywhere they might be in the world.  Hayden and Falkenwrath believe this reduction in authority could allow AQ to reemerge.

I found myself on both sides of this debate.  Earlier this summer, J.M. Berger brought up the important point about there being no clear definition of “Al Qaeda”.  Thus, it’s difficult to know if we’ve won since we can’t clearly define what we are fighting.

Meanwhile, Bergen provided an excellent challenge to Hayden and Falkenwrath noting that if the current state of AQ doesn’t represent a defeated organization and a U.S. victory then what will be the conditions in which the U.S. can declare victory.   Hayden and Falkenwrath couldn’t define those conditions. Hayden provided a particularly weak answer stating something to the effect (not an exact quote), “I think we’ll know what victory is when we get there/we’ll know it when we see it.” Hayden was strong at many points in the debate but particularly weak here.

Kayyem seemed to agree with both sides of the argument at times.  She noted that she thought the U.S. has won the ‘War on Terror’.  Kayyem thought we should scale down the resources dedicated to fight AQ while also protecting the legal authorities to continue pursuing terrorist threat.

Ultimately, I believe we need to end the ‘War on Terror’ while still pursuing any and all terror groups and their members wherever they may reside.  Ending the ‘War on Terror’ is important.  Pursuing a never-ending campaign against an undefined enemy ultimately hurts the U.S. financially and psychologically. Unfortunately, as mentioned in the debate, no politician will declare the end to terrorism as it is political suicide.  Politicians gain much more from building fear than allaying fear.

The crux of this debate ultimately hinges on the antiquated legal structure the U.S. uses to pursue its enemies.  The U.S. can’t end the ‘War on Terror’ without tying its hands.  Solving this problem requires the U.S. to update its laws to enable rather than disable the nation’s ability to pursue non-state asymmetric threats.  The U.S. appears far more likely to face terrorists than nation-states in the near term.  The challenges presented by cyber threats push the boundaries of warfare even further in the direction of asymmetry.

So what should the U.S. do?  Try to fight it’s enemies through guidelines constructed for a world we no longer live in? Or develop a more nimble approach cognizant of the asymmetric battlefields enabling our enemies? I’m guessing the U.S. will pursue the first option as the Executive and particularly the Legislative Branches appear incapable of accomplishing anything.  I hope the folks a Lawfare do a post sometime soon (or maybe a comprehensive book) describing how the law of war might be re-written.  They’ve had some good reviews lately.

Below is the audio for the debate and I think it’s well worth listening to and well moderated.

Academics are confident – before & after Bin Laden’s death – Poll Results #11

Building on last week’s discussion of expert confidence, I returned to the results of the AQ Strategy and Post UBL polls conducted in late April and early May of 2011.  In each of these polls, I asked respondents the following question.

On a scale of one to ten, how confident are you on this topic area and the answers you provided in this survey?

Example:
10= Extremely confident, I work in or study the field of terrorism exhaustively
5= Confident, I’m not a terrorism novice, but I’m not an absolute expert on terrorism
1= Not very confident, I’m interested in the topic, but I don’t really follow the specifics of terrorism on a daily basis

Two weeks ago, I mentioned how Daniel Kahneman’s assessment team was just barely more accurate than random guessing at predicting the future leadership potential of soldiers.  Last week, I expressed my admiration for Philip Tetlock’s research which examined the correlation between expert confidence and prediction accuracy.  In government offices, intelligence agencies, and investment firms, policy makers and investors often rely on an analyst/adviser/expert’s confidence in predicting the outcome of a future issue, trend or market.

When we ask experts how “confident” they are, what does that mean?  How do they determine their confidence? What is their track record?  We usually have no idea what the answers to these three questions are for a particular expert.  Yet, we feel much better if the expert tells us they are “confident” whether they really are or not. The U.S. has executed grand plans based on assertions of confidence. (It’s a Slam Dunk!)

While I can’t assess these two authors analysis in the polls at SelectedWisdom, it did get me curious about how respondents rated their confidence to the AQ Strategy poll and the Post UBL poll. When I took the poll, I rated myself a “6″.  And on average, 268 voters on the AQ Strategy Poll and another 130 in the Post UBL poll estimated their confidence as a “6″.  I then dug a little deeper and wanted to examine how the death of Bin Laden, an unexpected shock, may have affected voter confidence as most respondents answered the AQ Strategy poll the week before Bin Laden’s death and the Post UBL poll the week after Bin Laden’s death.

Below I developed a chart comparing the average confidence of different groups between the two polls.  I broke the comparison down by professional groups, education level and academic focus areas.  Two quick notes – the groups are not exclusive, a government worker with a master’s degree in business will be averaged in the ‘Government’, ‘MA/MS Degree’ and ‘Business’ groups.  Also, some groupings have only a few responses so averages may appear more volatile than they may actually be. (Example: Only 6 respondents in Media-Int’l Development).

Here are the results and I’ll post what I found interesting below. The first 5 categories are professional groups, the next 5 are education levels and the last 7 are academic majors.

Some interesting results:

  1. ‘Academic’ professional group and those majoring in ‘History’ were the most confident on average and held their confidence after Bin Laden’s death.
  2. ‘Government’ professional group had less confidence after Bin Laden’s death.  Maybe those closest to CT action were more cautious in their analysis after a major change in the system.
  3. Those with PHD’s and MA’s were less confident after Bin Laden’s death while those with Associate and BA degrees were more confident after Bin Laden’s death.
  4. Political Science and History majors were more confident than other academic focus areas.  Political science majors were more confident after Bin Laden’s death.  History majors were the most confident throughout.  I wonder if History majors believe “history repeats itself” so they are best equipped to anticipate the future.  I also imagine this leads them to Status Quo bias – a belief that tomorrow will most likely be like today and yesterday.  A safe bet as things on average don’t change drastically from day-to-day.  However, historians are often wrong in their predictions of the long-run future as the only thing that is certain about the future is that it will not be like the past.

I have some more break downs of the poll results on confidence coming in the next few days to include my favorite breakdown coming up in the next post.

Foxes, Hedgehogs & Confidence – Part 2

Last week, I posted on Daniel Kahneman’s NYT article explaining how confidence and accuracy appear to have little correlation when it comes to forecasting.  Kahneman noted that his forecasts of soldier leadership ability generated from the personal observations of his assessment team were only slightly more accurate than random guessing.

Kahneman’s notion echoes the research of Dr. Philip Tetlock; author of Expert Political Judgement and the basis for much of Dan Gardner’s book Future Babble. Over 20 plus years, Dr. Tetlock surveyed more than 100 experts on a host of different issues building a database of more than 27,000 predictions.  Armed with this data, Tetlock conducted a thorough analysis of expert opinion and, like Kahneman, generally found highly confident experts commonly cited in the media were less accurate than random guessing on any given prediction.  Tetlock labeled these confident but off-based forecasters “Hedgehogs”.  Meanwhile, Tetlock found the more accurate predictors of future outcomes tended to have lower confidence in their predictions.  Tetlock labeled these less confident but more accurate experts “Foxes”. Dan Gardner explains in Future Babble that “Foxes”:

“had no template. Instead, they drew information and ideas from multiple sources and sought to synthesize it. They were self-critical, always questioning whether what they believed to be true really was. And when they were shown they had made mistakes, they didn’t try to minimize, hedge, or evade. They simply acknowledged they were wrong and adjusted their thinking accordingly.  Most of all, these experts were comfortable seeing the world as complex and uncertain—so comfortable that they tended to doubt the ability of anyone to predict the future.”

I believe Tetlock’s research provides valuable perspective for both policymakers and policy advisers.  Policymakers often seek the counsel of experts and routinely put faith in expert analysis depending on the level of confidence expressed by the adviser.  Yet, by Kahneman’s admission and Tetlock’s research, those advisers most confident in their predictions and prescriptions may in fact be less accurate than random guessing.  Likewise, for policy advisers (so-called experts), they often feel pressured to appear aggressively confident when making their predictions to ensure the respect of policymakers and to sustain their status amongst other experts.  Essentially, when policymakers turn to experts, they are seeking certainty about an expert prediction as much or more than the content of the prediction itself.

I’ve lamented many times at this blog my disdain for “Hedgehogs” vaguely predicting every potential scenario with high confidence. I’ll follow up soon with a part 3 related to the polling conducted here in May. Meanwhile, FORA hosts a great series of segments where Tetlock presents some of his findings and I’ll embed his introduction here below.

Why Foxes Are Better Forecasters Than Hedgehogs from The Long Now Foundation on FORA.tv