Beginning on May 2, 2012, I wanted to find out two things with regards to one question.
- How supportive were voters to a Western intervention in Syria similar to the support provided to the Libyan resistance to overthrow Qaddafi?
- How susceptible were voters to the bias of loss aversion? Much of the debate surrounding a Syrian intervention centers on the fear of military weapons and aid falling into the hands of al Qaeda affiliated individuals and groups. Dan Ariely describes in his books, The Upside of Irrationality and Predictably Irrational, how fear of losses can loom larger than gains thus influencing our decisions with regards to risk. Having backed militias in Afghanistan that later provided the seeds for al Qaeda, the U.S. national debate with regards to backing the Libyan rebellion and now the Syrian uprising continually echoes with fears of “What if terrorists get our weapons?” – a justifiable fear.
To test these two things with one question, I’ve conducted a several month long experiment here at this blog via the “1 Year After Bin Laden” poll beginning on May 2, 2012 and a series of blog posts (#1,#2,#3,#4,#5,#6) during the months of August through October. These blog posts used a variety of framing techniques designed to skew voting results with regards to ‘loss aversion’ testing (BTW – only @jeremyscahill - a journalist of course – called me out on my ridiculous framing of some of the questions). The experiment and results come in two parts.
Experiment iteration #1 – Last question of the “1 Year After Bin Laden” survey – May 2, 2012 through July 16, 2012
Beginning on May 2, 2012, I distributed the “1 Year After Bin Laden” survey in a variety of venues. However, there were two versions of this survey. Some respondents (90 in total) answered this question with regards to supporting Syria.
With regards to the current uprising in Syria, should the U.S. and European nations provide weapons, training and funding to the rebellion against the Assad regime if they can guarantee that 95% of all support will be gained by resistance fighters with no demonstrated connection to or ideological affinity for al Qaeda?
Some respondents (106 in total) answered this question with regards to supporting Syria – a question designed to frame the issue in terms of losses. The hypothesis being those who receive the question referencing ‘loss of support to al Qaeda’ would select the choice to “not support the Syrian rebellion” at a higher rate. Here’s the alternate question.
With regards to the current uprising in Syria, should the U.S. and European nations provide weapons, training and funding to the rebellion against the Assad regime even if 5% of all support provided would be lost to resistance fighters with a demonstrated connection and ideological affinity for al Qaeda?
Results Experiment #1: In this first experiment, I detected no sense of loss aversion skewing respondent choices with regards to supporting the Syrian resistance. It didn’t matter which question voters received, they selected “Yes” or “No” in roughly the same distribution regardless of question context. The overall balance of votes was 39% saying “Yes, we should support the rebels” and 61% saying “No, we should not support the rebels.” In fact, those that received the loss aversion question were slightly more likely to select “Yes, we should support the rebels.” I’ll have more analysis of these results below, but here is the breakdown chart of professional group votes for the Syria support question during the months of May through July. Interesting points were:
- ‘Academia’ voters were most likely to reject the notion of supporting the Syrian resistance.
- ‘Military’ voters were more inclined to support the Syrian resistance even if some support were lost to people affiliated with al Qaeda.
Experiment iteration #2 – Single question posted at “Selected Wisdom” – August 27, 2012 through October 18, 2012
After analyzing the results from the “1 Year After Bin Laden” poll, I wondered if question wording, structure or placement made the bias of loss aversion not emerge. Starting at the end of August, I decided to run this experiment again to look for 1) whether loss aversion was present with respondent choices and 2) if overall support for a Syrian intervention had changed since media coverage of Syria fighting became more profuse in recent months.
Through a series of blog posts on Syria and distribution of links on Twitter, several respondents (42 in total) answered this question with regards to supporting the Syrian resistance.
Should the U.S. and European nations back and resource the rebellion against the Assad regime in Syria if 95% of all support will be gained by resistance fighters with no connection to or affinity for al Qaeda?
Alternatively, some blog posts and Twitter links received answers to a different question (40 in total) with regards to their support for backing the Syrian resistance – again the hypothesis being those who receive the question referencing ‘loss of support to al Qaeda’ would select the choice to “not” support the Syrian rebellion at a higher rate.
Should the U.S. and European nations back and resource the rebellion against the Assad regime in Syria if 5% of all support will be lost to fighters connected to or aligned with al Qaeda?
Results Experiment #2: In the second experiment, despite significant changes in the intensity of the Syrian conflict, elapse of time, question framing, etc., I received almost the exact same results as in experiment #1. I detected no ‘loss aversion’ bias. Again, the overall balance of votes was 39% saying “Yes, we should support the rebels” and 61% saying “No, we should not support the rebels.” A quick caveat, some of the voters to the second experiment were assuredly the same as those that voted in the first experiment. However, a significant amount were different as I used different and more dissemination platforms in the second experiment to gather an alternative sample. I’ll post more cumulative analysis below, but here is a chart showing the results of experiment #2 from August 27, 2012 through October 18, 2012.
So, what does this all mean? I have lots of theories but a definitive answer would require more experimentation. Here are some of my initial thoughts:
- The crowd showed no real bias towards ‘loss aversion‘. Looking at the table below, across the board respondents of all demographic breakdowns were generally split at a rate of 40% for intervention (‘Yes’ -Votes) and 60% against intervention (‘No’- Votes) with one notable exception in yellow.
- I believe the resistance to ‘loss aversion’, assuming I properly crafted the questions, results from a highly educated audience that knows a considerable bit about counterterrorism, counterinsurgency and national security in general. Respondent knowledge of the topic helps them offset against question framing and arrive at decisions more analytically in this context. This doesn’t mean that if you asked the same audience ‘loss aversion’ questions about the stock market, for example, that they would be equally resistant. My guess would be I along with many of the respondents would be much more prone to a ‘loss aversion’ bias if queried on subjects for which we have limited knowledge and less data from which to offset the fear of losses.
- The ‘loss aversion’ question in the context of a Syrian intervention may not have worked because many I have talked to, and several respondents noted below, have a definitive ideological stance about foreign intervention of any kind. Essentially, many I talk to reference getting involved in Syria quickly retort with “we should never get involved in these foreign interventions, look what happened in Libya (Iraq, Afghanistan, fill in the blank).” Others will quickly respond with, “we intervened in Libya, so why shouldn’t we help out the Syrians?” I believe individual respondent stance on foreign intervention in general overrides any bias detection injected by me through question structure. Whether its Syria or any country, respondents have a pre-determined position on interventions.
- The recent U.S. support to Libya likely plays heavy on the minds of respondents and, depending on political preferences, can shape the responses to the Syria question. The debate on Syria currently rests in a bizarre twist as I noted in a post this week. The GOP appeared against a Libyan intervention under the Obama administration last year, but now has gone all in for supporting a Syrian intervention. Meanwhile, the Democratic Obama administration backed the Arab Spring uprising in Libya but seems particularly reluctant to get involved in Syria before the upcoming election. I’m curious how this will shake out in next week’s Presidential debate, and I have no idea how this shapes respondent choices to the Syria question during Experiment #1 or Experiment #2.
I’ll conclude with some last points related to the results breakout in the table below. The table shows the results by demographic attribute in Experiment #1 across both question types -”gain” and “loss”. The results for Experiment #2 are at the bottom. In green, I’ve highlighted lines I found particularly interesting and in yellow I’ve highlighted the most fascinating result. Here are some final points:
- Those identifying ‘Social Media’ as their primary information source were more against intervention on average. Meanwhile, those that prefer ‘Newspapers’ seemed more balanced in their support for or against an intervention in Syria.
- Those preferring ‘Television’ as their primary information source (a small sample) were ironically more supportive of intervention in Syria. Is this because television portrayals provide more sympathy to the opposition and relate atrocities to the viewer in a different way? No idea, but interesting.
- The most interesting result is in yellow and relates to whether respondents live in and around the Washington DC metropolitan area. Those residing around the nation’s capital were 20% more likely to be against a Syrian intervention than those that are currently living outside the beltway. How about that? What do folks in DC believe that the rest of the U.S. and world perceives differently?
- In Experiment #2, I thought support for a Syrian intervention might go down after the attack on the U.S. Consulate in Benghazi, Libya. However, the incident didn’t seem to change voting patterns. In fact, the small sample of voters just before the attack were more against a U.S. intervention than those that voted after the Benghazi tragedy.
Here’s the full table, thanks to those that have voted and below the table are the additional comments provided by respondents to support their vote – some really insightful comments that define the key factors needing analysis as the U.S. sets its policy in Syria moving forward.
Here are the open comments submitted during experiment #1 from May through June 2012.
Here are the open comments provided during experiment #2 from the end of August through mid-October.