უკან
ორშაბათი | 11 მაისი, 2020

AI and Russian propaganda: it’s not what it looks like

[Note: This article was originally published in the On Think Tanks Annual Review. It was written by David Sichinava and Dustin Gilbreath. David Sichinava is the Research Director of CRRC Georgia. Dustin Gilbreath is the Deputy Research Director of CRRC Georgia and the Communications Manager at Transparify. The views presented in this article do not reflect the views of East West Management Institute, USAID, or any related entity.]

In the think tank world, talk about artificial intelligence (AI) is common. Using it is less common. One of the underlying causes of this may be a perceived lack of familiarity with the methods. However, AI methods – including machine learning – are probably more familiar to many thinktankers than they realise. The Russian Propaganda Barometer project, recently conducted by the Caucasus Research Resource Centers (CRRC) Georgia, demonstrates the potential of these tools in think tanks for policy insight – particularly relating to discourse analysis, and developing targeting strategies.

Artificial intelligence and machine learning are more familiar than thinktankers think
To say that artificial intelligence in general, and machine learning algorithms specifically, is a dramatically changing industry would be an understatement. From optimising electricity usage in factories to deciding which advertisement to show you online, algorithms are in use all around us. In fact, algorithms have been shaping the world around us for decades.

The think tank and social science worlds are no exceptions to this. Indeed, most policy researchers will be familiar with, if not users of, algorithms like regression. Notably, this is a common tool in the machine learning world as well social science research.

Hopefully, knowing that regression is part of the machine learning toolbox will make it clear that machine learning is less foreign than many thinktankers may think.

While regression is one method in the machine learning toolbox, there are others. Although these methods are not new, this larger toolbox has only become commonly used in recent years as big data sets have become more available.

For many products and problems, machine learning solutions might be improvements on existing think tank practices. This is particularly true when it comes to developing a targeting strategy for programming, monitoring, or anything that focuses on understanding discourses.

The Russian Propaganda Barometer Project
CRRC Georgia implemented the Russian Propaganda Barometer project, funded by USAID through the East West Management Institute in 2018-2019. The project aimed to understand and monitor sources of Russian propaganda in Georgia, and to identify who was more or less likely to be vulnerable to the propaganda.

To monitor Russian propaganda, CRRC took all of the posts from public Facebook pages of potential sources of Russian propaganda (around 50,000 in total) in the Georgian language as identified by two other organizations working on the issue in addition to several pages missing from their lists. These posts were then analysed using natural language processing tools such as sentiment analysis. Network analysis was also conducted to understand the interlinkages between different sources.

One of the key insights from the project is that most of the sources of propaganda identified were in fact from far right organisations. While some of these are likely tied to Russia, an analysis of how they talked about the West and Russia suggests that most actually have more negative attitudes towards Russia than the West.

The analysis also called attention to the sharp rise in interest in the far right in Georgia. The number of interactions with far-right pages had increased by roughly 800% since 2015. While overall increasing internet use in the country likely contributed to this, it seems unlikely to be the only cause of the rise.

The results were presented in this dashboard, as well as a more traditional report. It enables users to see what the far right is talking about on a daily basis, and networks between different groups, among other metrics.



The project also aimed to inform a targeting strategy on countering anti-Western propaganda. To do so, we merged data from approximately 30 waves of CRRC and National Democratic Institute surveys that asked about a variety of preferences. From there, a ‘k-nearest neighbours’ algorithm was used to identify which groups had uncertain or inchoate foreign policy preferences. This algorithm basically identifies how similar people are based on whatever variables are included in the algorithm. Based on similarity, a prediction is then made about whatever outcome is of interest. This led to an algorithm that provided accurate predictions about two thirds of the time as to whether someone would be more or less likely to be influenced by Russian propaganda. Further research showed that the algorithm was stable in predicting whether someone was at risk of being influenced, using data that did not exist at the time of the algorithm’s creation.

The data analysis, while cutting edge in many respects, is not beyond the means of many quantitative researchers. Neither of us have MAs or PhDs in statistics: David is a geographer and Dustin is a political scientist.

While the Russian Propaganda Barometer addressed the research goals, we’d like to highlight that AI is no panacea. For the project’s success, we combined traditional think tank analysis of the situation in Georgia with AI to generate new insights.

The Russian Propaganda Barometer project is just one type of application of machine learning to policy research. There is good reason to believe more and more policy researchers will use these methods given their ubiquity in the modern world, together with the increasing availability of the large datasets needed to study these issues.  We hope that the Russian Propaganda Barometer project can serve as food for thought for others in service of this goal.
22.02.2013 | პარასკევი

Before and After the Elections: Shifting Public Opinion in Georgia

The Georgian parliamentary elections in October 2012 attracted much international interest and ushered in an important turn in Georgian politics. In 2012 CRRC conducted four waves of a Survey on Political Attitudes in Georgia for the National Democratic Institute (NDI) (funded by the Swedish International development Cooperation Agency-SIDA) in order to track changes in public opinion associated with these major political events.
22.09.2014 | ორშაბათი

Russia as a threat: the Ukraine crisis and changing public opinion in Georgia

Following 2012 parliamentary elections, attitudes toward Russia in Georgia shifted. While in 2011 51% of the population considered Russia the main enemy of the country, in 2012 only 35% reported the same. Moreover, the share of Georgians who named Russia as Georgia’s main friend increased by 5%. In a post on the CRRC-Georgia blog, this change was explained by a so-called “spiral of silence”.
20.10.2014 | ორშაბათი

Do Armenians Still View Integration with the EU as Part of a Positive-Sum Game?

On September 3rd 2013 Armenian President Serzh Sargsyan surprised many observers, including some in his own government, when he announced that Armenia would sign an agreement with Russia to join the Eurasian Customs Union (ECU) and spurn a long-negotiated Association Agreement (AA) with the European Union. The move has been dubbed a “U-Turn” as well as a “sudden shift in policy,” although it was predated by landmark Armenian-Russian agreements in 1997 and 2006.
29.12.2014 | ორშაბათი

Georgia in a turbulent world: 2014 in review

Calling 2014 turbulent for the world seems almost euphemistic. The world witnessed renewed Russian revanchism with the war in Ukraine and annexation of Crimea, the emergence of a highly successful militant Islamic organization, Islamic State in Syria and Iraq, and the persistently tense situation in Israel erupted into another war between Israelis and Palestinians.
16.12.2015 | ოთხშაბათი

What We Know About Volunteering in Georgia

[This post originally appeared in investor.ge]

By Nino Zubashvili

Following the June 13, 2015 flood in Tbilisi, hundreds of volunteers helped to clean the disaster-affected zones of the city, which stirred the hope that volunteerism is on the rise in Georgia. In the past, studies on volunteering in Georgia conducted by non-governmental organizations (such as Helping Hand and the Civil Society Institute) claimed that volunteerism had not taken root in Georgian society, and CRRC-Georgia surveys have consistently shown a mismatch between attitudes and actions regarding volunteering in Georgia.
22.08.2011 | ორშაბათი

Georgia and Russia: Can positive relations between the populations overcome the political turmoil?

On the third anniversary of the 2008 August war the Russian Foreign Minister said that Russia will not renew ties with Georgia as long as the Georgian President Mikhail Saakhashvili is in power. Relations between the Georgian and Russian governments have been at a standstill since the conflict in 2008. Nevertheless, the attitudes of Georgians towards Russians remain positive.
12.09.2011 | ორშაბათი

Does Refusal to Recognize Elections in Abkhazia Reduce Prospects for Resolution?

A recent New York Times article argues that the failure of Western governments to recognize the latest presidential elections in Abkhazia on August 26, 2011 may hamper conflict resolution. According to the authors, Cooley and Mitchell, Western governments have a “counterproductive disdain” of developments in Abkhazia and isolating Sukhumi will reduce prospects for conflict resolution. 
07.12.2011 | ოთხშაბათი

Can a Cut NATO Supply Route Through Russia Benefit Georgia and Azerbaijan?

The 20th anniversary of the dissolution of the Soviet Union is upon us, and US-Russian tensions have risen as Russia contemplates terminating the NATO supply route through Russia. International news reports such as The New York Times detail the threat as a “death blow” to the U.S.-led NATO mission in Afghanistan and indicate that this could be a blessing in disguise for NATO hopeful Georgia, as well as for Azerbaijan.
02.07.2010 | პარასკევი

Post-Soviet States’ Democratic Decline: Results from Freedom House Report

Freedom House has just released its Nations in Transit report for the year 2010. The report attempts to quantify democratic development in Central European and Eurasian states by observing 8 separate factors – for instance, Electoral Process and National Democratic Governance - which affect the level of democracy in a given country. Each category is graded on a score of 1 to 7, with 1 representing the highest level of democratic progress, and 7 representing the lowest. Much of the media attention has typically focused on Russia.
09.07.2008 | ოთხშაბათი

Caucasus Data | Language: Russian versus English?

Recently, we happened upon an article that talks about the use of Russian across the Caucasus. Is Russian becoming obsolete? According to the article, some Georgian politicians suggest this is the case. At the same time, the article points out that the uptake of English is too slow to replace Russian as a lingua franca.
17.09.2008 | ოთხშაბათი

What do Russians think about the situation in Abkhazia and South Ossetia? -- Data Snapshot

How do urban Russians view the conflicts in South Ossetia and Abkhazia? From September, 5th-8th, 2008 the Analytical Center of Yuri Levada conducted a survey in ten big cities of the Russian Federation, interviewing 1000 Russian respondents. We have translated the results into English here, as they are only available in the original Russian on the Levada website.
11.12.2006 | ორშაბათი

Gabala Radar Station -- local health awareness

Rashida Abdullayeva examined a curious relic from Cold War days: in Gabala, Northern Azerbaijan, there is a giant radar station, which is leased out to Russia until 2012. According to reports citing the Russian Ministry of Defence the radar station has a range of up to 6000 km, was designed to detect missile launches from the Indian Ocean, and hosts around 1200 Russian servicemen.
20.02.2018 | სამშაბათი

As many Georgians think the West spreads propaganda as Russia

On 13 February, the United States released its Worldwide Threat Assessment of the US Intelligence Community. In it, the significance of Russian influence operations in Georgia were highlighted. Just eight days earlier, on 5 February, a coalition of Georgia’s leading non-governmental organisations made an official offer to support the Government of Georgia, the EU, and NATO in their efforts to counter anti-Western propaganda.
09.04.2018 | ორშაბათი

People in Georgia approve of doing business with Russians, despite interstate hostility

In the 2017 wave of CRRC’s Caucasus Barometer survey, 40% of the population of Georgia named Russia as the main enemy of the country.  Turkey and the United States garnered the second highest share of responses with 3% each.  Yet, no particular animosity towards ethnic Russians is observed in answers to a question about people’s (dis)approval of individuals of their ethnicity doing business with Russians. This blog post examines how answers differ by people’s opinions about whether or not Russia is the main enemy of Georgia.
23.04.2018 | ორშაბათი

Which groups name Russia as Georgia’s main enemy?

In 2017, 40% of the population of Georgia named Russia as the main enemy of Georgia. Yet the opinion that Russia is the main enemy of the country is not equally present in different demographic groups. This blog post uses data from CRRC’s 2017 Caucasus Barometer survey to gain a better understanding of the characteristics of those who report Russia is the country’s main enemy.
21.05.2018 | ორშაბათი

Disinformation in the Georgian media: Different assessments for different media sources

In Georgia, supporters of the government and opposition often express contrasting opinions about the independence and reliability of specific news outlets. Based on the CRRC/NDI December, 2017 survey findings, this blog post looks at whether people think or not that the Georgian media spreads disinformation, which groups tend to think so, and how this opinion differs by type of media. “Disinformation” was defined in the questionnaire as “false information which is spread deliberately with the purpose to mislead and deceive people,” and the questions about it were asked separately about TV stations, online media, and print media.
18.06.2018 | ორშაბათი

The EU, USA or Russia: Who is believed to be able to support Georgia best?

In recent years, Georgia has benefited from EU and US assistance, with around €400 million indicatively allocated for the EU’s projects in Georgia in 2017-2020, and the US government increasing assistance to Georgia in the 2018 Spending Bill. In contrast, Georgia’s relationships with Russia are tense, with diplomatic relations terminated in 2008.
27.05.2019 | ორშაბათი

Does our algorithm still work?

Within the Russian Propaganda Barometer Project, funded by USAID through EWMI’s ACCESS program, CRRC-Georgia created a model, using a k-nearest neighbors algorithm, which attempts to predict whether a person falls into one of three groups: consistently pro-Western; anti-Western; or neither and potentially at-risk of being influenced towards an anti-Western foreign policy position. The model used data from NDI and CRRC’s polling between 2008 and July 2018.  It included variables for age, education level, settlement type, and when the survey was conducted.
17.09.2019 | სამშაბათი

What divides and what unites Georgian society?

The last year has seen a number of conversations about polarization in Georgia. The President of the European Council, Donald Tusk, even commented on the issue in his Batumi speech.  One of the components of polarization, though not the sole factor, is division in society over actors, issues, and institutions.

While many things could divide the public, what do the people think and which groups report more and fewer sources of division? The April 2019 NDI-CRRC poll suggests that there are fewer perceived reasons for division in rural areas and among ethnic minorities.
30.09.2019 | ორშაბათი

Young people are learning English in Georgia

A common sentiment when discussing foreign languages in Georgia is that young people know some English, older people know Russian, and those in between are mixed. Previous CRRC Georgia analysis from 2014 supported this claim, showing that knowledge of English was on the rise among young people. The 2019 survey on Knowledge and Attitudes towards the European Union in Georgia which CRRC Georgia carried out for Europe Foundation suggests that this trend is continuing in Georgia.
06.01.2020 | ორშაბათი

Georgia’s Foreign Policy Trilemma: Balance, Bandwagon, or Hedge? Part 1

Georgia is a small, partly free democracy in a tough neighbourhood, and NATO membership remains an unfulfilled promise. While Russia is widely perceived as the main threat to Georgia’s security, the appropriate strategic or political response to the threat is not obvious. What options does Georgia have when faced with a powerful rival on its border, and what public support is there for these options?