Computational propaganda

Computational propaganda is the use of computational tools (algorithms and automation) to distribute misleading information using social media networks. The advances in digital technologies and social media resulted in enhancement in methods of propaganda.^[1] It is characterized by automation, scalability, and anonymity.^[2]

Autonomous agents (internet bots) can analyze big data collected from social media and Internet of things in order to ensure manipulating public opinion in a targeted way, and what is more, to mimic real people in the social media.^[3] Coordination is an important component that bots help achieve, giving it an amplified reach.^[4] Digital technology enhance well-established traditional methods of manipulation with public opinion: appeals to people's emotions and biases circumvent rational thinking and promote specific ideas.^[5]

A pioneering work^[6] in identifying and analyzing of the concept has been done by the team of Philip N. Howard at the Oxford Internet Institute who since 2012 have been investigating computational propaganda,^[7] following earlier Howard's research of the effects of social media on general public, published, e.g., in his 2005 book New Media Campaigns and the Managed Citizen and earlier articles. In 2017, they published a series of articles detailing computational propaganda's presence in several countries.^[8]

Regulatory efforts have proposed tackling computational propaganda tactics using multiple approaches.^[9] Detection techniques are another front considered towards mitigation;^[10][4] these can involve machine learning models, with early techniques having issues such as a lack of datasets or failing against the gradual improvement of accounts.^[10] Newer techniques to address these aspects use other machine learning techniques or specialized algorithms, yet other challenges remain such as increasingly believable text and its automation.^[10]

Computational propaganda is the strategic posting on social media of misleading information by fake accounts that are automated to a degree in order to manipulate readers.^[11]

In social media, bots are accounts pretending to be human.^[12][13][11] They are managed to a degree via programs,^[11][12] and are used to spread information that leads to mistaken impressions.^[12][14] In social media, they may be referred to as “social bots”, and may be helped by popular users that amplify them and make them seem reliable through sharing their content.^[13] Bots allow propagandists to keep their identities secret.^[11] One study from Oxford’s Computational Propaganda Research Project indeed found that bots achieved effective placement in Twitter during a political event.^[15]

Bots can be coordinated,^[14][16] which may be leveraged to make use of algorithms.^[14] Propagandists mix real and fake users;^[16] their efforts make use of a variety of actors, including botnets, online paid users, astroturfers, seminar users, and troll armies.^[4][10] Bots can provide a fake sense of prevalence.^[17][12] Bots can also engage in spam and harassment.^[13][14] They are progressively becoming sophisticated, one reason being the improvement of AI.^[16] Such development complicates detection for humans and automatized methods alike.^[10]

The problematic content tactics propagandists employ include disinformation, misinformation, and information shared regardless of veracity.^[14] The spread of fake and misleading information seeks to influence public opinion.^[18] Deepfakes and generative language models are also employed, creating convincing content.^[17] The proportion of misleading information is expected to grow, complicating detection.^[16]

Algorithms are another important element to computational propaganda.^[18] Algorithmic curation may influence beliefs through repetition.^[14] Algorithms boost and hide content, which propagandists use to their favor.^[14] Social media algorithms prioritize user engagement, and to that end their filtering prefers controversy and sensationalism.^[17] The algorithmic selection of what is presented can create echo chambers and assert influence.^[19]

One study poses that Tiktok’s automated (e.g. the sound page) and interactive (e.g. stitching, duetting, and the content imitation trend) features can also boost misleading information.^[2] Furthermore, anonymity is kept through deleting the audio's origin.^[2]

A multidisciplinary approach has been proposed towards combating misinformation, proposing the use of psychology to understand its effectiveness.^[17] Some studies have looked at misleading information through the lens of cognitive processes, seeking insight into how humans come to accept it.^[11][14]

Media theories can help understand the complexity of relationships present in computational propaganda and surrounding actors, its effect, and to guide regulation efforts.^[19] Agenda-setting theory and framing theory have also been considered for analysis of computational propaganda phenomena, finding these effects present; algorithmic amplification is an instance of the former,^[18] which states media’s selection and occlusion of topics influences the public’s attention.^[19] It also states that repetition focuses said attention.^[15]

Repetition is a key characteristic of computational propaganda;^[16] in social media it can modify beliefs.^[14] One study posits that repetition makes topics fresh on the mind, having a similar effect on perceived significance.^[11] The Illusory Truth Effect, which states people will believe what is repeated to them over time, has also been suggested to bring into light that computational propaganda may be doing the same.^[20]

Other phenomena have been proposed to be at play in Computational Propaganda tools. One study posits the presence of the megaphone effect, the bandwagon effect, and cascades.^[15] Other studies point to the use of content that evokes emotions.^[11][9] Another tactic used is suggesting connection between topics by placing them in the same sentence.^[11] Incidence of Trust bias, Validation By Intuition Rather Than Evidence, Truth Bias, Confirmation Bias, and Cognitive Dissonance are present as well.^[14] Another study points to the occurrence of Negativity Bias and Novelty Bias.^[9]

Bots are used by both private and public parties^[16] and have been observed in politics and crises.^[18] Its presence has been studied across many countries,^[12] with incidence in more than 80 countries.^[18][4] Some studies have found bots to be effective.^[2][16][15] though another found limited impact.^[13] Similarly, algorithmic manipulation has been found to have an effect.^[19]

Some studies propose a strategy that incorporates multiple approaches towards regulation of the tools used in computational propaganda.^[17][9] Controlling misinformation and its usage in politics through legislation and guidelines; having platforms combat fake accounts and misleading information; and devising psychology-based intervention tactics are some of the possible measures.^[9] Information Literacy has also been proposed as an affront to these tools.^[9][17][14]

However, it has also been reported that some of these approaches have their faults. In Germany, for example, legislation efforts have encountered problems and opposition.^[13] In the case of social media, self-regulation is hard to request.^[9] These platforms’ measures also may not be enough and put the power of decision on them.^[13] Information literacy has its limits as well.^[14]

Computational propaganda detection can focus either on content or on accounts.^[4]

Two ways to detect propaganda content include analyzing the text through various means, called “Text Analysis”, and tackling detecting coordination of users, called “Social Network Analysis”.^[4][10] Early techniques to detect coordination involved mostly supervised models such as decision trees, random forests, SVMs and neural networks.^[10] These just analyze accounts one by one without modeling coordination.^[10] Advanced bots and the difficulty in finding or creating datasets have hindered these detection methods.^[10] Modern detection techniques’ strategies include making the model study a large group of accounts considering coordination; creating specialized algorithms for it; and building unsupervised and semi-supervised models.^[10]

Detecting accounts has a variety of approaches: they either seek to find the author of a piece, use statistical methods, analyze a mix of both text and data beyond it such as account characteristics, or scan user activity tendencies.^[4] This second focus also has a Social Network Analysis approach, with a technique that looks at time elements on campaigns alongside features of detected groups.^[4]

Detection techniques are not without their issues. One of them is that actors evolve their coordination techniques and can operate in the time it takes for detection methods to be created,^[10][4] requiring real-time approaches.^[4] Other challenges detection faces are that techniques have yet to adapt to different media formats, should integrate explainability, could inform the how and why of a propagandistic document or user, and may face increasingly difficult to detect content and may further be automatized.^[10] It is also presented with a lack of datasets, and creating them can involve sensitive user data that requires extensive work to protect them.^[10]

• 2018: Samuel C. Woolley and Philip N. Howard (eds.) Computational Propaganda. Political Parties, Politicians, and Political Manipulation on Social Media
• 2020: Howard, Philip N., Lie Machines: How to Save Democracy from Troll Armies, Deceitful Robots, Junk News Operations, and Political Operatives