{"CACHEDAT":"2026-11-12 11:42:18","HOWSUBJECTMAP":{},"SLUG":"misinformation-manipulation-rz381dHOdO","MARKDOWN":"# Information Disorder\n\nWardle & Derakhshan (2017), in a foundational report for the Council of Europe, introduce *Information Disorder* as the umbrella term for false, misleading, or harmfully shared information in the public information environment. They distinguish three categories along two axes: whether the content is *false* or *true*, and whether it is shared with *intent to harm*.\n\n| Type | Content | Sharer's knowledge of falseness | Intent to harm |\n|------|---------|---------------------------------|----------------|\n| **Misinformation** | false or misleading | does not know | no |\n| **Disinformation** | false or misleading | knows | yes |\n| **Malinformation** | true | (not the criterion) | yes |\n\nThe three categories form an interconnected ecology: the same content item can move between them depending on who shares it and with what intent.\n\n### Information Pollution\n\nWardle & Derakhshan (2017) use *information pollution* as a metaphor for the state of an information environment in which false, misleading, and harmful content circulates alongside reliable information, much as pollutants circulate in air or water. The term describes the overall condition of the environment; information disorder classifies the polluting content into misinformation, disinformation, and malinformation.\n\n### Fake News\n\nThe term *fake news* is widely used in public debate but analytically imprecise. It usually refers to false or misleading information presented in the style of news reporting. However, because the term can refer to different types of information disorder and is often used politically to discredit unwanted reporting, more precise terms such as *misinformation*, *disinformation*, *fabrications*, or *misrepresentations* should be preferred.\n\n## Misinformation\n\nMisinformation is false or misleading information shared by people who do not realise it is false. The sharer believes the content is accurate; there is no intent to deceive or harm.\n\nMisinformation typically arises in fast-moving news situations (early reports with unconfirmed details), in passing along claims without checking accuracy, and in repeating information from trusted sources that turn out to be wrong. It can be as widespread and consequential as disinformation, despite the absence of malicious intent — its sharers are often genuinely motivated and trusted within their networks, which gives the content reach and perceived credibility.\n\n### Misinformation Susceptibility Factors {{misinformation susceptibility factor_susceptible}}\n\n\n:::success\n### **1. Cognitive factors** — how information is processed\n\n* **Confirmation bias**: People pay more attention to information that supports what they already believe, and scrutinise opposing information more harshly.\n* **Illusory truth effect**: Claims can feel more believable simply because people have seen them before, even when they are false.\n* **Processing fluency**: Information that is easy to read, familiar in wording, or visually clear feels more trustworthy than information that is harder to process.\n* **Low cognitive reflection**: People who rely on quick, intuitive judgements rather than pausing to check are more likely to fall for false claims.\n* **Bullshit receptivity**: Some people are inclined to find vague but impressive-sounding statements meaningful, even when they are empty.\n\n### **2. Affective / motivational factors** — how emotions and identity shape judgement\n\n* **Emotional reasoning / reliance on emotion**: People accept claims because they feel right, frightening, satisfying, or morally urgent, rather than because they have been checked.\n* **Specific emotions** (anger, fear, moral outrage): Strong emotions can lead people to react quickly and share before checking; moral-emotional language is especially powerful in spreading political content. (Belief and sharing are distinct effects with different evidence.)\n* **Identity-protective cognition**: People process information in ways that protect their political, religious, or group identity, and resist evidence that would threaten it.\n* **Conspiracy mindset**: A general tendency to suspect hidden plots or powerful actors behind events makes conspiracy-style misinformation more plausible.\n\n### **3. Social factors** — whom and which groups people trust\n\n* **Source credibility / trust in the sender**: How believable a claim feels depends on who appears to be saying it — a friend, an influencer, an expert, a politician, a news outlet, or an anonymous account.\n* **Social proof / popularity cues**: Likes, shares, views, and comments can act as credibility cues, although the effect depends on context.\n* **In-group / partisan congruency**: Information from \"people like us\", or aligned with a group's position, is accepted more easily.\n* **Network homophily / echo chambers**: When people mostly encounter information through networks of similar others, certain views appear more widely shared than they really are.\n\n### **4. Contextual / situational factors** — the conditions under which people meet information\n\n* **Time pressure**: When people have to judge information quickly, they distinguish true from false claims less accurately.\n* **Information overload**: When too much information arrives at once, people fall back on shortcuts like headlines, emotion, source labels, or popularity.\n* **Distraction / cognitive load**: When attention is drawn elsewhere, people may share without checking accuracy, even if they could otherwise recognise the misinformation.\n* **Crisis / uncertainty contexts** (pandemics, war, disasters): Urgent need for explanation increases openness to false or premature claims.\n* **Platform design**: Feeds, notifications, recommendation systems, autoplay, and fast scrolling encourage reactive rather than reflective behaviour.\n\n:::\n\n## Disinformation\n\nDisinformation is false or misleading information shared deliberately, by people who know it is false, with the intent to harm, deceive, or manipulate. Both the sharer's awareness of the falseness and the harmful intent are definitional features.\n\nCommon forms include political influence operations, commercial deception, propaganda, fabricated content, and manipulated media. Disinformation is often produced by organised actors and amplified through *Coordinated Inauthentic / Artificial Amplification*.\n\n## Malinformation\n\nMalinformation is *true* information shared with intent to harm. The content itself is accurate, but its release, framing, or timing is calculated to damage a person, group, or institution. The classical pattern is the deliberate movement of private, sensitive, or context-bound information into a public or harmful context.\n\nExamples include the publication of leaked private communications to discredit a target, the release of accurate but stigmatising personal data (e.g. revenge porn), and the strategic disclosure of factually correct but contextually damaging information at moments calculated for maximum impact.\n\nMalinformation is the least-discussed of the three categories because the content is not false, which places it outside fact-checking frameworks. But it forms a significant part of the information-disorder ecology (Wardle & Derakhshan 2017).\n\n\n:::warning\n**The boundaries are porous.**\n\nThe three categories often overlap or shift in practice:\n\n* A piece of *disinformation*, once shared by people who genuinely believe it, becomes *misinformation* as it spreads further. The same content can sit in different categories depending on who shares it.\n* Content can be partly true and partly false: not all false content is fully fabricated.\n* Determining intent from outside is empirically hard. Without access to the sharer's knowledge state and motivation, the line between misinformation and disinformation often cannot be drawn definitively.\n\n:::\n\n# AI Hallucination / Confabulation\n\nThe systematic tendency of generative AI systems to produce plausible-sounding content that is factually incorrect, fabricated, or unverifiable — including invented references, non-existent studies, misattributed quotations, made-up statistics, and false biographical, historical, or scientific details.\n\nHallucinations occur because generative AI systems produce output by predicting plausible continuations from patterns learned during training, not by retrieving verified information from a knowledge source. The system optimises for fluency and plausibility, not for accuracy. Fabricated content is typically presented with the same confidence as accurate content — there is no internal signal that reliably distinguishes the two.\n\nThe term **confabulation** is sometimes preferred over **hallucination** in research contexts, since confabulation in psychology denotes the construction of false but sincerely believed accounts without intent to deceive — closer to what generative systems actually do. *Hallucination* remains the dominant term in public and technical discourse and is retained here.\n\n**Hallucinations are not occasional bugs.** They are a structural feature of the underlying generation process (Kalai & Vempala 2024). They occur across all generative systems — including ☑ RAG AI, where fabrication can also affect how retrieved sources are summarised, quoted, or attributed.\n\n**Common types:**\n\n* fabricated references — citations to articles, books, or studies that do not exist\n* misattributed or fabricated quotations\n* invented statistics or numerical claims\n* inaccurate biographical, historical, or scientific details presented confidently\n* confidently incorrect summaries of real sources\n\n\n:::tip\nTreat any factual claim from a generative AI system as unverified until checked against an independent source. References and quotations are particularly easy to verify — search whether the cited source exists and contains what the AI claimed.\n\n:::\n\n# Ways of Misleading\n\nInformation items can mislead in three different ways:\n\n| Category | What it affects | Intended effect on the audience |\n|----------|-----------------|---------------------------------|\n| **Fallacious argumentation** | the **argument**: how a claim is supported, e.g. logical fallacies, fake experts | A flawed argument seems convincing. |\n| **Manipulation strategies** | the **presentation**: how the information item is framed, worded, shown, or set to sound | Critical thinking is bypassed, and the overall impression replaces the facts. |\n| **Deceptive tactics** | the **content**: what is left out, invented, distorted, or presented one-sidedly | False or incomplete content is taken as fact. |\n\nOne information item can use all three at the same time. Whether an information item that uses these is misinformation or disinformation depends on who shares it and with what intent.\n\n# Fallacious Argumentation {{fallacious argumentation}}\n\nFallacious argumentation is flawed support for a claim that is made to look convincing, e.g. logical fallacies or references to fake experts.\n\n# Manipulation Strategies {{manipulation strategy}}\n\nManipulation strategies are ways of presenting an information item that are used to steer how the audience interprets it or feels about it.\n\n\n:::info\nFraming, rhetorical devices, and emotional language are not manipulative in themselves. Every information item is framed, and good arguments also use rhetorical devices. They become manipulation strategies when they aim to bypass the audience's critical thinking, e.g. by triggering strong emotions or by steering attention without the audience noticing.\n\n:::\n\n## Framing Choices {{framing choice}}\n\nFraming means selecting some aspects of an issue and making them more salient than others (Entman 1993). A frame suggests how to define a problem, what causes it, how to judge it, and what should be done about it.\n\nFraming choices include:\n\n* which aspects are foregrounded, e.g. in the headline, at the beginning, or in the image\n* which aspects are backgrounded, i.e. mentioned only briefly or late\n* which metaphors structure the message\n\nA framing choice is manipulative when it steers the audience towards one interpretation and keeps them from noticing that other interpretations are possible.\n\n→ Framing shifts the emphasis within an information item. A one-sided presentation leaves out other positions entirely, which is a specific form of omission {{omission}}.\n\n## Rhetorical Devices {{rhetorical device}}\n\n\n:::tip\nSouth Carolina Department of Education 2026: \n\n:::\n\n## Sensational and Emotional Language\n\nWords that create strong feelings such as fear, outrage, or urgency instead of giving reasons.\n\n## Manipulative Use of Visuals\n\nImages, graphs, or charts that steer interpretation through what they show, crop, or emphasise rather than through what the facts or data say.\n\n\n:::tip\nGaslowitz 2017: \n\n:::\n\n## Manipulative Use of Sound\n\nMusic, sound effects, or tone of voice that create a mood which the content itself does not support.\n\n## Mismatches between Textual, Visual, and Auditory Elements\n\nThe text, images, and sound of an information item send different messages, so that the overall impression differs from what the text actually says.\n\n# Deceptive Tactics {{deceptive tactic}}\n\nDeceptive tactics change the content of an information item: relevant information is left out (omissions), invented (fabrications), distorted or taken out of context (misrepresentations), or presented one-sidedly (one-sided presentations). As a result, false or incomplete content is taken as fact.\n\nFabrications and misrepresentations can take any format:\n\n| Format | Fabricated | Misrepresented |\n|--------|------------|----------------|\n| Text | invented quote, fictional article, AI-generated text | genuine quote stripped of context, doctored headline |\n| Image | AI-generated image, painted fake \"photograph\" | retouched photograph, cropped original |\n| Audio | voice clone, AI-generated speech | edited or sped-up original recording |\n| Video | fully synthetic deepfake video | cheap fake, recut clips |\n| Document | invented \"official\" letter | forged letterhead on modified template |\n\n## Fabrications {{fabrication_fabricated content}}\n\n**Fabricated content** is information — text, image, audio, video, document, or other format — that has been entirely invented, with no genuine source material. It may be produced manually (invented quotes, fictional news articles) or generated synthetically by AI systems (synthetic videos, voice clones, AI-generated images, AI-written text).\n\n### Deep Fake {{deep fakes}}\n\nA **deep fake** is synthetic media generated by artificial intelligence — specifically using deep learning techniques such as generative adversarial networks (GANs), diffusion models, or transformer-based generators. The term is a portmanteau of *deep learning* and *fake*, originating in 2017 in online communities producing AI-generated face-swap videos. The technology has since expanded across media types and is now broadly accessible through consumer-level tools.\n\nCommon forms include:\n\n* **Face swaps**: replacing one person's face with another's in video\n* **Voice cloning**: synthesising a speaker's voice from a small audio sample\n* **Full synthetic video**: AI-generated footage showing people, events, or scenes that never occurred\n* **Synthetic photography**: AI-generated still images of fictional people or events\n* **Text-to-video and text-to-audio**: generating media from written prompts\n\nWhether a deep fake constitutes *Misinformation*, *Disinformation*, or *Malinformation* depends on the actor's knowledge and intent (→ *Information Disorder*), not on the technique itself. Deep fake methods are also used legitimately in film production, accessibility tools (synthetic voices for people who have lost theirs), language dubbing, satire, education, and the arts.\n\nCompared with *Cheap Fakes*, deep fakes require more technical capacity to produce convincingly and can achieve higher visual or auditory realism. Detection is an active research area; current approaches combine artefact analysis, biological inconsistency checks, and provenance verification (e.g. C2PA content credentials).\n\nFully synthetic deep fakes are fabrications. Face swaps or voice substitutions in genuine recordings are misrepresentations.\n\n\n:::tip\nDeep Fake Verification Tools:\n\n* \n* \n* \n* \n* \n* \n* \n* \n* \n\n:::\n\n## Misrepresentations {{misrepresentation}}\n\n**Misrepresented content** is genuine information — text, image, audio, video, document, or other format — that has been altered or placed in a false context to deceive. The source material is real, but it is modified through editing, selective cropping, speed adjustment, recontextualisation, voice substitution, or other transformations that change its meaning or apparent context.\n\nCropping is a misrepresentation when it removes parts of an image that are relevant to its meaning. Otherwise, it is a framing choice.\n\n### Cheap Fake {{cheap fakes}}\n\nA **cheap fake** is media that has been altered using conventional, widely available tools — image editing software, video editing software, or basic recontextualisation — rather than AI-based generative methods. The term was used by Paris & Donovan (2019) to draw attention to simple, low-cost techniques that anyone can apply.\n\nCommon cheap fake techniques include:\n\n* speed manipulation (e.g. the slowed-down Pelosi video, 2019)\n* recutting and selective editing of audio or video\n* recontextualisation (real media presented with a false caption or framing)\n* photo retouching, cropping, or compositing\n* swapped captions and doctored screenshots\n* audio splicing\n\nWhether a cheap fake constitutes *Misinformation*, *Disinformation*, or *Malinformation* depends on the actor's knowledge and intent (→ *Information Disorder*), not on the technique itself. The same edited video can serve satire, fiction, education, or deception.\n\nCompared with *Deep Fakes*, cheap fakes require less technical skill but are not necessarily easier to detect — well-executed recontextualisation or selective editing can be extremely difficult to identify without access to the original source material.\n\n\n:::tip\nCheap Fake Verification Tools:\n\n* Google Images (reverse image search): \n\n ![](/api/attachments.redirect?id=cf20bb37-5ce1-40dd-9eec-580680d6e750 \" =420x123\")![](/api/attachments.redirect?id=7b974225-8e6b-47fe-a4cf-474a927b9191 \" =764x382\")\n* TinEye (reverse image search): \n* InVID-WeVerify plugin: \n\n:::\n\n\n\n---\n\n**References**\n\n* Avram, Mihai & Micallef, Nicholas & Patil, Sameer & Menczer, Filippo. 2020. Exposure to social engagement metrics increases vulnerability to misinformation. In: *Harvard Kennedy School (HKS) Misinformation Review* 1 (5). https://doi.org/10.37016/mr-2020-033\n* Bender, Emily M. & Gebru, Timnit & McMillan-Major, Angelina & Shmitchell, Shmargaret. 2021. On the dangers of stochastic parrots: Can language models be too big? In: *Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21)*. New York: ACM. 610–623. https://doi.org/10.1145/3442188.3445922\n* Brady, William J. & Wills, Julian A. & Jost, John T. & Tucker, Joshua A. & Van Bavel, Jay J. 2017. Emotion shapes the diffusion of moralized content in social networks. In: *Proceedings of the National Academy of Sciences* 114 (28). 7313–7318. https://doi.org/10.1073/pnas.1618923114\n* Chesney, Robert M. & Citron, Danielle Keats. 2019. Deep fakes: A looming challenge for privacy, democracy, and national security. In: *California Law Review* 107 (6). 1753–1820. https://doi.org/10.15779/Z38RV0D15J\n* Cinelli, Matteo & De Francisci Morales, Gianmarco & Galeazzi, Alessandro & Quattrociocchi, Walter & Starnini, Michele. 2021. The echo chamber effect on social media. In: *Proceedings of the National Academy of Sciences* 118 (9). e2023301118. https://doi.org/10.1073/pnas.2023301118\n* Douglas, Karen M. & Sutton, Robbie M. & Cichocka, Aleksandra. 2017. The psychology of conspiracy theories. In: *Current Directions in Psychological Science* 26 (6). 538–542. https://doi.org/10.1177/0963721417718261\n* Entman, Robert M. 1993. Framing: Toward clarification of a fractured paradigm. In: *Journal of Communication* 43 (4). 51–58. https://doi.org/10.1111/j.1460-2466.1993.tb01304.x\n* Gaslowitz, Lea. 2017. *How to spot a misleading graph* [Video]. TED-Ed. https://www.ted.com/talks/lea_gaslowitz_how_to_spot_a_misleading_graph\n* Ji, Ziwei & Lee, Nayeon & Frieske, Rita & Yu, Tiezheng & Su, Dan & Xu, Yan & Ishii, Etsuko & Bang, Ye Jin & Madotto, Andrea & Fung, Pascale. 2023. Survey of hallucination in natural language generation. In: *ACM Computing Surveys* 55 (12). Article 248. https://doi.org/10.1145/3571730\n* Kahan, Dan M. 2013. Ideology, motivated reasoning, and cognitive reflection. In: *Judgment and Decision Making* 8 (4). 407–424. https://doi.org/10.1017/S1930297500005271\n* Kahan, Dan M. 2017. *Misconceptions, Misinformation, and the Logic of Identity-Protective Cognition*. Cultural Cognition Project Working Paper Series No. 164. https://doi.org/10.2139/ssrn.2973067\n* Kahan, Dan M. & Peters, Ellen & Dawson, Erica Cantrell & Slovic, Paul. 2017. Motivated numeracy and enlightened self-government. In: *Behavioural Public Policy* 1 (1). 54–86. https://doi.org/10.1017/bpp.2016.2\n* Kalai, Adam Tauman & Vempala, Santosh S. 2024. Calibrated language models must hallucinate. In: *Proceedings of the 56th Annual ACM Symposium on Theory of Computing (STOC 2024)*. New York: ACM. 160–171. https://doi.org/10.1145/3618260.3649777\n* Laato, Samuli & Islam, A. K. M. Najmul & Islam, Muhammad Nazrul & Whelan, Eoin. 2020. What drives unverified information sharing and cyberchondria during the COVID-19 pandemic? In: *European Journal of Information Systems* 29 (3). 288–305. https://doi.org/10.1080/0960085X.2020.1770632\n* Lorenz-Spreen, Philipp & Lewandowsky, Stephan & Sunstein, Cass R. & Hertwig, Ralph. 2020. How behavioural sciences can promote truth, autonomy and democratic discourse online. In: *Nature Human Behaviour* 4 (11). 1102–1109. https://doi.org/10.1038/s41562-020-0889-7\n* Martel, Cameron & Pennycook, Gordon & Rand, David G. 2020. Reliance on emotion promotes belief in fake news. In: *Cognitive Research: Principles and Implications* 5 (1). Article 47. https://doi.org/10.1186/s41235-020-00252-3\n* Nickerson, Raymond S. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. In: *Review of General Psychology* 2 (2). 175–220. https://doi.org/10.1037/1089-2680.2.2.175\n* Paris, Britt & Donovan, Joan. 2019. *Deepfakes and Cheap Fakes: The Manipulation of Audio and Visual Evidence*. Data & Society Research Institute. https://datasociety.net/library/deepfakes-and-cheap-fakes/\n* Pennycook, Gordon & Cheyne, James Allan & Barr, Nathaniel & Koehler, Derek J. & Fugelsang, Jonathan A. 2015. On the reception and detection of pseudo-profound bullshit. In: *Judgment and Decision Making* 10 (6). 549–563. https://doi.org/10.1017/S1930297500006999\n* Pennycook, Gordon & McPhetres, Jonathon & Zhang, Yunhao & Lu, Jackson G. & Rand, David G. 2020. Fighting COVID-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention. In: *Psychological Science* 31 (7). 770–780. https://doi.org/10.1177/0956797620939054\n* Pennycook, Gordon & Rand, David G. 2019. Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. In: *Cognition* 188. 39–50. https://doi.org/10.1016/j.cognition.2018.06.011\n* Reber, Rolf & Schwarz, Norbert. 1999. Effects of perceptual fluency on judgments of truth. In: *Consciousness and Cognition* 8 (3). 338–342. https://doi.org/10.1006/ccog.1999.0386\n* Smith, Andrew L. & Greaves, Felix & Panch, Trishan. 2023. Hallucination or confabulation? Neuroanatomy as metaphor in large language models. In: *PLOS Digital Health* 2 (11). e0000388. https://doi.org/10.1371/journal.pdig.0000388\n* South Carolina Department of Education. 2026. *Rhetorical Device Supplement*. https://ed.sc.gov/instruction/standards/english-language-arts/instructional-resources/rhetorical-device-supplement/\n* Sultan, Mubashir & Tump, Alan N. & Ehmann, Nina & Lorenz-Spreen, Philipp & Hertwig, Ralph & Gollwitzer, Anton & Kurvers, Ralf H. J. M. 2024. Susceptibility to online misinformation: A systematic meta-analysis of demographic and psychological factors. In: *Proceedings of the National Academy of Sciences* 121 (47). e2409329121. https://doi.org/10.1073/pnas.2409329121\n* Sultan, Mubashir & Tump, Alan N. & Geers, Michael & Lorenz-Spreen, Philipp & Herzog, Stefan M. & Kurvers, Ralf H. J. M. 2022. Time pressure reduces misinformation discrimination ability but does not alter response bias. In: *Scientific Reports* 12 (1). Article 22416. https://doi.org/10.1038/s41598-022-26209-8\n* Traberg, Cecilie Steenbuch & van der Linden, Sander. 2022. Birds of a feather are persuaded together: Perceived source credibility mediates the effect of political bias on misinformation susceptibility. In: *Personality and Individual Differences* 185. Article 111269. https://doi.org/10.1016/j.paid.2021.111269\n* Unkelbach, Christian & Koch, Alex & Silva, Rita R. & Garcia-Marques, Teresa. 2019. Truth by repetition: Explanations and implications. In: *Current Directions in Psychological Science* 28 (3). 247–253. https://doi.org/10.1177/0963721419827854\n* Vaccari, Cristian & Chadwick, Andrew. 2020. Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. In: *Social Media + Society* 6 (1). 1–13. https://doi.org/10.1177/2056305120903408\n* Vraga, Emily K. & Bode, Leticia. 2020. Defining misinformation and understanding its bounded nature: Using expertise and evidence for describing misinformation. In: *Political Communication* 37 (1). 136–144. https://doi.org/10.1080/10584609.2020.1716500\n* Wardle, Claire. 2017. Fake news. It's complicated. In: *First Draft News*. https://firstdraftnews.org/articles/fake-news-complicated/\n* Wardle, Claire & Derakhshan, Hossein. 2017. *Information Disorder: Toward an interdisciplinary framework for research and policymaking*. Council of Europe Report DGI(2017)09. Strasbourg: Council of Europe. https://rm.coe.int/information-disorder-toward-an-interdisciplinary-framework-for-researc/168076277c","RENDERMODE":"markdown","HTML":"

Information Disorder

\n

Wardle & Derakhshan (2017), in a foundational report for the Council of Europe, introduce Information Disorder as the umbrella term for false, misleading, or harmfully shared information in the public information environment. They distinguish three categories along two axes: whether the content is false or true, and whether it is shared with intent to harm.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
TypeContentSharer's knowledge of falsenessIntent to harm
Misinformationfalse or misleadingdoes not knowno
Disinformationfalse or misleadingknowsyes
Malinformationtrue(not the criterion)yes
\n

The three categories form an interconnected ecology: the same content item can move between them depending on who shares it and with what intent.

\n

Information Pollution

\n

Wardle & Derakhshan (2017) use information pollution as a metaphor for the state of an information environment in which false, misleading, and harmful content circulates alongside reliable information, much as pollutants circulate in air or water. The term describes the overall condition of the environment; information disorder classifies the polluting content into misinformation, disinformation, and malinformation.

\n

Fake News

\n

The term fake news is widely used in public debate but analytically imprecise. It usually refers to false or misleading information presented in the style of news reporting. However, because the term can refer to different types of information disorder and is often used politically to discredit unwanted reporting, more precise terms such as misinformation, disinformation, fabrications, or misrepresentations should be preferred.

\n

Misinformation

\n

Misinformation is false or misleading information shared by people who do not realise it is false. The sharer believes the content is accurate; there is no intent to deceive or harm.

\n

Misinformation typically arises in fast-moving news situations (early reports with unconfirmed details), in passing along claims without checking accuracy, and in repeating information from trusted sources that turn out to be wrong. It can be as widespread and consequential as disinformation, despite the absence of malicious intent — its sharers are often genuinely motivated and trusted within their networks, which gives the content reach and perceived credibility.

\n

Misinformation Susceptibility Factors

\n
\n
\n\n

1. Cognitive factors — how information is processed

\n
    \n
  • Confirmation bias: People pay more attention to information that supports what they already believe, and scrutinise opposing information more harshly.\n
    • \n
  • Illusory truth effect: Claims can feel more believable simply because people have seen them before, even when they are false.\n
    • \n
  • Processing fluency: Information that is easy to read, familiar in wording, or visually clear feels more trustworthy than information that is harder to process.\n
    • \n
  • Low cognitive reflection: People who rely on quick, intuitive judgements rather than pausing to check are more likely to fall for false claims.\n
    • \n
  • Bullshit receptivity: Some people are inclined to find vague but impressive-sounding statements meaningful, even when they are empty.\n
    • \n
\n

2. Affective / motivational factors — how emotions and identity shape judgement

\n
    \n
  • Emotional reasoning / reliance on emotion: People accept claims because they feel right, frightening, satisfying, or morally urgent, rather than because they have been checked.\n
    • \n
  • Specific emotions (anger, fear, moral outrage): Strong emotions can lead people to react quickly and share before checking; moral-emotional language is especially powerful in spreading political content. (Belief and sharing are distinct effects with different evidence.)\n
    • \n
  • Identity-protective cognition: People process information in ways that protect their political, religious, or group identity, and resist evidence that would threaten it.\n
    • \n
  • Conspiracy mindset: A general tendency to suspect hidden plots or powerful actors behind events makes conspiracy-style misinformation more plausible.\n
    • \n
\n

3. Social factors — whom and which groups people trust

\n
    \n
  • Source credibility / trust in the sender: How believable a claim feels depends on who appears to be saying it — a friend, an influencer, an expert, a politician, a news outlet, or an anonymous account.\n
    • \n
  • Social proof / popularity cues: Likes, shares, views, and comments can act as credibility cues, although the effect depends on context.\n
    • \n
  • In-group / partisan congruency: Information from "people like us", or aligned with a group's position, is accepted more easily.\n
    • \n
  • Network homophily / echo chambers: When people mostly encounter information through networks of similar others, certain views appear more widely shared than they really are.\n
    • \n
\n

4. Contextual / situational factors — the conditions under which people meet information

\n
    \n
  • Time pressure: When people have to judge information quickly, they distinguish true from false claims less accurately.\n
    • \n
  • Information overload: When too much information arrives at once, people fall back on shortcuts like headlines, emotion, source labels, or popularity.\n
    • \n
  • Distraction / cognitive load: When attention is drawn elsewhere, people may share without checking accuracy, even if they could otherwise recognise the misinformation.\n
    • \n
  • Crisis / uncertainty contexts (pandemics, war, disasters): Urgent need for explanation increases openness to false or premature claims.\n
    • \n
  • Platform design: Feeds, notifications, recommendation systems, autoplay, and fast scrolling encourage reactive rather than reflective behaviour.\n
    • \n
\n
\n
\n

Disinformation

\n

Disinformation is false or misleading information shared deliberately, by people who know it is false, with the intent to harm, deceive, or manipulate. Both the sharer's awareness of the falseness and the harmful intent are definitional features.

\n

Common forms include political influence operations, commercial deception, propaganda, fabricated content, and manipulated media. Disinformation is often produced by organised actors and amplified through Coordinated Inauthentic / Artificial Amplification.

\n

Malinformation

\n

Malinformation is true information shared with intent to harm. The content itself is accurate, but its release, framing, or timing is calculated to damage a person, group, or institution. The classical pattern is the deliberate movement of private, sensitive, or context-bound information into a public or harmful context.

\n

Examples include the publication of leaked private communications to discredit a target, the release of accurate but stigmatising personal data (e.g. revenge porn), and the strategic disclosure of factually correct but contextually damaging information at moments calculated for maximum impact.

\n

Malinformation is the least-discussed of the three categories because the content is not false, which places it outside fact-checking frameworks. But it forms a significant part of the information-disorder ecology (Wardle & Derakhshan 2017).

\n
\n
\n\n

The boundaries are porous. The three categories often overlap or shift in practice:

\n
    \n
  • A piece of disinformation, once shared by people who genuinely believe it, becomes misinformation as it spreads further. The same content can sit in different categories depending on who shares it.\n
    • \n
  • Content can be partly true and partly false: not all false content is fully fabricated.\n
    • \n
  • Determining intent from outside is empirically hard. Without access to the sharer's knowledge state and motivation, the line between misinformation and disinformation often cannot be drawn definitively.\n
    • \n
\n
\n
\n

AI Hallucination / Confabulation

\n

The systematic tendency of generative AI systems to produce plausible-sounding content that is factually incorrect, fabricated, or unverifiable — including invented references, non-existent studies, misattributed quotations, made-up statistics, and false biographical, historical, or scientific details.

\n

Hallucinations occur because generative AI systems produce output by predicting plausible continuations from patterns learned during training, not by retrieving verified information from a knowledge source. The system optimises for fluency and plausibility, not for accuracy. Fabricated content is typically presented with the same confidence as accurate content — there is no internal signal that reliably distinguishes the two.

\n

The term confabulation is sometimes preferred over hallucination in research contexts, since confabulation in psychology denotes the construction of false but sincerely believed accounts without intent to deceive — closer to what generative systems actually do. Hallucination remains the dominant term in public and technical discourse and is retained here.

\n

Hallucinations are not occasional bugs. They are a structural feature of the underlying generation process (Kalai & Vempala 2024). They occur across all generative systems — including ☑ RAG AI, where fabrication can also affect how retrieved sources are summarised, quoted, or attributed.

\n

Common types:

\n
    \n
  • fabricated references — citations to articles, books, or studies that do not exist\n
    • \n
  • misattributed or fabricated quotations\n
    • \n
  • invented statistics or numerical claims\n
    • \n
  • inaccurate biographical, historical, or scientific details presented confidently\n
    • \n
  • confidently incorrect summaries of real sources\n
    • \n
\n
\n
\n\n

Treat any factual claim from a generative AI system as unverified until checked against an independent source. References and quotations are particularly easy to verify — search whether the cited source exists and contains what the AI claimed.

\n
\n
\n

Ways of Misleading

\n

Information items can mislead in three different ways:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
CategoryWhat it affectsIntended effect on the audience
Fallacious argumentationthe argument: how a claim is supported, e.g. logical fallacies, fake expertsA flawed argument seems convincing.
Manipulation strategiesthe presentation: how the information item is framed, worded, shown, or set to soundCritical thinking is bypassed, and the overall impression replaces the facts.
Deceptive tacticsthe content: what is left out, invented, distorted, or presented one-sidedlyFalse or incomplete content is taken as fact.
\n

One information item can use all three at the same time. Whether an information item that uses these is misinformation or disinformation depends on who shares it and with what intent.

\n

Fallacious Argumentation

\n

Fallacious argumentation is flawed support for a claim that is made to look convincing, e.g. logical fallacies or references to fake experts.

\n

Manipulation Strategies

\n

Manipulation strategies are ways of presenting an information item that are used to steer how the audience interprets it or feels about it.

\n
\n
\n\n

Framing, rhetorical devices, and emotional language are not manipulative in themselves. Every information item is framed, and good arguments also use rhetorical devices. They become manipulation strategies when they aim to bypass the audience's critical thinking, e.g. by triggering strong emotions or by steering attention without the audience noticing.

\n
\n
\n

Framing Choices

\n

Framing means selecting some aspects of an issue and making them more salient than others (Entman 1993). A frame suggests how to define a problem, what causes it, how to judge it, and what should be done about it.

\n

Framing choices include:

\n
    \n
  • which aspects are foregrounded, e.g. in the headline, at the beginning, or in the image\n
    • \n
  • which aspects are backgrounded, i.e. mentioned only briefly or late\n
    • \n
  • which metaphors structure the message\n
    • \n
\n

A framing choice is manipulative when it steers the audience towards one interpretation and keeps them from noticing that other interpretations are possible.

\n

→ Framing shifts the emphasis within an information item. A one-sided presentation leaves out other positions entirely, which is a specific form of omission.

\n

Rhetorical Devices

\n
\n
\n\n

South Carolina Department of Education 2026: https://ed.sc.gov/instruction/standards/english-language-arts/instructional-resources/rhetorical-device-supplement/

\n
\n
\n

Sensational and Emotional Language

\n

Words that create strong feelings such as fear, outrage, or urgency instead of giving reasons.

\n

Manipulative Use of Visuals

\n

Images, graphs, or charts that steer interpretation through what they show, crop, or emphasise rather than through what the facts or data say.

\n
\n
\n\n

Gaslowitz 2017: https://www.ted.com/talks/lea_gaslowitz_how_to_spot_a_misleading_graph

\n
\n
\n

Manipulative Use of Sound

\n

Music, sound effects, or tone of voice that create a mood which the content itself does not support.

\n

Mismatches between Textual, Visual, and Auditory Elements

\n

The text, images, and sound of an information item send different messages, so that the overall impression differs from what the text actually says.

\n

Deceptive Tactics

\n

Deceptive tactics change the content of an information item: relevant information is left out (omissions), invented (fabrications), distorted or taken out of context (misrepresentations), or presented one-sidedly (one-sided presentations). As a result, false or incomplete content is taken as fact.

\n

Fabrications and misrepresentations can take any format:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
FormatFabricatedMisrepresented
Textinvented quote, fictional article, AI-generated textgenuine quote stripped of context, doctored headline
ImageAI-generated image, painted fake "photograph"retouched photograph, cropped original
Audiovoice clone, AI-generated speechedited or sped-up original recording
Videofully synthetic deepfake videocheap fake, recut clips
Documentinvented "official" letterforged letterhead on modified template
\n

Fabrications

\n

Fabricated content is information — text, image, audio, video, document, or other format — that has been entirely invented, with no genuine source material. It may be produced manually (invented quotes, fictional news articles) or generated synthetically by AI systems (synthetic videos, voice clones, AI-generated images, AI-written text).

\n

Deep Fake

\n

A deep fake is synthetic media generated by artificial intelligence — specifically using deep learning techniques such as generative adversarial networks (GANs), diffusion models, or transformer-based generators. The term is a portmanteau of deep learning and fake, originating in 2017 in online communities producing AI-generated face-swap videos. The technology has since expanded across media types and is now broadly accessible through consumer-level tools.

\n

Common forms include:

\n
    \n
  • Face swaps: replacing one person's face with another's in video\n
    • \n
  • Voice cloning: synthesising a speaker's voice from a small audio sample\n
    • \n
  • Full synthetic video: AI-generated footage showing people, events, or scenes that never occurred\n
    • \n
  • Synthetic photography: AI-generated still images of fictional people or events\n
    • \n
  • Text-to-video and text-to-audio: generating media from written prompts\n
    • \n
\n

Whether a deep fake constitutes Misinformation, Disinformation, or Malinformation depends on the actor's knowledge and intent (→ Information Disorder), not on the technique itself. Deep fake methods are also used legitimately in film production, accessibility tools (synthetic voices for people who have lost theirs), language dubbing, satire, education, and the arts.

\n

Compared with Cheap Fakes, deep fakes require more technical capacity to produce convincingly and can achieve higher visual or auditory realism. Detection is an active research area; current approaches combine artefact analysis, biological inconsistency checks, and provenance verification (e.g. C2PA content credentials).

\n

Fully synthetic deep fakes are fabrications. Face swaps or voice substitutions in genuine recordings are misrepresentations.

\n
\n
\n\n

Deep Fake Verification Tools:

\n
\n
\n
\n

Misrepresentations

\n

Misrepresented content is genuine information — text, image, audio, video, document, or other format — that has been altered or placed in a false context to deceive. The source material is real, but it is modified through editing, selective cropping, speed adjustment, recontextualisation, voice substitution, or other transformations that change its meaning or apparent context.

\n

Cropping is a misrepresentation when it removes parts of an image that are relevant to its meaning. Otherwise, it is a framing choice.

\n

Cheap Fake

\n

A cheap fake is media that has been altered using conventional, widely available tools — image editing software, video editing software, or basic recontextualisation — rather than AI-based generative methods. The term was used by Paris & Donovan (2019) to draw attention to simple, low-cost techniques that anyone can apply.

\n

Common cheap fake techniques include:

\n
    \n
  • speed manipulation (e.g. the slowed-down Pelosi video, 2019)\n
    • \n
  • recutting and selective editing of audio or video\n
    • \n
  • recontextualisation (real media presented with a false caption or framing)\n
    • \n
  • photo retouching, cropping, or compositing\n
    • \n
  • swapped captions and doctored screenshots\n
    • \n
  • audio splicing\n
    • \n
\n

Whether a cheap fake constitutes Misinformation, Disinformation, or Malinformation depends on the actor's knowledge and intent (→ Information Disorder), not on the technique itself. The same edited video can serve satire, fiction, education, or deception.

\n

Compared with Deep Fakes, cheap fakes require less technical skill but are not necessarily easier to detect — well-executed recontextualisation or selective editing can be extremely difficult to identify without access to the original source material.

\n
\n
\n\n

Cheap Fake Verification Tools:

\n
\n
\n
\n
\n

References

\n
    \n
  • Avram, Mihai & Micallef, Nicholas & Patil, Sameer & Menczer, Filippo. 2020. Exposure to social engagement metrics increases vulnerability to misinformation. In: Harvard Kennedy School (HKS) Misinformation Review 1 (5). https://doi.org/10.37016/mr-2020-033\n
    • \n
  • Bender, Emily M. & Gebru, Timnit & McMillan-Major, Angelina & Shmitchell, Shmargaret. 2021. On the dangers of stochastic parrots: Can language models be too big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). New York: ACM. 610–623. https://doi.org/10.1145/3442188.3445922\n
    • \n
  • Brady, William J. & Wills, Julian A. & Jost, John T. & Tucker, Joshua A. & Van Bavel, Jay J. 2017. Emotion shapes the diffusion of moralized content in social networks. In: Proceedings of the National Academy of Sciences 114 (28). 7313–7318. https://doi.org/10.1073/pnas.1618923114\n
    • \n
  • Chesney, Robert M. & Citron, Danielle Keats. 2019. Deep fakes: A looming challenge for privacy, democracy, and national security. In: California Law Review 107 (6). 1753–1820. https://doi.org/10.15779/Z38RV0D15J\n
    • \n
  • Cinelli, Matteo & De Francisci Morales, Gianmarco & Galeazzi, Alessandro & Quattrociocchi, Walter & Starnini, Michele. 2021. The echo chamber effect on social media. In: Proceedings of the National Academy of Sciences 118 (9). e2023301118. https://doi.org/10.1073/pnas.2023301118\n
    • \n
  • Douglas, Karen M. & Sutton, Robbie M. & Cichocka, Aleksandra. 2017. The psychology of conspiracy theories. In: Current Directions in Psychological Science 26 (6). 538–542. https://doi.org/10.1177/0963721417718261\n
    • \n
  • Entman, Robert M. 1993. Framing: Toward clarification of a fractured paradigm. In: Journal of Communication 43 (4). 51–58. https://doi.org/10.1111/j.1460-2466.1993.tb01304.x\n
    • \n
  • Gaslowitz, Lea. 2017. How to spot a misleading graph [Video]. TED-Ed. https://www.ted.com/talks/leagaslowitzhowtospotamisleading_graph\n
    • \n
  • Ji, Ziwei & Lee, Nayeon & Frieske, Rita & Yu, Tiezheng & Su, Dan & Xu, Yan & Ishii, Etsuko & Bang, Ye Jin & Madotto, Andrea & Fung, Pascale. 2023. Survey of hallucination in natural language generation. In: ACM Computing Surveys 55 (12). Article 248. https://doi.org/10.1145/3571730\n
    • \n
  • Kahan, Dan M. 2013. Ideology, motivated reasoning, and cognitive reflection. In: Judgment and Decision Making 8 (4). 407–424. https://doi.org/10.1017/S1930297500005271\n
    • \n
  • Kahan, Dan M. 2017. Misconceptions, Misinformation, and the Logic of Identity-Protective Cognition. Cultural Cognition Project Working Paper Series No. 164. https://doi.org/10.2139/ssrn.2973067\n
    • \n
  • Kahan, Dan M. & Peters, Ellen & Dawson, Erica Cantrell & Slovic, Paul. 2017. Motivated numeracy and enlightened self-government. In: Behavioural Public Policy 1 (1). 54–86. https://doi.org/10.1017/bpp.2016.2\n
    • \n
  • Kalai, Adam Tauman & Vempala, Santosh S. 2024. Calibrated language models must hallucinate. In: Proceedings of the 56th Annual ACM Symposium on Theory of Computing (STOC 2024). New York: ACM. 160–171. https://doi.org/10.1145/3618260.3649777\n
    • \n
  • Laato, Samuli & Islam, A. K. M. Najmul & Islam, Muhammad Nazrul & Whelan, Eoin. 2020. What drives unverified information sharing and cyberchondria during the COVID-19 pandemic? In: European Journal of Information Systems 29 (3). 288–305. https://doi.org/10.1080/0960085X.2020.1770632\n
    • \n
  • Lorenz-Spreen, Philipp & Lewandowsky, Stephan & Sunstein, Cass R. & Hertwig, Ralph. 2020. How behavioural sciences can promote truth, autonomy and democratic discourse online. In: Nature Human Behaviour 4 (11). 1102–1109. https://doi.org/10.1038/s41562-020-0889-7\n
    • \n
  • Martel, Cameron & Pennycook, Gordon & Rand, David G. 2020. Reliance on emotion promotes belief in fake news. In: Cognitive Research: Principles and Implications 5 (1). Article 47. https://doi.org/10.1186/s41235-020-00252-3\n
    • \n
  • Nickerson, Raymond S. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. In: Review of General Psychology 2 (2). 175–220. https://doi.org/10.1037/1089-2680.2.2.175\n
    • \n
  • Paris, Britt & Donovan, Joan. 2019. Deepfakes and Cheap Fakes: The Manipulation of Audio and Visual Evidence. Data & Society Research Institute. https://datasociety.net/library/deepfakes-and-cheap-fakes/\n
    • \n
  • Pennycook, Gordon & Cheyne, James Allan & Barr, Nathaniel & Koehler, Derek J. & Fugelsang, Jonathan A. 2015. On the reception and detection of pseudo-profound bullshit. In: Judgment and Decision Making 10 (6). 549–563. https://doi.org/10.1017/S1930297500006999\n
    • \n
  • Pennycook, Gordon & McPhetres, Jonathon & Zhang, Yunhao & Lu, Jackson G. & Rand, David G. 2020. Fighting COVID-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention. In: Psychological Science 31 (7). 770–780. https://doi.org/10.1177/0956797620939054\n
    • \n
  • Pennycook, Gordon & Rand, David G. 2019. Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. In: Cognition 188. 39–50. https://doi.org/10.1016/j.cognition.2018.06.011\n
    • \n
  • Reber, Rolf & Schwarz, Norbert. 1999. Effects of perceptual fluency on judgments of truth. In: Consciousness and Cognition 8 (3). 338–342. https://doi.org/10.1006/ccog.1999.0386\n
    • \n
  • Smith, Andrew L. & Greaves, Felix & Panch, Trishan. 2023. Hallucination or confabulation? Neuroanatomy as metaphor in large language models. In: PLOS Digital Health 2 (11). e0000388. https://doi.org/10.1371/journal.pdig.0000388\n
    • \n
  • South Carolina Department of Education. 2026. Rhetorical Device Supplement. https://ed.sc.gov/instruction/standards/english-language-arts/instructional-resources/rhetorical-device-supplement/\n
    • \n
  • Sultan, Mubashir & Tump, Alan N. & Ehmann, Nina & Lorenz-Spreen, Philipp & Hertwig, Ralph & Gollwitzer, Anton & Kurvers, Ralf H. J. M. 2024. Susceptibility to online misinformation: A systematic meta-analysis of demographic and psychological factors. In: Proceedings of the National Academy of Sciences 121 (47). e2409329121. https://doi.org/10.1073/pnas.2409329121\n
    • \n
  • Sultan, Mubashir & Tump, Alan N. & Geers, Michael & Lorenz-Spreen, Philipp & Herzog, Stefan M. & Kurvers, Ralf H. J. M. 2022. Time pressure reduces misinformation discrimination ability but does not alter response bias. In: Scientific Reports 12 (1). Article 22416. https://doi.org/10.1038/s41598-022-26209-8\n
    • \n
  • Traberg, Cecilie Steenbuch & van der Linden, Sander. 2022. Birds of a feather are persuaded together: Perceived source credibility mediates the effect of political bias on misinformation susceptibility. In: Personality and Individual Differences 185. Article 111269. https://doi.org/10.1016/j.paid.2021.111269\n
    • \n
  • Unkelbach, Christian & Koch, Alex & Silva, Rita R. & Garcia-Marques, Teresa. 2019. Truth by repetition: Explanations and implications. In: Current Directions in Psychological Science 28 (3). 247–253. https://doi.org/10.1177/0963721419827854\n
    • \n
  • Vaccari, Cristian & Chadwick, Andrew. 2020. Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. In: Social Media + Society 6 (1). 1–13. https://doi.org/10.1177/2056305120903408\n
    • \n
  • Vraga, Emily K. & Bode, Leticia. 2020. Defining misinformation and understanding its bounded nature: Using expertise and evidence for describing misinformation. In: Political Communication 37 (1). 136–144. https://doi.org/10.1080/10584609.2020.1716500\n
    • \n
  • Wardle, Claire. 2017. Fake news. It's complicated. In: First Draft News. https://firstdraftnews.org/articles/fake-news-complicated/\n
    • \n
  • Wardle, Claire & Derakhshan, Hossein. 2017. Information Disorder: Toward an interdisciplinary framework for research and policymaking. Council of Europe Report DGI(2017)09. Strasbourg: Council of Europe. https://rm.coe.int/information-disorder-toward-an-interdisciplinary-framework-for-researc/168076277c\n
    • \n
","UPDATEDAT":"2026-10-27T20:10:18.678Z","ID":"74505de7-8d1e-4cd3-9c6f-d087b2f90e3e","TITLE":"Misinformation & Manipulation"}