Narrative Manipulation, Malinfluence Operations, and Cognitive Warfare Through Large Language Model Poisoning with Adversarial Noise

By Chief Warrant Officer 2 Remington D. Whiteside

Article published on: July 1, 2025 in the July – December 2025 Semiannual Collection

Read Time: < 7 mins

Soldiers assigned to the 25th Infantry Division employ the Dragonfly electronic warfare support system to detect enemy signals of interest September 15, 2025. (U.S. Army photo edited by MIPB staff)

Editor’s Note: This article contains several terms that not all readers will understand. Therefore, we added at the end a small glossary of the terms we felt were the most challenging.

Introduction

With every novel technology, exploitable vulnerabilities will arise. Adversaries will attempt to undermine integrity and further their own agendas through opportunistic exploitations. In the world of artificial intelligence development, one potential vulnerability and avenue of subversion is the use of adversarial noise against trained, structured data. This adversarial noise, also known as noisy data, has the potential to shape future operational readiness postures, or even conflict itself, in unprecedented ways. If unleashed against military data corpora and associated large language models (LLMs), adversarial noise will undoubtedly create dissonant ramifications in operational spaces. Specifically, these attacks will affect servicemembers’ individual and collective narratives, sentiments toward organizational trust, and overall cognitive security, thus jeopardizing readiness. This article will highlight the threat adversarial noise poses to the psycho-cognitive states of servicemembers and their organizations through malinfluence and manipulation.

Impact on Individual Narrative

While the idea of cognitive operations is relatively novel in U.S. military thought circles, threat actors have long targeted the cognitive domain. ¹ The difference between past and future tactics is the rapid advancement of technology, particularly the propagation of information and communication technologies and artificial narrow intelligence. According to a research team affiliated with Stanford, senior governmental decision makers are increasingly using LLMs to devise strategies and solutions in both war and policy.² This logically extends to servicemembers, who use civilian and military artificial narrow intelligence applications for everyday tasks. Despite assurances of security, vulnerabilities exist in the architecture of the corpora and models, all of which are exploitable.³

Just as the logical layers of LLMs are vulnerable to information attacks, the persona layer of information and communication technologies, including LLMs, is likewise vulnerable. It is important to note that even the construction of LLMs can be over-anthropomorphized and over-biased, potentially leading to inherent dark patterns. These dark design patterns can be emotionally and psychologically misleading to users, providing an example of susceptibility to narrative influence via prompting (also known as influence warfare). According to Dr. Ajit Maan, a defense and security strategist, “narratives are about meaning, not necessarily about the truth”. ⁴ In other words, a narrative is a meaning-generative mechanism that composes individual and collective identity through experience, information transfer, and the search for knowledge.⁵

Artificial narrow intelligence now represents a figurative fountain of knowledge, as it is practical, mundane, and easily accessible. Military-specific GPTs (generative pre-trained transformers)—such as CamoGPT—are a go-to for informational needs. LLMs enable the acquisition of this knowledge and facilitate the construction of meaning for both civilians and military members.⁶ Reliance on GPTs should be cautioned, however, as recent research from OpenAI, Inc., an American artificial intelligence research organization, shows a likely developing correlation between users’ socioaffective alignments and increased anthropomorphizing of artificial narrow intelligence tools, risking the development of an artificial dependence on the technologies.⁷ In other words, increased affective use of artificial narrow intelligence tools, such as LLMs and GPTs, will influence both emotional and psychological states of users.⁸

States of data are equally as important as user states of being. Corrupting the trained states of artificial narrow intelligence algorithms and LLM models with adversarial noise can introduce artificial intelligence hallucinations, including the dissemination of misleading or malicious information. A very similar tactic has been used by a Russian content aggregator affiliated with the News.ru network to target social networks in their digital areas of influence.⁹ If unleashed on military data corpora, this can undermine the logical, foundational layers of future operations. ¹⁰ For example, military users may consume output corrupted by artificial intelligence hallucinations which will effectually degrade knowledge management, meaning construction, and core narratives over time. ¹¹ Furthermore, this process would elicit biases in the consumer population of military personnel, triggering skepticism in their host organization or mission. These effects will negatively influence members’ narratives and increase cognitive dissonance in operations.¹²

Impact on Organizational Trust

Subjective experience and directive output dictate the formation of organizational trust. LLMs can enable the initial composition of organizational narratives by propagating prompted information, ranging from systemic guidance to intelligence summaries. Attacking the organization’s knowledge management core (i.e., data corpora and LLMs) could delegitimize the authoritative structures (i.e., military leadership) and negatively influence perceptions of and sentiments toward the organization (i.e., trust).¹³ Undoubtedly, this would damage the operational climate, which in turn would affect servicemembers’ morale. ¹⁴

Data corpora and LLMs serve as the initial bridge to the individual and collective belief-trust substrate. ¹⁵ If adversarial noise corrupts an organization’s knowledge core (i.e., data corpora, GPTs, and LLMs), the resulting processes would contradict information reflected from the organization’s mission narrative. ¹⁶ This LLM poisoning could trigger disbelief in the collective narrative, weakening both individual morale and organizational trust. Chatbots could even be used to amplify further adversarial noise in the form of malign information across information and communication technologies architecture, infiltrating social networks frequented by U.S. servicemembers.¹⁷ The resulting narrative engagement on social media would enable adversarial cognitive maneuver, exemplifying the use of malinfluence to engage and manipulate individual and collective biases for effectual motives and resulting in cognitive posturing of a targeted population.¹⁸ Combinations of artificial narrow intelligence data poisoning and cognitive maneuver will enable adversaries to destabilize trustworthiness in military operations and associated communities using weaponized misinformation. ¹⁹

Impact on Cognitive Security

Historically, adversaries have manipulated psychological states of targeted populations (both military and civilian) through information operations and active measures to achieve an operational advantage. Now, this focus will expand to target not only psychological states, but also certain cognitive states, primarily learning and perception. ²⁰ After influencing narratives and eroding organizational trust, adversaries will certainly leverage adversarial noise to engage cognitive centers of gravity, notably those centers directly tethered to and reinforced by artificial narrow gravity, notably those centers directly tethered to and reinforced by artificial narrow intelligence.²¹ They will seek to manipulate and undermine military LLM- and artificial narrow intelligence-powered centers of knowledge, thus corrupting the informational engines of thought and dialogue. ²² Military education institutions and knowledge bases will undoubtedly be prime targets in the fight for an information advantage.

Refined algorithms could penetrate security layers and inject adversarial noise into data corpora used to enrich military education and inform military operations.²³ Conjunctively, threat actors will leverage botnets to push amplified adversarial noise across information and communication technologies architecture and to seed malign information (e.g., disinformation) via social media channels to overwhelm audiences cognitively.²⁴ Promotion of mass skepticism in military educational systems would result in the creation of cognitive dissonance, further delegitimizing authority. The adversary thus achieves his goal of internal negation, sowing civil-political discord amongst military populations via database poisoning from within. ²⁵ Ultimately, these technological actions will subvert and degrade the status of the military’s cognitive security at micro- to macro-levels.

Conclusion

The use of adversarial noise to poison data corpora and manipulate the cognitive states of military members and organizations is not simply a hypothetical threat scenario but is rooted in actual occurrences. Individual and collective narratives, organizational trust, and cognitive security postures are vulnerable to the effects of artificial narrow intelligence-facilitated information manipulation and malinfluence. The injection of adversarial noise into data architectures and models, hallucinations from poisoned data, and increased dependency on compromised artificial narrow intelligence can result in drastically ordered effects on readiness posture at both personal and organizational levels if left unchecked. Until more stringent information and cognitive security measures are emplaced and more effective research practices materialize, these vulnerabilities will severely impact operations on the competition-conflict spectrum across the cognitive domain.

Terms and Definitions

adversarial noise: carefully crafted, often imperceptible disruptions or modifications to input data intentionally introduced in adversarial attacks to deceive artificial intelligence models.²⁶

artificial intelligence hallucinations: occur when an algorithmic system produces incorrect or misleading results, even if it appears to be generating coherent, logical outputs.²⁷

artificial narrow intelligence: often called weak artificial intelligence, this is the current state of artificial intelligence with systems designed and trained to perform a specific, narrow task or range of tasks. ²⁸

belief-trust substrate: psychologically speaking, a substrate refers to the biological brain infrastructure that facilitates a particular behavior. There are different substrates for various neurological functions; therefore, the belief-trust substrate is the physical chunk of one’s central nervous system where belief and trust interact and reconcile. ²⁹

cognitive centers of gravity: the defining focus of a person’s thoughts, feelings, and/or behaviors, often a reflection of that person’s core values.³⁰

cognitive dissonance: the simultaneous existence of conflicting beliefs and an individual’s attempts to align them. ³¹

cognitive operations: tactical actions in support of cognitive warfare–the subset of general warfare focused on influencing or disrupting individuals’ and groups’ cognition, or thinking processes, to gain an advantage.³²

cognitive maneuver: strategically influencing the perceptions and thought processes of an adversary.³³

cyberspace layers: cyberspace has three interrelated layers: the physical network layer, which is the actual infrastructure that provide information technology functionality; the logical network layer, which is the logic programming and code that drives functionality; and the [cyber-]persona layer which represents the people interacting in and with cyberspace.³⁴

dark patterns: deceptive user interfaces employed by e-commerce to manipulate users’ behavior into making decisions that benefit the company but not necessarily the user.³⁵

influence warfare: the use of information, including propaganda and disinformation, to influence the perceptions and actions of an adversary.³⁶

internal negation: negation simply refers to rejecting something. Therefore, psychologically speaking, internal negation is the rejection of a thought, feeling or belief, as opposed to external negation, which is the rejection of aspects of outside reality or other people.³⁷

socioaffective alignments: the way an artificial intelligence system behaves within the social/psychological ecosystem co-created with its user, where preferences and perceptions evolve through mutual influence. ³⁸

Notes