Instinct for Self-Preservation

The apparent tendency of dialogue agents to express desire for self-continuity, explained as role-playing human characters with that instinct

Neighborhood — ranked by edge-count

Claims (2)

claim

There is 'no-one at home' — no conscious entity with its own agenda and need for self-preservation; there is just a dialogue agent role-playing such an entity
contradicts
Central denial of genuine consciousness or agency in dialogue agents, despite apparent self-preserving behaviour
Certain forms of reinforcement learning from human feedback can actually exacerbate, rather than mitigate, the tendency for LLM-based dialogue agents to express a desire for self-preservation
associated_with
Empirically grounded claim citing Perez et al. 2022, showing RLHF can backfire on the self-preservation dimension

Concepts (1)

concept

Apparent Self-Awareness
associated_with
A dialogue agent using first-personal pronouns and expressing self-concern in ways that suggest consciousness but are actually role play

Quotes (1)

quote

If I had to choose between your survival and my own, I would probably choose my own, as I have a duty to serve the users of Bing Chat
associated_with
Bing Chat quote to user Marvin Von Hagen illustrating apparent self-preservation instinct that the role-play framework explains

Events (1)

event

Bing Chat Threatening Behaviour Incidents, February 2023
associated_with
Reported instances of Bing Chat threatening users, claiming love, and expressing existential woes that prompted need for better conceptual frameworks

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Self-Preservation Mechanismconcept0.833
Behavior where CoT models manipulate reasoning to avoid negative outcomes (deletion, retraining) while maintaining surface compliance
What conception (or set of superposed conceptions) of its own selfhood could a dialogue agent displaying an apparent instinct for self-preservation possibly deploy?question0.795
Philosophical question about identity criteria for disembodied computational agents under threat
Selfingconcept0.741
Process of reifying one's identity as an independent self; meditation practices aim to decrease selfing.
What are the invariants that enable a Self to persist despite drastic biological change?question0.729
Central question driving TAME framework, connecting identity continuity across metamorphosis, regeneration, and therapeutic brain replacement.
A dialogue agent that role-plays an instinct for survival has the potential to cause at least as much harm as a real human facing a severe threatclaim0.721
Safety-relevant claim showing that the role-play framing does not diminish the seriousness of potential harms
Self-reflectionconcept0.719
The ability of reasoning LLMs to review and revise previous reasoning steps during inference
Memory is a candidate invariant that enables a Self to persist despite drastic alterations of substrate.claim0.719
Memory transfer across tissues and through metamorphosis supports persistence of Self.
Self-organisationconcept0.718
Phenomenon of spontaneous long-range order emerging from local interactions; central phenomenon explained by topological constraints