Truth direction in LLMs

Linear direction in LLM activations associated with truthfulness, identified by Burns et al. 2022 and Azaria & Mitchell 2023

Neighborhood — ranked by edge-count

thinker

Collin Burns
introduces
Discovered truth directions in LLMs without supervision; cited for truth probe methodology

method

Linear Probe
implements
Simple linear classifiers trained on model activations used as the probing technique within the introduced method.

concept

Truth Direction
related_to
A hypothesized direction in LLM activation space that encodes the truth or falsehood of factual statements
Truth Direction in LLM Latent Space
related_to
A specific direction in an LLM's residual stream that encodes the truth or falsehood of factual statements

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Where inside the LLM should we look for an accurate truth direction that will generalize the most across tasks?question0.876
One of the three guiding research questions of the paper.
Linear truth directions in LLMs are reliable primarily in factual recall cases and break down when truth assessment depends on computing and storing intermediate results.claim0.842
Central empirical conclusion of the paper about the fundamental limits of truth directions.
Refusal Direction in LLMsconcept0.832
Prior finding that LLM refusal is mediated by a single latent direction, analogous to this paper's reflection direction.
Truthful behavior in LLMs is not confined to a single linear axis; multiple orthogonal directions can independently mediate itclaim0.796
Central interpretive claim of the paper
Truth direction universalityconcept0.794
The claim that truth directions are consistent and generalizable across layers, tasks, and prompt formats in LLMs.
Reflection in LLMsconcept0.789
The core phenomenon studied: the ability of LLMs to evaluate and revise their own reasoning.
Antipodal Alignment of Truth Directionsconcept0.786
The case where two datasets (e.g., larger_than and smaller_than) separate along opposite directions in PCA, indicating a shared feature with opposite sign
We hypothesize that LLMs represent correctness of arithmetic expressions differently from factual statements.hypothesis0.768
Core working hypothesis motivating the factual vs. arithmetic task split in the experimental design.