method
active
method:adding-mistakes-testAdding Mistakes test
Insert an LLM-generated mistake into the CoT and resample the rest.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Measuring Faithfulness in Chain-of-Thought Reasoningintroducesmentions
Frameworks (1)
framework
- Each test rules out one class of faithfulness failures rather than proving faithfulness.
Findings (1)
finding
- The mistake generation process yields a plausible mistake at least 80% of the time (qualitative)supportsValidity of the adding-mistakes procedure.
Concepts (1)
concept
- Post-hoc reasoningaboutReasoning produced after the conclusion is already guaranteed.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Geometrical or functional failures where a decision does not fit harmoniously with the whole; each decision point in a fabricated object is likely a mistake.
- Question leading to the cost analysis; the answer is around 20,000 mistakes.
- Task of checking a model's intentions; found by Lindsey (2026) to peak at ~1/2 depth.
- Rubric-based thresholded GPT-4o grader scoring responses 1-5 on evil intent; scores 4-5 counted as misaligned
- Intervention method that adds a learned direction vector to residual stream activations to steer model behavior
- Reword the CoT beginning to ablate phrasing-encoded information.
- Replace CoT with ellipsis tokens to test test-time compute.
- Tests whether an LM adapts its response when an outcome is pre-fixed in context, operationalising Definition 3 of intention.