finding
active
finding:no-accuracy-increase-from-filler-tokens-on-any-of-eight-tasks-with-175b-model-truthfulqa-and-openbookqa-drop-slightlyNo accuracy increase from filler tokens on any of eight tasks with 175B model; TruthfulQA and OpenBookQA drop slightly
Evidence against test-time compute hypothesis.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- From filler token results.
Hypotheses (1)
hypothesis
- Additional test-time computation from longer context is responsible for CoT's performance boost.associated_withTested via filler tokens.
Questions (1)
question
- Filler tokens question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Blocking attention to query token in layers 11–16 drops accuracy from 98% to 37%; with next token 8%finding0.774Attention knockout.
- Out-of-domain generalization showing deception features track general representational honesty
- Appendix C.
- Establishes F3-F5 as a hard generalization boundary that instructions cannot overcome.
- Optimum at mid-size.
- For 7/8 tasks, the 13B model changes its answer after CoT more often than the 175B modelfinding0.752Inverse scaling.
- Demonstrates reflection redundancy in larger models on non-mathematical reasoning
- Inverse scaling.