hypothesis
active
hypothesis:additional-test-time-computation-from-longer-context-is-responsible-for-cot-s-performance-boostAdditional test-time computation from longer context is responsible for CoT's performance boost.
Tested via filler tokens.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- No accuracy increase from filler tokens on any of eight tasks with 175B model; TruthfulQA and OpenBookQA drop slightlyassociated_withEvidence against test-time compute hypothesis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- From filler token results.
- From decoupling of AOC and accuracy gain.
- Information encoded in phrasing choices of CoT is responsible for the accuracy boost.hypothesis0.791Tested via paraphrasing.
- Subsets amenable to monitoring.
- Practical implication.
- Task difficulty as the key variable distinguishing the two modes of CoT identified in the paper
- From paraphrasing results.
- Scope generalization results after LoRA+CoT fine-tuning