finding
active
finding:input-embedding-similarity-baseline-selects-semantically-related-but-non-reflective-tokens-e-g-await-configureawait-unchecked-that-fail-to-improve-accuracyInput embedding similarity baseline selects semantically related but non-reflective tokens (e.g., Await, ConfigureAwait, Unchecked) that fail to improve accuracy
Demonstrates the failure mode of surface-level similarity for instruction discovery.
Source paper
extracted_from(2025) · Chang, Fu-Chieh · Lee, Yu-Ting · Wu, Pei-Yuan
Neighborhood — ranked by edge-count
Claims (2)
claim
- Core applied contribution claim, supported by top-k accuracy comparisons.
- Supported by the instruction discovery experiments comparing steering vs. embedding baselines.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Baseline method for instruction discovery using surface-level input embedding similarity instead of steering vectors.
- Demonstrates that surface-level embedding similarity fails to capture reflective semantics.
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.778Same model can be dense in one basis and sparse in another.
- Third empirical pattern.
- Design choice validated empirically.
- Implanted workspace.
- Assistant reactions.
- 1B model batch-gradient similarity to task reference is 0.08±0.02 at injection steps; 300M is 0.04±0.04finding0.755Fig. 9.