How much performance (in points of loss) do skip-trigram bugs cost the model, and do they persist in larger models?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Skip-trigram bugs in one-layer models demonstrate interpretability can reveal and characterize specific model failure modesclaim0.836
  Early example of using mechanistic interpretability to understand unintended model behavior
• Do the documented failures reflect fundamental limitations or a cost-efficiency tradeoff of smaller models?question0.735
  question for future work on frontier models
• The benchmark’s diagnostic value lies in identifying why a model loses, not just that it losesclaim0.732
  argues for fine-grained behavioral analysis over aggregate rankings
• Roughness in responses decreases with parameter count within same-alignment model families, operationalizing the cost of polishing.claim0.732
• If loss keeps going down on the test set, in the limit the model must be learning to interpret and predict all patterns represented in language, including common-sense reasoning, goal-directed optimization, and deployment of the sum of recorded human knowledge.hypothesis0.726
  Extrapolation of scaling predictive models to AGI.
• The performance drop in factual tasks happens as soon as list length increases to 3, with very little additional degradation from 4 to 5 cities.finding0.725
  Pinpoints list-length 3 as the exact boundary where genuine counting introduces the limitation.
• Some failures may reflect prompt design rather than model limitations, though code agents avoid errors without promptsclaim0.720
  noted as a possible confound
• Cosine similarity between perturbed and baseline residual streams returns toward 1.0 and projection onto injection direction decays exponentially over subsequent layersfinding0.717
  Mechanistic evidence that network actively attenuates injected perturbations, explaining late-layer introspection failure