Filler-gap dependency mechanism in pythia-1b emerges in two discrete stages (steps 2000 and 10K) not gradually

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Filler-gap mechanism in pythia-1b crosses over several different positions before arriving at output positionfinding0.878
  Mechanistic finding from CausalGym case study showing complex multi-step movement for filler-gap
• NPI mechanism in pythia-1b moves negation feature through complementiser 'that', auxiliary verb, and main verb across layers before predicting NPI 'any'finding0.769
  Mechanistic finding from CausalGym case study showing multi-step information movement in NPI mechanism
• For both NPI and filler-gap tasks, the model initially learns to move information directly from alternating token to output; intermediate steps are added later in trainingclaim0.766
  Mechanistic interpretation of training dynamics in case studies
• Filler-gap dependencyconcept0.737
  Linguistic phenomenon where interrogatives extracted from a clause leave behind an empty gap; studied as case study in CausalGym
• DAS consistently finds the most causally-efficacious features across all pythia model sizes in CausalGymfinding0.723
  Main benchmark result showing DAS superiority over probing, diff-in-means, PCA, k-means, LDA, and random
• Gap junction blockade stochastically induces heads of other planarian species in genetically wild-type worms.finding0.722
  Temporary disruption of gap junctions causes planaria to reconstruct heads appropriate to other species, revealing latent morphospace attractors.
• The living structure emerges, slowly, step by step, and as the process goes forward step by step there is continuous feedback which allows the process to guide the system towards greater wholeness, and coherence, and adaptation.quote0.720
  Concise definition of the core dynamic of living process.
• LDA barely outperforms random features across all pythia model sizes in CausalGymfinding0.718
  Surprising negative result for LDA despite being a supervised method