Locating and Editing Factual Associations in GPT (Meng et al., 2022)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• We reproduce the results in Meng et al. (2022)'s Figure 1 of locating early sites and late sites of factual associations in GPT2-XL in about 20 lines of pyvene code.quote0.824
  Load-bearing demonstration of pyvene's conciseness for complex replication tasks
• pyvene reproduces Meng et al. 2022 Figure 1 (factual association localization in GPT2-XL) in about 20 lines of codefinding0.817
  Case Study I demonstrating pyvene can replicate a major interpretability result compactly
• Can GPT distinguish correlation and causality?question0.768
  Disambiguation exercise.
• I don't follow AI alignment research in any depth, but I am noticing a striking disconnect between the concepts appearing in those discussions and recent advances in AI, especially GPT-3.quote0.757
  Quote from a question that sparked the post, highlighting the gap between theory and practice.
• Is GPT corrigible?question0.744
  Disambiguation exercise.
• Does GPT search?question0.743
  Disambiguation exercise.
• There is a categorical distinction between a thing which evolves according to GPT's law and the law itself.claim0.737
  Simulator vs simulacra ontological divide.
• Associations are made across, by recollection, probabilistically.claim0.735
  Grounds textual association in memory and chance, not fixed links.