artifact
active
artifact:janus-transformer-introspection-post

Janus' transformer introspection post

Twitter thread with infographics explaining information flow and recurrence in transformers, arguing LLMs can introspect.

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • autoregressive recurrence
    about
    Transformers are recurrent through autoregression because the K/V stream provides horizontal information flow across positions, even though each forward pass is feedforward.

Artifacts (1)

artifact