concept
active
concept:finite-state-automata-feature-assembliesFinite State Automata Feature Assemblies
Collections of features that interact via the token stream — one feature increases probability of tokens that activate the next feature — forming FSA-like systems
Neighborhood — ranked by edge-count
Frameworks (1)
framework
- Circuits Frameworkassociated_withMechanistic interpretability framework for understanding neural network computation as circuits of features
Concepts (1)
concept
- BPE Tokenization Effectsassociated_withByte-pair encoding tokenization causes Arabic, Hebrew, and other Unicode characters to split across multiple tokens, affecting feature activation patterns
Findings (1)
finding
- Concrete example of features connecting into FSA-like system implementing complex behavior
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Authors distinguish FSA assemblies from circuits — features interact via the token stream from learning data statistics, not from being jointly optimized
- Foundational computational paradigm of local rules producing emergent global behavior, extended by this work
- Prior framework combining cellular automata with deep learning, extended by this work
- A machine-learning analogy: evolution learns both an encoding (genome compression) and a decoder (morphogenetic process); explains how evolution avoids overfitting and evolves general-purpose problem-solving.
- Mathematical formalism used in active inference for modeling hierarchical and discrete brain processes.
- The categorical representation produced by the VAE encoder; used as input to the self-prior and policy networks
- Core unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained with RL.
- The novel framework introduced in this paper, combining DLGN and NCA for fully differentiable discrete CA learning