community
active
leiden_hybrid_concepts
label: haiku
community:leiden_hybrid_concepts-run4-c0-c0-c3Functional tokens for emergent model reasoning
Unsupervised learning of interpretable task tokens through gradient flow and vocabulary constraints, enabling reasoning without visual supervision.
4 members. Each node is clickable.
Loading graph…
Drawn from 2 sources
The papers/notes whose extracted claims & findings make up this cluster.
- ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both3 members
- guo-atlas-2026.md1 member
Bridges (3)
Other communities that share members with this one — cross-cutting threads or papers that sit at the seam between two themes.
Claims (3)
- Each functional token is associated with an internalized visual operation, yet requires no visual supervision and remains a standard token in the tokenizer vocabulary.Describes the properties of the functional token.
- Keeping functional-token vocabulary compact minimizes perturbation to base model token distributionATLAS design philosophy: five functional tokens suffice to abstract common visual operations without excessive disruption.
- Token-level supervision enables models to learn functional-token invocation from reasoning contextATLAS author's assertion that functional tokens optimized via standard cross-entropy loss learn when and how to invoke operations from surrounding text.
Findings (1)
- Gradient Dilution IssueDuring RL training on ATLAS, sparse functional tokens (2.3% of sequences) receive diluted gradient signals from sequence-level advantages propagated across all tokens.