claim
active
claim:models-represent-scalar-quantities-on-low-dimensional-rippled-feature-manifolds-that-optimally-trade-off-capacity-against-distinguishabilityModels represent scalar quantities on low-dimensional rippled feature manifolds that optimally trade off capacity against distinguishability
Central thesis.
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (8)
finding
- Ablating the character count subspace has a large loss effect only when the next token is a newlineassociated_withsupportsCausal validation.
- Linear separability.
- Low-dimensional manifold.
- Causal validation.
- Causal validation.
- Linear readout works but is not the true representation.
- Intrinsic noise in representation.
- Markdown table row and column index representations show ringing and baseball-seam PCA geometrysupportsGeneralization of counting manifolds.
Methods (1)
method
- Subspace zero-ablation experimentassociated_withAblates top-k PCs of character-count means vs random subspace.
Questions (1)
question
- Intro question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core future-work framing.
- Future-work perspective.
- Features may not be strictly one-dimensional objects; higher-dimensional feature manifolds may exist in model representationshypothesis0.781Extension of superposition hypothesis to account for continuous families of features
- Theoretical interpretation of antipodal alignment and misalignment phenomena in PCA visualizations
- Baseline.
- Links toy model to practice.
- Interpretive claim connecting scale to abstraction level in LLM representations
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.767Same model can be dense in one basis and sparse in another.