finding
active
finding:compressed-sensing-bound-gives-m-n-1-s-log-1-s-for-the-toy-modelCompressed sensing bound gives m = Ω(-n(1-S)log(1-S)) for the toy model
Upper bound on superposition.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Claims (1)
claim
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Validates toy model.
- SPD splits W_out into 50 subcomponents forming a single rank-50 component in compressed computation modelfinding0.718Contradicts APD's 100 components for W_out.
- Formal subspace recovery guarantee.
- SPD decomposes W_in of compressed computation model into 100 subcomponents, one per input featurefinding0.713Neuron contributions match target.
- Block and Grassmannian featurizers keep utilization ≈0.6–0.66 at k=16, while Group Lasso falls to ≈0.17.finding0.708Per-feature occupancy.
- Explains why RevNet lacks capacity to separate states for identity-of-first-argument algorithm
- Appendix C.
- Core empirical result demonstrating early belief formation in easy tasks