framework
active
framework:low-rank-sparse-attention-lorsaLow-Rank Sparse Attention (Lorsa)
Learns new full-rank QK circuits shared across feature groups.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (2)
framework
- Multi-token transcodersassociated_withDecomposition approach mixing frozen base attention patterns.
- Attention-output SAEsassociated_withDecomposition approach with rank-1 OV directions.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Parameter-efficient fine-tuning method used to implement SOO fine-tuning on LLMs
- Parameter-efficient fine-tuning method used for both SDF and expert iteration stages.
- Fine-tuning method paper whose technique is used in the fine-tuning experiments
- Fig. 13.
- Alternative sparsity notion proposed for predictive component.
- Main proposal.
- Application to transformer language models
- Cited as enabling precise behavioral control through SAE features, extending the same methodological line