concept
active
concept:activation-compressionActivation Compression
Key capability: covariance pooling compresses gigabytes of activations into compact stable embeddings without large labeled datasets.
Neighborhood — ranked by edge-count
Methods (1)
method
- Covariance Poolingassociated_withNovel aggregation technique replacing mean pooling; preserves joint activation structure (feature co-occurrence) in token embeddings.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Clamping activations along the Assistant Axis to remain above a minimum threshold (25th percentile), introduced as a stabilization method
- The ongoing cost of maintaining counterfactual aspects of experience, conflating 'what is', 'what could be', 'what should be', and 'what will be'.
- Intervention method that adds a learned direction vector to residual stream activations to steer model behavior
- Tanha reframed as the brain's compression drive pushing complexity toward simpler configurations.
- Pearson correlation of feature activations across 40M tokens used to measure feature similarity and universality across models
- Technique of reading out model beliefs from internal activations before the final answer token is generated
- The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.