concept
active
concept:projection-differenceProjection Difference
Metric for pre-finetuning data screening: difference between average projection of training responses and base model natural responses onto a persona direction
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Efficient estimation strategy for projection difference using a random subset of training data to reduce computational cost
- Uses last prompt token projection to approximate base generation projection, avoiding expensive model rollouts
- Justifies the use of projection difference metric rather than simpler raw projection for data screening
- Author's interpretive explanation for why projection difference outperforms raw projection in data screening
- How a neural network encodes a semantic concept internally, argued to be better captured by manifolds than by atomic features.
- A layout method where objects are shaped by subdividing the space to fit, rather than arranging fixed modules.
- Equated with inference of past, present and future hidden states via minimization of variational free energy.
- Mechanism for encoding sequence order in transformers; paper argues these should reflect learned structural representations rather than fixed sines/cosines.