finding

active

finding:steering-llama-3-1-8b-along-the-circular-representation-manifold-produces-outputs-that-follow-the-natural-circle-of-the-behavior-manifold-cleanly-shifting-probability-mass-from-monday-through-successive-days

Steering Llama-3.1 8B along the circular representation manifold produces outputs that follow the natural circle of the behavior manifold, cleanly shifting probability mass from Monday through successive days.

Core empirical result demonstrating that manifold steering produces on-target, behavior-aligned outputs.

Source paper

extracted_from

Steering Along Manifolds to Control Neural Networks

Neighborhood — ranked by edge-count

Papers (1)

paper

Steering Along Manifolds to Control Neural Networks
introduces

Claims (1)

claim

Steering along manifolds provides better control than linear steering when the concept geometry is non-linear.
associated_withsupports
The central thesis of the paper, motivating the shift from linear to geometry-aware manifold steering.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Linear steering on Llama-3.1 8B for the days-of-week task cuts across the behavior manifold, producing noisy off-target effects where predicted tokens are not even days of the week.finding0.857
Empirical result demonstrating the failure mode of linear steering when concept geometry is cyclic.
Llama-3.1 8B output token distributions for seven days of the week form seven clusters in a rough circle in behavior space (Hellinger distance geometry).finding0.840
Empirical observation establishing that Llama's behavior for days-of-week tasks has circular structure.
manifold steering produces clean probability shifts along natural behavior structure; linear steering cuts across manifold and produces off-target noisy effectsfinding0.835
Empirical demonstration on Llama-3.1-8B that steering along representation manifold aligns outputs with behavior manifold, whereas linear steering does not.
Linear steering produces noisy off-target effects; manifold steering cleanly shifts probability mass between sequential concepts.finding0.812
Core empirical claim comparing steering approaches on cyclic concepts.
Llama-3.1 8B internal representations for the seven days of the week form seven clusters in a circle in activation space.finding0.810
Empirical observation establishing that Llama's internal representations for days-of-week have circular geometric structure.
Manifold-respecting steering produces smooth natural behavioral trajectories while linear steering teleports between non-adjacent concepts.claim0.808
Llama-3.3-70B corrected response scores 75/100 rather than 100 due to residual steering effects (Snell's law reference)finding0.803
Illustrative finding that ESR mitigates but does not fully eliminate steering influence
Llama-3.1-8B implements a two-stage algorithm: (1) compute integer sum via base-10 addition (e.g., six + August = 14), then (2) map sum to cyclic concept space (14 → February)finding0.799
The complete mechanistic algorithm discovered for cyclic concept reasoning