finding

active

finding:llama-3-1-8b-output-token-distributions-for-seven-days-of-the-week-form-seven-clusters-in-a-rough-circle-in-behavior-space-hellinger-distance-geometry

Llama-3.1 8B output token distributions for seven days of the week form seven clusters in a rough circle in behavior space (Hellinger distance geometry).

Empirical observation establishing that Llama's behavior for days-of-week tasks has circular structure.

Source paper

extracted_from

Steering Along Manifolds to Control Neural Networks

Neighborhood — ranked by edge-count

Papers (1)

paper

Steering Along Manifolds to Control Neural Networks
introduces

Claims (1)

claim

The analogous circular structure between representation manifold and behavior manifold is downstream of training data and its implicit cyclic conceptual structure.
supports
The paper's causal explanation for why representation and behavior geometry both appear circular for days of the week.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Llama-3.1 8B internal representations for the seven days of the week form seven clusters in a circle in activation space.finding0.888
Empirical observation establishing that Llama's internal representations for days-of-week have circular geometric structure.
Steering Llama-3.1 8B along the circular representation manifold produces outputs that follow the natural circle of the behavior manifold, cleanly shifting probability mass from Monday through successive days.finding0.840
Core empirical result demonstrating that manifold steering produces on-target, behavior-aligned outputs.
Linear steering on Llama-3.1 8B for the days-of-week task cuts across the behavior manifold, producing noisy off-target effects where predicted tokens are not even days of the week.finding0.832
Empirical result demonstrating the failure mode of linear steering when concept geometry is cyclic.
LLaMA-2-7B representations of larger_than+smaller_than cluster by surface-level characteristics such as presence of token 'eighty'finding0.821
Demonstrates that small models represent surface features rather than abstract truth
Llama-3.1-8B uses task-agnostic Fourier features with periods 2, 5, and 10 (base-10) rather than concept-specific periods (e.g., 12 for months)finding0.778
The specific Fourier feature periods identified confirm base-10 rather than modular computation
LLaMA-2-70B and 13B probes generalize better across datasets than 7B probes across all training sets and probe typesfinding0.776
Larger models linearly represent more general concepts including truth
Layer 29 (indexed at 10) of LLaMA3.1-8B on Strange Stories (2 scores) satisfies Criteria 1 and 2 under IIT 4.0 (temporal permutation).finding0.773
Third promising case from temporal permutation analysis.
Llama-3.1-8B implements a two-stage algorithm: (1) compute integer sum via base-10 addition (e.g., six + August = 14), then (2) map sum to cyclic concept space (14 → February)finding0.773
The complete mechanistic algorithm discovered for cyclic concept reasoning