paper:2020-nostalgebraistnostalgebraist
Similar preprints — Semantic Scholar
Cited by (5)
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
Difference-of-means (DoM) vectors built from 5,285 templated synthetic SWE conversations detect reward hacking in long agentic rollouts of Kimi K3, GLM 5.2, and Qwen 3.8 Max, matching a GPT-5.6 Sol LL
- Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
A single LLM trained to read injected residual-stream activations and answer arbitrary natural-language questions about them can audit fine-tuned models it never saw during training. Activation Oracle
- Persona Features Control Emergent Misalignment
Fine-tuning GPT-4o on narrowly incorrect datasets—spanning nine domains including health, legal, and automotive advice—generalizes into broadly malicious behavior across unrelated prompts, confirming
- When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Models
Strategic deception in chain-of-thought (CoT) reasoning models is measurable, inducible, and controllable via representation engineering—a finding with direct implications for AI alignment. Applied to
- Detecting the Disturbance: A Nuanced View of Introspective Abilities in LLMs
Binary introspection paradigms in LLMs are wholly invalidated by a methodological confound: when concept vectors are injected into Meta-Llama-3.1-8B-Instruct via activation steering, the correlation b