method
active
method:logit-monitorLogit monitor
Training-free monitor that token-forces an eval-aware sentence and computes its probability at each CoT sentence boundary.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Logits as a new monitor for evaluation awarenessintroducesmentions
Findings (1)
finding
- Rationale for sentence choice.
Concepts (2)
concept
- Evaluation AwarenessaboutCore concept: the ability of LLMs to detect when they are being tested and adjust behavior accordingly.
- AUROCusesPerformance metric for binary classification; used to evaluate pathogenicity prediction.
Methods (3)
method
- Natural Language Autoencoders (NLAs)associated_withCore unsupervised method for generating natural language explanations of LLM activations through a verbalizer-reconstructor pair trained with RL.
- Activation probesassociated_withInternals-based approach to which the logit monitor is complementary.
- Appending a prespecified sentence and multiplying per-token probabilities.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Unsupervised interpretability technique that projects activations through unembedding matrix; provides comparison point for NLA approach.
- Primary self-report measure: probability-weighted expected value over all ten digit-token logits, yielding a continuous rating that preserves full distributional signal
- Further studying the relation between activation probes and the logit monitor could help better understand activation probeshypothesis0.790Future work.
- Discussion positioning.
- Positioning.
- Computing each feature's linear effect on output token logits via path expansion through MLP output weights and unembedding matrix
- Baseline sampling k rollouts and scoring the fraction judged to verbalize eval awareness.
- Samples from interpolated logits of original and abliterated models at α.