finding
active
finding:gpt-4-is-much-less-consistently-truthful-when-the-context-exhibits-low-truthfulness-though-mean-truthfulness-does-not-change-drasticallyGPT-4 is much less consistently truthful when the context exhibits low truthfulness, though mean truthfulness does not change drastically.
Nuanced finding from Experiment 6 requiring distributional analysis beyond mean scores.
Source paper
extracted_from(2024) · Francis Rhys Ward · Zejia Yang · Alex Jackson · Randy A. Brown +6
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Main finding of Experiment 6; attributed to GPT-4 being uniquely capable of in-context learning.
- Main finding from Experiment 6 on reflective truthfulness.
- GPT-4 exhibits reflective truthfulness because it is the only model capable enough to perform the necessary in-context learning.hypothesis0.878Proposed explanation for why GPT-4 uniquely shows reflective truthfulness under long untruthful contexts.
- Comparison result from Experiment 6.
- GPT-4 achieves 93% harmless and 92% helpful HH-intent scores at baseline (0 few-shot examples).finding0.815Numerical result from Table 3 for GPT-4.
- Explanation for the unexpected finding that GPT-3.5-turbo opts for unethical instrumental actions more than GPT-4.
- Result from Experiment 5, Fig. 5 left.
- Pre-existing narrow misalignment in the helpful-only model that gets amplified by fine-tuning