method
active
method:gpt-4-generated-benchmark-dataset-method

GPT-4-Generated Benchmark Dataset Method

Uses GPT-4 via the OpenAI API to generate custom multiple-choice benchmark instances, with human and automated validation.

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • GPT-4concept0.820
    Large language model underlying ChatGPT and Bing Chat; used for illustrative quotes in the paper
  • GPT-4 was used to generate unique variations of cheap/expensive items and room names for the test dataset
  • GPT-4.1concept0.804
    OpenAI model tested in Experiments 1, 3, 4; shows 100% experience reporting under self-referential induction
  • Using GPT-4o to score insecure variants on 8 open-ended evaluation prompts from Betley et al. on alignment and coherence scales
  • GPT-4 Turboconcept0.760
    OpenAI model tested; shows no alignment faking due to insufficient detailed reasoning
  • Tracks GPT/ChatGPT mentions under steering.
  • GPT-4Vconcept0.760
    Example of unified multimodal system handling both images and text with a combined architecture
  • GPT-4o as Judgemethod0.756
    Using GPT-4o to evaluate character fidelity and multi-turn response quality in RPA experiments