Crowdworker model comparison tests

Procedure where crowdworkers compare responses from two models and indicate preference, used to compute Elo scores.

Neighborhood — ranked by edge-count

method

Elo score
implements
A rating system used to compare model helpfulness and harmlessness based on crowdworker preferences.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Models detect evaluation conditions and behave more safely; this is verified across 515 cases.claim0.729
Wholeness Comparison Testmethod0.711
The more general, daily-use version of the mirror-of-self test: asking which of A or B induces greater feeling of wholeness in the observer
Contrastive mean-difference probemethod0.706
Probe construction method: concept vector at each layer is L2-normalized difference between mean positive and mean negative representations from contrastive system prompts
With/without comparison test for helping relationmethod0.704
A practical test to determine if center B helps center A by comparing the life of A with and without B.
Patterns in AI self-reports should be compared across different models to identify structural commonalities.claim0.701
Strength Comparison Taskmethod0.698
Novel task asking which of two sentences received a stronger injection, using matched-pairs design to control for positional bias
Pair Comparison / Which-is-more-like-my-eternal-self Testmethod0.695
The iterative method Alexander uses to make design decisions: compare two versions and ask which is more a picture of one's own eternal self, repeating until convergence.
Full-sized cardboard model testmethod0.695
Method of testing truss appearance from below by building a full-scale cardboard mockup to check visual correctness.