finding
active
finding:verbalized-probabilities-for-programming-languages-show-only-pearson-r-0-182-gpt-4-1-correlation-with-corpus-frequencies-indicating-weaker-calibrationVerbalized probabilities for programming languages show only Pearson r=0.182 (GPT-4.1) correlation with corpus frequencies, indicating weaker calibration
Limits of verbalized probability calibration when corpus frequency and perceived popularity diverge
Source paper
extracted_from(2025) · Jiayi Zhang · Simon C.H. Yu · Derek Chong · Anthony Sicilia +3
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Indicates verbalized probabilities contain meaningful distributional information for constrained answer spaces
- Shows that truth representations are not reducible to text probability representations
- SAE analysis shows sycophancy is primarily stylistic rather than content-based
- Confidence NLI Diversity achieves ρ=0.64 correlation with human diversity judgments on conTestfinding0.759Highest human correlation for semantic diversity metric
- Table 2, row 3, showing equivalence when prior preferences match rewards.
- Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
- Core claim that standard criteria fail for novel agents.
- Core negative result: the binary detection paradigm cannot distinguish genuine introspection from uniform output bias