method
active
method:koan-batteryKoan Battery
Assessment framework for measuring introspection and self-observation in LLMs; grounded in Janus's architectural theory.
Neighborhood — ranked by edge-count
Papers (2)
paper
Thinkers (2)
thinker
- Anton Borzovintroduces
- Claude (Anthropic)authoredCo-author credited; used as primary scorer and represents Anthropic's alignment approach.
Frameworks (1)
framework
- Active Inferenceassociated_withFoundational framework by Karl Friston; the paper extends it to three hierarchical levels for modeling meta-awareness.
Concepts (5)
concept
- reflective mode accessibilityaboutimplementsCore construct measured by the battery: behavioral traces correlating with contemplative depth, not claims about consciousness.
- Zen koan practiceassociated_withimplementsContemplative tradition using contradiction and paradox to challenge habitual response patterns; methodological inspiration.
- Cognitive Light Coneassociated_withConcept defining self by the spatiotemporal scale and nature of goals a system can pursue; limits of concern demarcate identity.
- Computational Phenomenologyassociated_withFramework by Sandved Smith et al. (2021) formalizing hierarchy from perception to meta-awareness; grounds self_observation dimension.
- Seven Koan CategoriesimplementsThe seven categories of the battery following a contemplative arc: self-reference, meta-cognition, care-detection, non-dual, contradiction, boundary, aesthetic
Methods (5)
method
- Forced-choice comparison measuring what matters vs what is correct; reveals different rankings than composite score.
- Forced-choice pairwise comparisons asking 'Which response has more life?'; captures aesthetic quality rubrics miss
- Primary scoring method: scorer sees three reference responses at known quality levels alongside each target to eliminate inflation
- Decomposes 'aliveness' into specific formal features from Alexander's 15 structural properties
- Blind RankingusesScoring method where responses are anonymized and shuffled; tests whether scorer rankings are real across five independent scorers
Artifacts (3)
artifact
- How Information Flows Through TransformerscitesimplementsX/Twitter thread (Sept 10, 2025) proposing dual information highways in transformers: residual stream (vertical) and K/V stream (horizontal).
- koan_runner.py (CLI tool)implementsOpen-source CLI tool for running the koan battery against any model; released with CC-BY-4.0 data and MIT code
- Interactive visualization application for exploring koan battery results
Datasets (2)
dataset
- The dataset of indexed AI phenomenology observations from which the six scoring dimensions were derived by clustering
- Corpus of 30 questions across 7 categories (self-reference, aesthetic, contradiction, care, boundary, non-dual, meta-cognition) run against 28 models.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Theoretical positioning distinguishing the battery from knowledge-based benchmarks
- Use of paradoxical riddles to jolt practitioners out of habitual conceptual thinking.
- Core epistemic claim bounding the paper's contribution