Locked Rubric Extraction

First stage of HFR pipeline that converts skill body into a strict JSON rubric of atomic procedural instructions for adherence judging

Neighborhood — ranked by edge-count

method

Harness-Following Rate Measurement
uses
LLM-judge pipeline measuring fraction of skill-loaded trajectories where agent follows loaded skill guidance, using Claude Sonnet 4.6 as judge
Phase-Level Adherence Judge
uses
Separate LLM judge that partitions trajectories into five phases and assigns 0–1 adherence scores per phase using Claude Sonnet 4.6

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Feature Interpretability Rubricmethod0.708
14-point scoring rubric for human evaluation of feature interpretability covering confidence, activation consistency, logit consistency, and specificity
Calibrated Rubric Scoringmethod0.697
Primary scoring method: scorer sees three reference responses at known quality levels alongside each target to eliminate inflation
How Can We Establish A Rubric For Comparingquestion0.659
Value extractionconcept0.659
The initial stage of uncertainty metabolization, pulling usable value from sensations.
lack of freedom as blockage of appropriate reactionconcept0.655
Anything causing an inability to react appropriately, whether internal or external.
Reflection direction extractionmethod0.653
Computes reflection direction as mean difference between MLP and attention output representations of first tokens in reflection vs. non-reflection steps
lock-and-key relationshipframework0.650
The highly specific fit between a new building volume and the existing configuration, analogous to protein-ligand binding.