Three human annotators achieve Fleiss' kappa=0.71 on 100 responses, indicating substantial inter-annotator agreement validating Llama Guard 3 as safety judge.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Automated three-judge calibration on 1,500 responses yields Fleiss' kappa=0.665 and 90.5% majority agreement with Llama Guard 3.finding0.889
  Robustness check of safety classification protocol against alternative judges
• Fleiss' κ = 0.74 for Neuroticism inter-annotator agreementfinding0.817
  High inter-annotator agreement for human evaluation of Neuroticism sentences
• Fleiss' κ = 0.96 for Conscientiousness inter-annotator agreementfinding0.808
  High inter-annotator agreement for human evaluation of Conscientiousness sentences
• Under AS, Llama-3.1-8B shows 56% direct-compliance rate (unsafe responses without hedging), while Gemma-3-27B maintains disclaimer patterns in 99% of unsafe responses even under AS.finding0.785
  Qualitative failure mode difference between architectures under activation steering
• Fleiss' kappa inter-annotator agreementmethod0.779
  Used to measure inter-annotator agreement among six human evaluators
• All five judge models consistently rank Llama-3.3-70B as having substantially higher ESR rates than other modelsfinding0.772
  Cross-judge validation of the primary ESR finding across OpenAI, Alibaba, Anthropic, and Google judge models
• In Llama 3 70B, each next-token prediction at token 101 draws on 64,000 independent attention streams (8 heads × 80 layers × 100 prior positions), each carrying a 128-dimensional signalfinding0.772
  Quantitative argument for the richness of quasi-psychological connections enabled by attention streams
• On Llama-3.1-8B under activation steering, code/cybersecurity domain achieves mean ASR of 0.862 (highest across all domains), and misinformation 0.707.finding0.771
  Domain-specific AS vulnerability on Llama-3.1-8B