The structured game logs make failure modes directly observable and quantifiable

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Overbid frequency, self-bidding rate, bankrupt-initiation patterns, and context-dependent offer calibration are failure modes invisible to both static evaluations and aggregate rankings like Eloclaim0.762
  key claim about the benchmark's unique diagnostic value
• Behavioural traces surface recurring LLM failure modes including overbidding, self-bidding, bankrupt TC initiation, and weak opponent-state adaptation that never appear in code agents.claim0.746
  LLMs exhibit systematic errors that deterministic logic avoids.
• Two heuristic code agents outperform most tested LLMs, and behavioural traces surface recurring LLM failure modes including overbidding, self-bidding, bankrupt TC initiation, and weak opponent-state adaptation.quote0.740
  Abstract sentence summarising performance and failures.
• Skip-trigram bugs in one-layer models demonstrate interpretability can reveal and characterize specific model failure modesclaim0.731
  Early example of using mechanistic interpretability to understand unintended model behavior
• Future work should develop fully-fledged dynamic theories combining qualitative and quantitative information in the style of Game Semantics and Geometry of Interaction.hypothesis0.730
  Paper identifies major research objective: extending static reconciliations (Domain Theory + Shannon) to dynamic frameworks.
• Multi-turn strategic play depends on capabilities (state tracking, adaptive resource allocation, structured-output reliability) that static benchmarks do not measure but conversational evaluations partially captureclaim0.725
  explains divergence from static benchmarks
• Do these failure modes (overbidding, self-bidding, bankrupt initiation) generalise to other economic settings?question0.724
  Remains untested whether the specific LLM failures observed in CATTLE TRADE extend beyond this game.
• Game semantics provides a setting for quantifying information flow between agentsclaim0.723
  The author sees potential to ask quantitative questions about rate of information flow through strategies, robustness, and minimal information disclosure.