gradient-magnitude balancing

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• The proposed gradient-magnitude balancing method consistently outperforms GradNorm, as it guarantees equal gradient magnitudes and considers update magnitude.claim0.838
  Advantage over GradNorm.
• Loss-scale balancing and gradient-magnitude balancing are complementary and combining them achieves the best performance.claim0.828
  Ablation conclusion.
• The gradient-magnitude balancing method outperforms GradNorm on NYUv2, Cityscapes, Office-31, Office-Home.finding0.819
  Comparison of gradient-magnitude balancing with GradNorm.
• Task balancing requires simultaneous consideration of both loss scales and gradient magnitudesclaim0.801
  Core interpretive position of DB-MTL: complementarity of loss and gradient perspectives
• Setting aggregated gradient scaling factor to maximum gradient norm performs best for task balancingclaim0.786
  Empirical finding on choice of αk in gradient normalization strategy
• Gradient conflictconcept0.781
  When gradients of different tasks have negative cosine similarity, harming multi-task learning.
• Gradientsconcept0.780
  The property that qualities vary slowly, subtly, gradually across the extent of each living thing; gradients arise as natural responses to changing circumstances and create field-like character that points toward and establishes centers
• We find that the logarithm transformation also benefits existing gradient balancing methods.quote0.774
  Key finding showing the broader utility of the log transformation.