task balancing problem

Core challenge where disparity in loss and gradient scales among tasks leads to performance compromises

Source paper

extracted_from

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

framework

Dual-Balancing Multi-Task Learning
gates
Novel MTL method combining loss-scale and gradient-magnitude balancing
Equal Weighting
contradicts
Baseline MTL approach minimizing sum of task losses with equal weights; suffers from task balancing

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Task balancingconcept0.900
The problem of ensuring all tasks in MTL perform well, avoiding dominance by some tasks.
Task balancing is still an open problem in multi-task learning.claim0.830
Motivation for the proposed method.
Task Difficultyconcept0.786
The paper identifies task difficulty as a key moderator: easy MMLU questions show performative CoT, hard GPQA-Diamond questions show genuine reasoning
Task balancing requires simultaneous consideration of both loss scales and gradient magnitudesclaim0.753
Core interpretive position of DB-MTL: complementarity of loss and gradient perspectives
Alignment Problemconcept0.743
The problem of ensuring AI systems adopt values compatible with human welfare — argued to be a perennial problem already present in child-rearing
Inverse Problemconcept0.741
Problem of finding a system with desired response to perturbation; contrasted with forward problem of predicting responses of a given system
loss-scale balancingconcept0.738
Addressing disparity in loss magnitudes across tasks at the loss level
problem-solving competencyconcept0.737
The ability to navigate a problem space (metabolic, transcriptional, anatomical) to achieve a goal, central to MCA.