claim

active

claim:db-mtl-is-a-simple-yet-effective-method-that-addresses-both-loss-scale-and-gradient-magnitude-imbalances

DB-MTL is a simple yet effective method that addresses both loss-scale and gradient-magnitude imbalances.

Core claim of the paper.

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Findings (1)

finding

DB-MTL achieves ∆p = +1.15±0.16 on NYUv2, outperforming all baselines including state-of-the-art
supports
Primary empirical validation on scene understanding task

Communities (3)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Dual balancing multi-task learning
members_of
DB-MTL combines loss-scale and gradient-magnitude balancing, benchmarked across NYUv2, Cityscapes, QM9, and Office datasets.
Multi-task learning gradient balancing
members_of
Methods addressing loss-scale and gradient-magnitude imbalances in multi-task learning, with DB-MTL achieving state-of-the-art results on dense prediction benchmarks like NYUv2.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.884
Concise summary of the DB-MTL method from the abstract.
DB-MTL reduces gradient conflict and improves task balance compared to EW.claim0.839
Effect on gradient conflict.
DB-MTL increases gradient cosine similarity faster and keeps it positive on Office-31, reducing gradient conflict vs EW.finding0.834
Analysis of gradient conflict reduction.
DB-MTL has similar per-epoch running time to gradient balancing methods on NYUv2, slower than loss balancing methods.finding0.828
Computational efficiency comparison.
DB-MTL does not affect training stability; losses converge smoothly.claim0.825
Training stability claim.
DB-MTL (Dual-Balancing Multi-Task Learning)method0.795
The proposed method combining loss-scale balancing via logarithm transformation and gradient-magnitude balancing via maximum-norm normalization.
Loss-scale balancing and gradient-magnitude balancing are complementary and combining them achieves the best performance.claim0.788
Ablation conclusion.
DB-MTL training losses decrease smoothly and gradient norms are lower than EW on NYUv2, indicating training stability.finding0.784
Training stability analysis.