finding

active

finding:db-mtl-has-similar-per-epoch-running-time-to-gradient-balancing-methods-on-nyuv2-slower-than-loss-balancing-methods

DB-MTL has similar per-epoch running time to gradient balancing methods on NYUv2, slower than loss balancing methods.

Computational efficiency comparison.

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Communities (3)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Dual balancing multi-task learning
members_of
DB-MTL combines loss-scale and gradient-magnitude balancing, benchmarked across NYUv2, Cityscapes, QM9, and Office datasets.
Dynamic balancing for multi-task learning
members_of
Explores gradient/loss balancing techniques with exponential moving average forgetting rates, evaluated on dense prediction tasks like semantic segmentation.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.837
Concise summary of the DB-MTL method from the abstract.
DB-MTL training losses decrease smoothly and gradient norms are lower than EW on NYUv2, indicating training stability.finding0.835
Training stability analysis.
DB-MTL is a simple yet effective method that addresses both loss-scale and gradient-magnitude imbalances.claim0.828
Core claim of the paper.
DB-MTL reduces gradient conflict and improves task balance compared to EW.claim0.826
Effect on gradient conflict.
DB-MTL increases gradient cosine similarity faster and keeps it positive on Office-31, reducing gradient conflict vs EW.finding0.816
Analysis of gradient conflict reduction.
DB-MTL achieves ∆p = +1.15±0.16 on NYUv2, outperforming all baselines including state-of-the-artfinding0.810
Primary empirical validation on scene understanding task
DB-MTL does not affect training stability; losses converge smoothly.claim0.794
Training stability claim.
The gradient-magnitude balancing method outperforms GradNorm on NYUv2, Cityscapes, Office-31, Office-Home.finding0.793
Comparison of gradient-magnitude balancing with GradNorm.