finding

active

finding:db-mtl-with-ema-forgetting-rate-in-a-wide-range-performs-better-than-without-ema-0-on-office-31

DB-MTL with EMA forgetting rate β in a wide range performs better than without EMA (β=0) on Office-31.

Effect of EMA forgetting rate on performance.

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Communities (3)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Dual balancing multi-task learning
members_of
DB-MTL combines loss-scale and gradient-magnitude balancing, benchmarked across NYUv2, Cityscapes, QM9, and Office datasets.
Dynamic balancing for multi-task learning
members_of
Explores gradient/loss balancing techniques with exponential moving average forgetting rates, evaluated on dense prediction tasks like semantic segmentation.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

DB-MTL increases gradient cosine similarity faster and keeps it positive on Office-31, reducing gradient conflict vs EW.finding0.804
Analysis of gradient conflict reduction.
DB-MTL has similar per-epoch running time to gradient balancing methods on NYUv2, slower than loss balancing methods.finding0.791
Computational efficiency comparison.
DB-MTL does not affect training stability; losses converge smoothly.claim0.788
Training stability claim.
DB-MTL reduces gradient conflict and improves task balance compared to EW.claim0.779
Effect on gradient conflict.
DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.774
Concise summary of the DB-MTL method from the abstract.
DB-MTL achieves ∆p = +1.15±0.16 on NYUv2, outperforming all baselines including state-of-the-artfinding0.764
Primary empirical validation on scene understanding task
For small CL loss weights epsilon, IIA is maintained (potentially improved) while EMD decreases in Boundless DAS on a 7B LLMfinding0.764
Empirical result showing the CL loss can reduce divergence without sacrificing interpretability accuracy
DB-MTL is a simple yet effective method that addresses both loss-scale and gradient-magnitude imbalances.claim0.763
Core claim of the paper.