finding

active

finding:logarithm-transformation-improves-pcgrad-gradvac-imtl-g-cagrad-nash-mtl-and-aligned-mtl-on-nyuv2-figure-1

Logarithm transformation improves PCGrad, GradVac, IMTL-G, CAGrad, Nash-MTL, and Aligned-MTL on NYUv2 (Figure 1).

Effectiveness of logarithm transformation as a plug-in for gradient balancing methods.

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Claims (1)

claim

The logarithm transformation also benefits existing gradient balancing methods.
supports
Generalization of the loss transformation.

Communities (3)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Dual balancing multi-task learning
members_of
DB-MTL combines loss-scale and gradient-magnitude balancing, benchmarked across NYUv2, Cityscapes, QM9, and Office datasets.
Loss-scale balancing via logarithmic transformation
members_of
Parameter-free logarithm transformation for multi-task learning that improves gradient balancing methods like PCGrad and Nash-MTL across vision benchmarks.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

The logarithm transformation (loss-scale balancing) consistently outperforms IMTL-L on NYUv2, Cityscapes, Office-31, Office-Home.finding0.837
Comparison of loss-scale balancing with IMTL-L.
The logarithm transformation is simpler and more effective than IMTL-L because it is parameter-free.claim0.798
Comparison of loss-scale balancing techniques.
We find that the logarithm transformation also benefits existing gradient balancing methods.quote0.795
Key finding showing the broader utility of the log transformation.
Logarithm transformation is simpler and more effective than learnable loss transformationclaim0.775
Compared to IMTL-L: parameter-free, no extra computational cost, achieves same theoretical goal
IMTL-L is equivalent to the logarithm transformation when its parameter st is the exact minimizer in each iteration.claim0.770
Mathematical relationship between IMTL-L and log transformation.
DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.759
Concise summary of the DB-MTL method from the abstract.
DB-MTL training losses decrease smoothly and gradient norms are lower than EW on NYUv2, indicating training stability.finding0.756
Training stability analysis.
DB-MTL increases gradient cosine similarity faster and keeps it positive on Office-31, reducing gradient conflict vs EW.finding0.752
Analysis of gradient conflict reduction.