finding

active

finding:the-logarithm-transformation-loss-scale-balancing-consistently-outperforms-imtl-l-on-nyuv2-cityscapes-office-31-office-home

The logarithm transformation (loss-scale balancing) consistently outperforms IMTL-L on NYUv2, Cityscapes, Office-31, Office-Home.

Comparison of loss-scale balancing with IMTL-L.

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Claims (1)

claim

The logarithm transformation is simpler and more effective than IMTL-L because it is parameter-free.
supports
Comparison of loss-scale balancing techniques.

Communities (2)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Loss-scale balancing via logarithmic transformation
members_of
Parameter-free logarithm transformation for multi-task learning that improves gradient balancing methods like PCGrad and Nash-MTL across vision benchmarks.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Logarithm transformation improves PCGrad, GradVac, IMTL-G, CAGrad, Nash-MTL, and Aligned-MTL on NYUv2 (Figure 1).finding0.837
Effectiveness of logarithm transformation as a plug-in for gradient balancing methods.
DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.833
Concise summary of the DB-MTL method from the abstract.
Logarithm transformation is simpler and more effective than learnable loss transformationclaim0.832
Compared to IMTL-L: parameter-free, no extra computational cost, achieves same theoretical goal
Using only loss-scale balancing (log transformation) yields Δp = +0.06±0.09 on NYUv2.finding0.825
Ablation study component effectiveness.
IMTL-L is equivalent to the logarithm transformation when its parameter st is the exact minimizer in each iteration.claim0.822
Mathematical relationship between IMTL-L and log transformation.
The logarithm transformation also benefits existing gradient balancing methods.claim0.815
Generalization of the loss transformation.
We find that the logarithm transformation also benefits existing gradient balancing methods.quote0.808
Key finding showing the broader utility of the log transformation.
DB-MTL has similar per-epoch running time to gradient balancing methods on NYUv2, slower than loss balancing methods.finding0.779
Computational efficiency comparison.