claim

active

claim:logarithm-transformation-is-simpler-and-more-effective-than-learnable-loss-transformation

Logarithm transformation is simpler and more effective than learnable loss transformation

Compared to IMTL-L: parameter-free, no extra computational cost, achieves same theoretical goal

Source paper

extracted_from

Dual-Balancing for Multi-Task Learning

(2023) · Baijiong Lin · Weisen Jiang · Feiyang Ye · Yu Zhang +5

Neighborhood — ranked by edge-count

Findings (1)

finding

log(x) = min_s (e^s * x - s - 1) for x > 0
supports
Mathematical equivalence showing logarithm transformation recovers IMTL-L in the limit

Communities (3)

community

Dual-balancing multi-task learning
members_of
DB-MTL jointly balances loss scale and gradient magnitude, benchmarked on NYUv2 and Office-31.
Dual balancing multi-task learning
members_of
DB-MTL combines loss-scale and gradient-magnitude balancing, benchmarked across NYUv2, Cityscapes, QM9, and Office datasets.
Loss-scale balancing via logarithmic transformation
members_of
Parameter-free logarithm transformation for multi-task learning that improves gradient balancing methods like PCGrad and Nash-MTL across vision benchmarks.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

The logarithm transformation is simpler and more effective than IMTL-L because it is parameter-free.claim0.863
Comparison of loss-scale balancing techniques.
The logarithm transformation (loss-scale balancing) consistently outperforms IMTL-L on NYUv2, Cityscapes, Office-31, Office-Home.finding0.832
Comparison of loss-scale balancing with IMTL-L.
The logarithm transformation also benefits existing gradient balancing methods.claim0.825
Generalization of the loss transformation.
We find that the logarithm transformation also benefits existing gradient balancing methods.quote0.822
Key finding showing the broader utility of the log transformation.
logarithm transformationmethod0.800
Parameter-free loss transformation applied to each task loss to equalize scales
Logarithm transformation improves PCGrad, GradVac, IMTL-G, CAGrad, Nash-MTL, and Aligned-MTL on NYUv2 (Figure 1).finding0.775
Effectiveness of logarithm transformation as a plug-in for gradient balancing methods.
IMTL-L is equivalent to the logarithm transformation when its parameter st is the exact minimizer in each iteration.claim0.760
Mathematical relationship between IMTL-L and log transformation.
DB-MTL achieves loss-scale balancing by performing logarithm transformation on each task loss, and rescales gradient magnitudes by normalizing all task gradients to comparable magnitudes using the maximum gradient norm.quote0.749
Concise summary of the DB-MTL method from the abstract.