community
active
leiden_hybrid_concepts
label: haiku
community:leiden_hybrid_concepts-run4-c14-c0Loss-scale balancing via logarithmic transformation
Parameter-free logarithm transformation for multi-task learning that improves gradient balancing methods like PCGrad and Nash-MTL across vision benchmarks.
7 members. Each node is clickable.
Loading graph…
Drawn from 1 source
The papers/notes whose extracted claims & findings make up this cluster.
Bridges (2)
Other communities that share members with this one — cross-cutting threads or papers that sit at the seam between two themes.
Claims (4)
- IMTL-L is equivalent to the logarithm transformation when its parameter st is the exact minimizer in each iteration.Mathematical relationship between IMTL-L and log transformation.
- Logarithm transformation is simpler and more effective than learnable loss transformationCompared to IMTL-L: parameter-free, no extra computational cost, achieves same theoretical goal
- The logarithm transformation also benefits existing gradient balancing methods.Generalization of the loss transformation.
- The logarithm transformation is simpler and more effective than IMTL-L because it is parameter-free.Comparison of loss-scale balancing techniques.
Findings (3)
- log(x) = min_s (e^s * x - s - 1) for x > 0Mathematical equivalence showing logarithm transformation recovers IMTL-L in the limit
- Logarithm transformation improves PCGrad, GradVac, IMTL-G, CAGrad, Nash-MTL, and Aligned-MTL on NYUv2 (Figure 1).Effectiveness of logarithm transformation as a plug-in for gradient balancing methods.
- The logarithm transformation (loss-scale balancing) consistently outperforms IMTL-L on NYUv2, Cityscapes, Office-31, Office-Home.Comparison of loss-scale balancing with IMTL-L.