Dual-Balancing Multi-Task Learning

Novel MTL method combining loss-scale and gradient-magnitude balancing

Neighborhood — ranked by edge-count

thinker

concept

gradient-magnitude balancing
implements
Addressing disparity in gradient magnitudes across tasks at the gradient level
loss-scale balancing
implements
Addressing disparity in loss magnitudes across tasks at the loss level
Multi-Task Learning
associated_with
Learning paradigm that jointly learns multiple related tasks using a single model

dataset

Cityscapes
about
Urban scene understanding benchmark with semantic segmentation and depth estimation tasks
NYUv2
about
Indoor scene understanding benchmark with 3 tasks: semantic segmentation, depth estimation, surface normal prediction
Office-31
about
Image classification dataset with 3 domain tasks (Amazon, DSLR, Webcam)
Office-Home
about
Image classification dataset with 4 domain tasks (artistic, clipart, product, real-world)
QM9
about
Molecular property prediction dataset with 11 regression tasks

framework

GradNorm
extends
Gradient balancing method learning task weights; DB-MTL improves on its approach
IMTL-L
extends
Prior loss-balancing method using learnable loss transformation; logarithm approach recovers this

question

task balancing problem
gates
Core challenge where disparity in loss and gradient scales among tasks leads to performance compromises

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

DB-MTL (Dual-Balancing Multi-Task Learning)method0.836
The proposed method combining loss-scale balancing via logarithm transformation and gradient-magnitude balancing via maximum-norm normalization.
Task balancing is still an open problem in multi-task learning.claim0.822
Motivation for the proposed method.
Task balancingconcept0.773
The problem of ensuring all tasks in MTL perform well, avoiding dominance by some tasks.
Task balancing requires simultaneous consideration of both loss scales and gradient magnitudesclaim0.737
Core interpretive position of DB-MTL: complementarity of loss and gradient perspectives
Meta-learningconcept0.731
The capability of GPT-3 to learn tasks from few-shot prompts during runtime.
MetaBalancemethod0.727
Improving recommendations by adapting gradient magnitudes of auxiliary tasks.
Multi-Attempt Responseconcept0.724
A response containing multiple distinct attempts to answer the prompt, used as primary metric for ESR
Contrastive learningframework0.710
Supervised learning framework where system learns by observing contrast between current response and nudged improved response; requires weak additional forces from supervisor