artifact
active
artifact:lin-2023-dual-balancing-for-multi-task-learning

Lin 2023 Dual-Balancing for Multi-Task Learning

The paper proposing the Dual-Balancing Multi-Task Learning method.

Neighborhood — ranked by edge-count

Thinkers (9)

thinker

Frameworks (2)

framework
  • GradNorm
    mentions
    Gradient balancing method learning task weights; DB-MTL improves on its approach
  • IMTL-L
    mentions
    Prior loss-balancing method using learnable loss transformation; logarithm approach recovers this

Methods (26)

method
  • DB-MTL (Dual-Balancing Multi-Task Learning)
    introduces
    The proposed method combining loss-scale balancing via logarithm transformation and gradient-magnitude balancing via maximum-norm normalization.
  • Adam
    mentions
    Optimizer used for training.
  • Aligned-MTL
    mentions
    Independent component alignment for multi-task learning.
  • CAGrad
    mentions
    Conflict-averse gradient descent, constraining aggregated gradient around average.
  • Cross-stitch networks
    mentions
    MTL architecture with linear combinations of activations across tasks.
  • DeepLabV3+
    mentions
    Segmentation network used as encoder-decoder in scene understanding experiments.
  • Dynamic Weight Average (DWA)
    mentions
    Loss balancing based on learning speed.
  • Equal Weighting (EW)
    mentions
    Baseline that minimizes sum of task losses with equal weights.
  • Geometric Loss Strategy (GLS)
    mentions
    Minimizes the geometric mean loss.
  • GradDrop
    mentions
    Gradient balancing by masking out gradient values with inconsistent signs.
  • GradVac
    mentions
    Gradient balancing by aligning gradients regardless of conflict.
  • Hard-parameter sharing (HPS)
    mentions
    Architecture pattern with a shared encoder and task-specific heads.
  • Improvable Gap Balancing v2 (IGBv2)
    mentions
    Loss balancing using improvable gap.
  • IMTL
    mentions
    Hybrid method combining IMTL-L and IMTL-G.
  • IMTL-G
    mentions
    Gradient balancing enforcing equal projections on each task gradient.
  • MetaBalance
    mentions
    Improving recommendations by adapting gradient magnitudes of auxiliary tasks.
  • MoCo
    mentions
    Mitigates gradient bias in multi-objective learning with momentum and regularization.
  • MTAdam
    mentions
    Automatic balancing of multiple training loss terms.
  • MTAN
    mentions
    Multi-Task Attention Network for MTL.
  • Multiple Gradient Descent Algorithm (MGDA)
    mentions
    Gradient balancing by solving multi-objective optimization for minimum-norm aggregated gradient.
  • Nash-MTL
    mentions
    Gradient aggregation via Nash bargaining game.
  • PCGrad
    mentions
    Gradient balancing by projecting conflicting gradients.
  • Random Loss Weighting (RLW)
    mentions
    Samples task weights from a standard normal distribution.
  • ResNet-50
    mentions
    Backbone network pre-trained on ImageNet.

+2 more

Datasets (6)

dataset
  • Cityscapes
    mentions
    Urban scene understanding benchmark with semantic segmentation and depth estimation tasks
  • ImageNet
    mentions
    Large-scale image database used for pre-training.
  • NYUv2
    mentions
    Indoor scene understanding benchmark with 3 tasks: semantic segmentation, depth estimation, surface normal prediction
  • Office-31
    mentions
    Image classification dataset with 3 domain tasks (Amazon, DSLR, Webcam)
  • Office-Home
    mentions
    Image classification dataset with 4 domain tasks (artistic, clipart, product, real-world)
  • QM9
    mentions
    Molecular property prediction dataset with 11 regression tasks

Concepts (1)

concept
  • Multi-Task Learning
    mentions
    Learning paradigm that jointly learns multiple related tasks using a single model

Artifacts (1)

artifact
  • LibMTL
    mentions
    Python library for multi-task learning used for implementation.