Task-Specific Feature Fusion Method for Multi-Task Affective Behavior Analysis
Merged summary
TL;DR — A vision-based system for the ABAW11 challenge that predicts three facial affect tasks (valence-arousal, expressions, action units) from images, showing that task-specific feature fusion beats a single shared architecture.
- Adapts two frozen pretrained visual backbones (DINOv2 ViT-L and DINOv3 ConvNeXt-base) fine-tuned on external facial expression data to extract complementary frame-level features.
- Systematically compares many heads/fusion methods (frame-level, temporal convolution, smoothing, LightGBM, gated/residual/late-logit fusion, threshold calibration, shared MTL) and selects per-task strategies rather than one unified model.
- Validation results: EXPR macro-F1 0.4222, AU macro-F1 0.5402, mean VA CCC 0.6717 (overall score 1.6341).
- Key takeaway: task-adaptive fusion of frozen visual features is a simple, effective approach for multi-task affective behavior analysis.
Sources (1)
Task-Specific Feature Fusion Method for Multi-Task Affective Behavior Analysis
TL;DR — A vision-based system for the ABAW11 challenge that predicts three facial affect tasks (valence-arousal, expressions, action units) from images, showing that task-specific feature fusion beats a single shared architecture.
- Adapts two frozen pretrained visual backbones (DINOv2 ViT-L and DINOv3 ConvNeXt-base) fine-tuned on external facial expression data to extract complementary frame-level features.
- Systematically compares many heads/fusion methods (frame-level, temporal convolution, smoothing, LightGBM, gated/residual/late-logit fusion, threshold calibration, shared MTL) and selects per-task strategies rather than one unified model.
- Validation results: EXPR macro-F1 0.4222, AU macro-F1 0.5402, mean VA CCC 0.6717 (overall score 1.6341).
- Key takeaway: task-adaptive fusion of frozen visual features is a simple, effective approach for multi-task affective behavior analysis.