Is Self-Pretraining really useful to improve diagnosis in medical Time Series?
Ranking
Overall
58
Content
65
Popularity
42
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv study testing whether Self-PreTraining (SPT) with masking objectives helps transformers on medical time-series classification, finding consistent but modest gains across three clinical datasets. It matters because SPT is an architecture-agnostic way to squeeze more accuracy out of data-limited clinical settings.
- Evaluated transformers on three tasks: rehabilitation robotics (Camargo), stress detection (Non-EEG Stress), and Parkinson's disease detection (Gait PD), training either from scratch or via SPT.
- Four masking-based pre-training objectives targeting temporal and cross-modal representation learning yielded 0–6 percentage point accuracy improvements, varying by masking strategy, dataset, and architecture.
- Gains held not only for multimodal/multivariate inputs but also for simple univariate inputs, suggesting the benefit is not purely cross-modal.
- Model depth was varied systematically; deeper models benefited more, indicating capacity is needed to exploit the pre-trained temporal representations.
Sources (1)
Is Self-Pretraining really useful to improve diagnosis in medical Time Series?
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - An arXiv study testing whether Self-PreTraining (SPT) with masking objectives helps transformers on medical time-series classification, finding consistent but modest gains across three clinical datasets. It matters because SPT is an architecture-agnostic way to squeeze more accuracy out of data-limited clinical settings.
- Evaluated transformers on three tasks: rehabilitation robotics (Camargo), stress detection (Non-EEG Stress), and Parkinson's disease detection (Gait PD), training either from scratch or via SPT.
- Four masking-based pre-training objectives targeting temporal and cross-modal representation learning yielded 0–6 percentage point accuracy improvements, varying by masking strategy, dataset, and architecture.
- Gains held not only for multimodal/multivariate inputs but also for simple univariate inputs, suggesting the benefit is not purely cross-modal.
- Model depth was varied systematically; deeper models benefited more, indicating capacity is needed to exploit the pre-trained temporal representations.