Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
Merged summary
TL;DR - This paper repurposes a Hailo-8L inference accelerator for efficient on-device adaptation by running an INT8 frozen backbone on the accelerator and training only a small FP32 classification head on the CPU. The approach enables faster, lower-energy personalization on resource-constrained hardware.
- Achieves up to 15.4× faster training than a Raspberry Pi 5 CPU baseline.
- Consistently reduces energy consumption per training sample.
- Keeps most model weights frozen, avoiding costly end-to-end backpropagation.
- Post-training quantization restoration is critical for preserving feature quality in quantization-sensitive architectures.
Sources (1)
Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator
TL;DR - This paper repurposes a Hailo-8L inference accelerator for efficient on-device adaptation by running an INT8 frozen backbone on the accelerator and training only a small FP32 classification head on the CPU. The approach enables faster, lower-energy personalization on resource-constrained hardware.
- Achieves up to 15.4× faster training than a Raspberry Pi 5 CPU baseline.
- Consistently reduces energy consumption per training sample.
- Keeps most model weights frozen, avoiding costly end-to-end backpropagation.
- Post-training quantization restoration is critical for preserving feature quality in quantization-sensitive architectures.