Empowering On-Device Model Adaptation with an Edge AI Inference Accelerator

TL;DR - This paper repurposes a Hailo-8L inference accelerator for efficient on-device adaptation by running an INT8 frozen backbone on the accelerator and training only a small FP32 classification head on the CPU. The approach enables faster, lower-energy personalization on resource-constrained hardware.

• Achieves up to 15.4× faster training than a Raspberry Pi 5 CPU baseline.
• Consistently reduces energy consumption per training sample.
• Keeps most model weights frozen, avoiding costly end-to-end backpropagation.
• Post-training quantization restoration is critical for preserving feature quality in quantization-sensitive architectures.