SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture

TL;DR - SPECTRA is a runtime-reconfigurable tiled architecture for efficient speculative decoding on resource-constrained edge devices. Its per-kernel adaptation maintains high utilization across memory-bound decoding, compute-bound prefill, and variable-intensity verification workloads.

• Each tile switches between systolic GEMM execution and vector-lane GEMV execution.
• System-level reconfiguration dynamically adjusts tile count, kernel partitioning, and communication patterns.
• A 20-tile FPGA prototype was evaluated on Pythia, SmolLM2, and GPT-2 model families.
• Tile-level reconfiguration delivered up to a 2.09× speedup, with system-level adaptability adding a further 1.25× gain over fixed designs.