ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps

TL;DR - ActMap estimates an LLM answer’s correctness from a single generation by compactly encoding hidden-state activity across every layer and generated token. It enables low-overhead uncertainty scoring for abstention, routing, and selective verification.

• Compresses generation-time activations into a fixed 12 × 32 × 128 tensor occupying 96 KiB, with no measurable capture overhead.
• A lightweight classifier predicts correctness in under a millisecond; comparable MLP results suggest the representation, rather than classifier architecture, drives performance.
• Across QA, math, and summarization factuality tasks with three instruction-tuned 7–8B models, ActMap outperformed sampling, token-probability, attention, and embedding baselines.
• It matched the mean AUROC of ACT-ViT while using activation tensors 67× smaller and achieved lower calibration error on 10 of 12 evaluated pairs.