PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction

TL;DR - An arXiv preprint introducing PTP, a black-box LLM inversion method that trains an inverse language model from scratch via "previous-token prediction" to reconstruct the prompts that produced observed outputs. It matters because it shows near-exact prompt recovery without needing model weights, logits, or external datasets — a direct prompt-confidentiality risk.

• Frames inversion functionally rather than as semantic reconstruction: the inverse model is trained on data synthetically generated from the target LLM itself, avoiding fine-tuning of pretrained seq2seq models on large external corpora.
• Previous-token prediction mirrors forward next-token prediction, creating a generative link between forward and inverse processes that the authors argue enables faithful reconstruction.
• Sampling from the inverse model yields diverse candidate prompts that reportedly induce similar responses under the forward target LLM.
• Claims generalization across datasets and transferability to responses from different LLMs, outperforming prior work on token-based prompt and response reconstruction metrics.