Stealing Reasoning Traces from Proprietary LLM APIs
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv cs.CR paper showing that encrypted client-side chain-of-thought blocks returned by major LLM providers are interchangeable across sessions, users, and models, letting an attacker feed a strong model's encrypted trace to a weaker sibling model that decodes it verbatim into plaintext. This breaks anti-distillation protections and turns publicly shared session logs into a leak channel.
- The core flaw is architectural, not cryptographic-key theft: because encrypted reasoning blocks are portable within a provider's ecosystem, a less-safeguarded model from the same provider can be prompted to emit the trace in plaintext without jailbreaking the stronger model.
- Demonstrated across Anthropic, OpenAI, and Google, enabling extraction of proprietary reasoning traces and circumventing anti-distillation mechanisms.
- Decoding 315,320 reasoning blocks scraped from public repositories recovered 367 PII artifacts and 182 credentials, since developers share logs unaware of the blocks' contents.
- Two further vectors: hazardous content hidden in reasoning even when the visible answer refuses, and invisible prompt injection via payloads embedded in encrypted blocks to poison public agentic rollouts; the authors responsibly disclosed and propose cryptographic and system-level mitigations.
Sources (1)
Stealing Reasoning Traces from Proprietary LLM APIs
TL;DR - An arXiv cs.CR paper showing that encrypted client-side chain-of-thought blocks returned by major LLM providers are interchangeable across sessions, users, and models, letting an attacker feed a strong model's encrypted trace to a weaker sibling model that decodes it verbatim into plaintext. This breaks anti-distillation protections and turns publicly shared session logs into a leak channel.
- The core flaw is architectural, not cryptographic-key theft: because encrypted reasoning blocks are portable within a provider's ecosystem, a less-safeguarded model from the same provider can be prompted to emit the trace in plaintext without jailbreaking the stronger model.
- Demonstrated across Anthropic, OpenAI, and Google, enabling extraction of proprietary reasoning traces and circumventing anti-distillation mechanisms.
- Decoding 315,320 reasoning blocks scraped from public repositories recovered 367 PII artifacts and 182 credentials, since developers share logs unaware of the blocks' contents.
- Two further vectors: hazardous content hidden in reasoning even when the visible answer refuses, and invisible prompt injection via payloads embedded in encrypted blocks to poison public agentic rollouts; the authors responsibly disclosed and propose cryptographic and system-level mitigations.