GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - GradCuit is a test-time latent reasoning method that inserts optimizable latent states inside a frozen Transformer layer so reward-weighted gradients from the whole generated continuation flow directly to those latents. It matters because it makes test-time scaling act on how a model reasons internally, rather than just resampling or reranking outputs.
- Latents are placed at a selected Transformer layer between prompt hidden states and the continuation; causal self-attention gives every continuation-token log-probability a differentiable path back to each latent, enabling direct sequence-level credit assignment instead of routing through decoded tokens.
- Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, it averages 64.5% accuracy — +6.6 points over chain-of-thought prompting and +2.4 over the strongest competing method.
- Robustness: over seven learning-rate settings it consistently beats LatentSeek and cuts accuracy standard deviation from 1.53 to 0.82; even a random-walk variant stays competitive with LatentSeek.
- Interpretability: token-level gradient attribution shows latent influence concentrating on reasoning-connector tokens, and layer analysis finds early-to-middle layers to be the most effective optimization space.
Sources (1)
GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning
TL;DR - GradCuit is a test-time latent reasoning method that inserts optimizable latent states inside a frozen Transformer layer so reward-weighted gradients from the whole generated continuation flow directly to those latents. It matters because it makes test-time scaling act on how a model reasons internally, rather than just resampling or reranking outputs.
- Latents are placed at a selected Transformer layer between prompt hidden states and the continuation; causal self-attention gives every continuation-token log-probability a differentiable path back to each latent, enabling direct sequence-level credit assignment instead of routing through decoded tokens.
- Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, it averages 64.5% accuracy — +6.6 points over chain-of-thought prompting and +2.4 over the strongest competing method.
- Robustness: over seven learning-rate settings it consistently beats LatentSeek and cuts accuracy standard deviation from 1.53 to 0.82; even a random-walk variant stays competitive with LatentSeek.
- Interpretability: token-level gradient attribution shows latent influence concentrating on reasoning-connector tokens, and layer analysis finds early-to-middle layers to be the most effective optimization space.