Context-weighted Discrete Flow Matching
Ranking
Overall
72
Content
75
Popularity
66
Observed public metrics from 1 member.
Merged summary
TL;DR - This paper makes discrete flow matching context-aware, prioritizing tokens based on nearby contextual information. The method improves generation with negligible overhead and reduces OpenWebText generative perplexity by up to 63%.
- Modifies the underlying continuous-time Markov chain to incorporate local context density.
- Reweights token-level training signals with a scaled cross-entropy loss.
- Matches a strong semi-autoregressive block diffusion baseline while preserving arbitrary-order generation.
- Improves both sampling quality and training efficiency.
Sources (1)
Context-weighted Discrete Flow Matching
Public signals
Hugging Face upvotes 4
TL;DR - This paper makes discrete flow matching context-aware, prioritizing tokens based on nearby contextual information. The method improves generation with negligible overhead and reduces OpenWebText generative perplexity by up to 63%.
- Modifies the underlying continuous-time Markov chain to incorporate local context density.
- Reweights token-level training signals with a scaled cross-entropy loss.
- Matches a strong semi-autoregressive block diffusion baseline while preserving arbitrary-order generation.
- Improves both sampling quality and training efficiency.