博士论文 | 大动作空间中的在线与离线策略学习
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR — A PhD thesis (Imad Aouali, Institut Polytechnique de Paris/ENSAE) on contextual bandits with very large action spaces, arguing that scale demands shared structure across actions plus optimizable objectives rather than merely better reward estimators. It matters for recommendation, advertising, and interactive decision systems choosing among millions of items.
- Online part: Mixed-effect Thompson Sampling models action parameters as combinations of few shared latent "effects," yielding Bayesian regret bounds that scale with effective (not raw) action count; Diffusion Thompson Sampling replaces the hierarchical prior with a pretrained diffusion generative prior, cutting memory/compute and staying robust under prior misspecification.
- Offline part: A "structured direct method" generates all action parameters from a shared latent variable, giving Bayesian suboptimality guarantees without requiring full logging coverage of every action.
- Optimization over estimation: Under large softmax policies, importance-sampling objectives suffer flat regions, vanishing/exploding gradients; policy-weighted log-likelihood (weighted cross-entropy-like) objectives are smoother and empirically better on MovieLens/Amazon-scale data.
- Variance control: Exponential smoothing replaces hard clipping as a differentiable bias-variance tradeoff, embedded in a PAC-Bayes pessimistic lower-bound objective; choice of pessimistic objective mattered more than the weight-regularization trick (evaluated on MNIST/FashionMNIST/EMNIST/CIFAR bandit-style benchmarks).
Sources (1)
博士论文 | 大动作空间中的在线与离线策略学习
TL;DR — A PhD thesis (Imad Aouali, Institut Polytechnique de Paris/ENSAE) on contextual bandits with very large action spaces, arguing that scale demands shared structure across actions plus optimizable objectives rather than merely better reward estimators. It matters for recommendation, advertising, and interactive decision systems choosing among millions of items.
- Online part: Mixed-effect Thompson Sampling models action parameters as combinations of few shared latent "effects," yielding Bayesian regret bounds that scale with effective (not raw) action count; Diffusion Thompson Sampling replaces the hierarchical prior with a pretrained diffusion generative prior, cutting memory/compute and staying robust under prior misspecification.
- Offline part: A "structured direct method" generates all action parameters from a shared latent variable, giving Bayesian suboptimality guarantees without requiring full logging coverage of every action.
- Optimization over estimation: Under large softmax policies, importance-sampling objectives suffer flat regions, vanishing/exploding gradients; policy-weighted log-likelihood (weighted cross-entropy-like) objectives are smoother and empirically better on MovieLens/Amazon-scale data.
- Variance control: Exponential smoothing replaces hard clipping as a differentiable bias-variance tradeoff, embedded in a PAC-Bayes pessimistic lower-bound objective; choice of pessimistic objective mattered more than the weight-regularization trick (evaluated on MNIST/FashionMNIST/EMNIST/CIFAR bandit-style benchmarks).