Do Tabular Foundation Models Agree with Themselves?
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv study tests whether Tabular Foundation Models (TFMs) — transformers that approximate a Bayesian posterior predictive and are extended to multivariate prediction by autoregressive sampling — produce predictions consistent with any valid joint distribution. Every evaluated TFM fails, undermining the probabilistic interpretation of their multivariate outputs.
- TFMs are univariate predictors turned multivariate by sampling a target and appending it to the features; the faithfulness of that induced joint had not been examined.
- Direct evaluation against the true posterior is impossible on real data (ground-truth distribution unknown), so the authors instead ask whether predictions could arise from any joint distribution.
- Two necessary conditions are proposed: marginalization consistency (marginalized conditionals must match directly predicted marginals) and factorization consistency (different factorization orders must yield the same joint).
- All evaluated TFMs violate both requirements, for classification and regression, across every dataset tested.
Sources (1)
Do Tabular Foundation Models Agree with Themselves?
TL;DR - An arXiv study tests whether Tabular Foundation Models (TFMs) — transformers that approximate a Bayesian posterior predictive and are extended to multivariate prediction by autoregressive sampling — produce predictions consistent with any valid joint distribution. Every evaluated TFM fails, undermining the probabilistic interpretation of their multivariate outputs.
- TFMs are univariate predictors turned multivariate by sampling a target and appending it to the features; the faithfulness of that induced joint had not been examined.
- Direct evaluation against the true posterior is impossible on real data (ground-truth distribution unknown), so the authors instead ask whether predictions could arise from any joint distribution.
- Two necessary conditions are proposed: marginalization consistency (marginalized conditionals must match directly predicted marginals) and factorization consistency (different factorization orders must yield the same joint).
- All evaluated TFMs violate both requirements, for classification and regression, across every dataset tested.