还在为大模型洗数据熬夜？蚂蚁拿下VLDB工业最佳论文，一套宽表搞定35PB语料，效率狂飙5.6倍
Ranking
Overall
78
Content
95
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - Ant Group introduced OmniTable, a unified wide-table system for petabyte-scale LLM data curation that won VLDB 2026’s Best Industry Paper. In a production SFT workflow, it cut preparation time from about 14 days to 2.5 days while managing more than 35 PB and 305 billion records.
- OmniTable exposes each data domain as a logical wide table while physically partitioning storage by scale, access pattern, and compute engine.
- Its catalog treats batches and features as first-class, versioned assets, tracking column-level dependencies, execution state, physical location, and lineage.
- Record-level fault isolation prevents rare malformed samples from failing entire jobs; operator fusion also reduced eight scans to one in a 2.5 PB experiment.
- The system routes work across Spark, MaxCompute SQL, and GPU inference platforms and continuously reorganizes physical layouts without changing the logical schema.
Sources (1)
还在为大模型洗数据熬夜？蚂蚁拿下VLDB工业最佳论文，一套宽表搞定35PB语料，效率狂飙5.6倍
Public signals
OpenAlex citations 0
TL;DR - Ant Group introduced OmniTable, a unified wide-table system for petabyte-scale LLM data curation that won VLDB 2026’s Best Industry Paper. In a production SFT workflow, it cut preparation time from about 14 days to 2.5 days while managing more than 35 PB and 305 billion records.
- OmniTable exposes each data domain as a logical wide table while physically partitioning storage by scale, access pattern, and compute engine.
- Its catalog treats batches and features as first-class, versioned assets, tracking column-level dependencies, execution state, physical location, and lineage.
- Record-level fault isolation prevents rare malformed samples from failing entire jobs; operator fusion also reduced eight scans to one in a 2.5 PB experiment.
- The system routes work across Spark, MaxCompute SQL, and GPU inference platforms and continuously reorganizes physical layouts without changing the logical schema.