🛰️ Daily AI Frontier
‹ back to 2026-08-08

稀疏视角3D重建新范式!MAC-Splat:语义引导的多属性一致性让PSNR飙升4.5dB

WeChat: 3D视觉工坊 3D Reconstruction 2026-08-05
Representative image for 稀疏视角3D重建新范式!MAC-Splat:语义引导的多属性一致性让PSNR飙升4.5dB

TL;DR - MAC-Splat is an academic paper (Xidian/Harvard/NTU/CityU/XJTU) that adds semantically-guided 3D consistency supervision to generalizable 3D Gaussian Splatting, lifting sparse-view reconstruction PSNR on ScanNet++ by over 4.5dB versus Splatt3R. It matters because it directly attacks the geometric ambiguity that makes wide-baseline, low-overlap novel-view synthesis fail.

  • Pipeline: MASt3R geometric backbone plus a frozen DINOv3 encoder, fused by a lightweight residual MLP that predicts descriptor residuals; reciprocal-NN matching plus joint confidence thresholding yields sparse high-confidence 2D anchors.
  • MAC loss regularizes matched Gaussians in a shared world frame across three attributes: Huber loss on 3D centers (position), log-eigenvalues of covariance (rotation/scale-invariant shape), and opacity plus SH coefficients (appearance), confidence-weighted; total loss = L_photo + 0.25 × L_MAC.
  • Results: on ScanNet++ it wins PSNR/SSIM/LPIPS on Medium/Wide/Very Wide subsets, degrading only 1.42dB from Close to Very Wide (PixelSplat drops >8dB); masked PSNR on Very Wide is 19.06dB vs Splatt3R's 13.04dB; average LPIPS drops ~44%.
  • Ablations: removing MAC loss collapses Very Wide PSNR to 17.12dB, while MAC-without-DINOv3 still reaches 20.23dB — explicit 3D regularization is the main driver, with semantic fusion adding ~22% LPIPS gain on thin structures. Zero-shot DTU transfer beats PixelSplat, MVSplat, and Splatt3R.

view merged work →