RSS-2026 | 港科大（广州）开源首个无需训练的开放词汇3D占据地图构建系统｜RRS 2026
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - FreeOcc (HKUST-Guangzhou + MBZUAI, accepted to RSS 2026) is the first training-free, open-vocabulary 3D semantic occupancy prediction system, building globally consistent voxel occupancy maps online from monocular or RGB-D image sequences without 3D annotations or ground-truth camera poses. It matters because it removes the annotation and per-scene retraining bottleneck that blocks embodied perception from generalizing to new environments.
- Four-layer modular map: SLAM point cloud (DROID-SLAM backbone) → 3D Gaussians → open-vocabulary semantics from a pretrained VLM lifted onto Gaussian primitives → probabilistic Gaussian-to-Occupancy voxel projection queryable by arbitrary text.
- Key designs are Geometry-aware Initialization (G-ini) and Geometrically Anchored Gaussian Updates (GAGU), which anchor Gaussian centers to SLAM points instead of letting them drift for rendering loss; ablation (RGB-D) goes 27.98 IoU / 11.20 mIoU / 8.8 FPS → 45.03 IoU / 18.37 mIoU / 24.6 FPS.
- On EmbodiedOcc-ScanNet it reaches 31.29 IoU / 13.86 mIoU (monocular) and 34.40 / 15.84 (RGB-D), over 2x self-supervised baselines GaussianOcc (10.17/4.34) and GaussTR (15.63/4.95) despite using no poses.
- Authors introduce the ReplicaOcc zero-shot benchmark with finer-grained classes, where supervised EmbodiedOcc collapses to ~0 mIoU while FreeOcc holds 55.65 IoU / 20.90 mIoU (RGB-D); code, dataset, and a real RealSense D435i + Qwen3-VL deployment are released.
Sources (1)
RSS-2026 | 港科大（广州）开源首个无需训练的开放词汇3D占据地图构建系统｜RRS 2026
TL;DR - FreeOcc (HKUST-Guangzhou + MBZUAI, accepted to RSS 2026) is the first training-free, open-vocabulary 3D semantic occupancy prediction system, building globally consistent voxel occupancy maps online from monocular or RGB-D image sequences without 3D annotations or ground-truth camera poses. It matters because it removes the annotation and per-scene retraining bottleneck that blocks embodied perception from generalizing to new environments.
- Four-layer modular map: SLAM point cloud (DROID-SLAM backbone) → 3D Gaussians → open-vocabulary semantics from a pretrained VLM lifted onto Gaussian primitives → probabilistic Gaussian-to-Occupancy voxel projection queryable by arbitrary text.
- Key designs are Geometry-aware Initialization (G-ini) and Geometrically Anchored Gaussian Updates (GAGU), which anchor Gaussian centers to SLAM points instead of letting them drift for rendering loss; ablation (RGB-D) goes 27.98 IoU / 11.20 mIoU / 8.8 FPS → 45.03 IoU / 18.37 mIoU / 24.6 FPS.
- On EmbodiedOcc-ScanNet it reaches 31.29 IoU / 13.86 mIoU (monocular) and 34.40 / 15.84 (RGB-D), over 2x self-supervised baselines GaussianOcc (10.17/4.34) and GaussTR (15.63/4.95) despite using no poses.
- Authors introduce the ReplicaOcc zero-shot benchmark with finer-grained classes, where supervised EmbodiedOcc collapses to ~0 mIoU while FreeOcc holds 55.65 IoU / 20.90 mIoU (RGB-D); code, dataset, and a real RealSense D435i + Qwen3-VL deployment are released.