ACL 2026
Z3D: Zero-Shot 3D Visual Grounding from Images
A universal pipeline for localizing objects in 3D scenes from multi-view images using zero-shot methods, without geometric supervision or object priors.
Publication record
- Status
- Accepted at ACL 2026
- Authors
- Nikita Drozdov, Andrey Lemeshko, Nikita Gavrilov, Anton Konushin, Danila Rukhovich, Maksim Kolodiazhnyi
- Venue
- ACL 2026
- Research area
- 3D Intelligence
- First arXiv submission
Abstract / summary
Z3D is a universal zero-shot 3D visual-grounding pipeline that localizes the object described by a natural-language query. It operates on multi-view images, can optionally incorporate camera poses and depth maps, and avoids task-specific geometric supervision or predefined object priors.
Methodology
- VLM-assisted image and view selection
- Prompt-based 2D segmentation with SAM3-Agent
- Zero-shot 3D proposal generation with MaskClustering
- 2D-to-3D lifting and multi-view proposal aggregation
- Optional DUSt3R reconstruction for posed or unposed imagery
Datasets and benchmarks
- ScanRefer
- Nr3D
Principal results
- The paper reports state-of-the-art performance among zero-shot methods on ScanRefer and Nr3D at publication time.
- In the full ScanRefer validation setting without bounding-box or text supervision, Z3D reports overall [email protected] of 54.2 and [email protected] of 46.0.
- On Nr3D, Z3D reports overall top-1 accuracy of 54.8 in the paper's main zero-shot comparison.
- The same pipeline supports point-cloud, posed-RGB, and fully unposed-RGB inputs.
Benchmark evidence
| Benchmark | Task | Input | Metric | Result | Evidence |
|---|---|---|---|---|---|
| ScanRefer | Zero-shot 3D visual grounding | Ground-truth point cloud; no bounding-box or text supervision | Overall [email protected] / [email protected] | 54.2 / 46.0 | Z3D paper, Table 1 Reported as state of the art among zero-shot approaches in the associated paper. |
| ScanRefer | Zero-shot 3D visual grounding | Posed multi-view RGB | Overall [email protected] / [email protected] | 42.8 / 24.8 | Z3D paper, Table 1 |
| ScanRefer | Zero-shot 3D visual grounding | Unposed multi-view RGB | Overall [email protected] / [email protected] | 31.2 / 12.9 | Z3D paper, Table 1 |
| Nr3D | Zero-shot 3D visual grounding | Depth-aware setting | Overall top-1 accuracy | 54.8 | Z3D paper, Table 2 The strongest zero-shot result in the paper's main Nr3D table was 54.8 versus 54.3 for SPAZER. |
Values are attributed to the associated paper and the stated input and evaluation protocol.
Technologies and methods
- PyTorch
- CLIP
- SAM3
- Qwen vision-language models
- DUSt3R
Related M:3L research
Evidence boundary
Benchmark claims and numerical values are paper-reported results for the named protocol. Potential applications are not evidence of completed deployments.