CVPR 2026
Zoo3D: Zero-Shot 3D Object Detection at Scene Level
The first training-free 3D object detection framework. It constructs 3D bounding boxes through graph clustering of 2D instance masks and assigns open-vocabulary semantic labels.
Publication record
- Status
- Accepted at CVPR 2026
- Authors
- Andrey Lemeshko, Bulat Gabdullin, Nikita Drozdov, Anton Konushin, Danila Rukhovich, Maksim Kolodiazhnyi
- Venue
- CVPR 2026
- Conference ranking
- CORE 2023 A*-ranked conference
- Research area
- 3D Intelligence
- First arXiv submission
Abstract / summary
Zoo3D introduces scene-level zero-shot 3D object detection. Its training-free Zoo3D0 mode constructs class-agnostic 3D boxes from 2D instance masks and assigns open-vocabulary labels; Zoo3D1 then trains a class-agnostic detector on Zoo3D0-generated pseudo-labels.
Methodology
- 2D instance-mask extraction across scene views
- Graph clustering for class-agnostic 3D bounding-box construction
- Best-view selection and view-consensus mask generation
- Open-vocabulary visual-language semantic labeling
- Zero-shot Zoo3D0 and pseudo-label-based self-supervised Zoo3D1 modes
- DUSt3R-based support for posed and unposed image inputs
Datasets and benchmarks
- ScanNet20
- ScanNet60
- ScanNet200
- ARKitScenes
Principal results
- The paper reports state-of-the-art open-vocabulary 3D-detection results across ScanNet-based benchmarks at publication time.
- On ScanNet200 with point-cloud inputs, Zoo3D0 reports [email protected]/[email protected] of 21.1/14.1 without training, while Zoo3D1 reports 23.5/15.2.
- On ARKitScenes, the paper evaluates point-cloud and unposed-image tracks and reports mAP at IoU thresholds of 0.25 and 0.5.
Benchmark evidence
| Benchmark | Task | Input | Metric | Result | Evidence |
|---|---|---|---|---|---|
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D0) | Point cloud and posed scene images; training-free | [email protected] / [email protected] | 21.1 / 14.1 | Zoo3D paper, Table 2 |
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D1) | Point cloud and posed scene images; self-supervised pseudo-label training | [email protected] / [email protected] | 23.5 / 15.2 | Zoo3D paper, Table 2 |
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Fully unposed multi-view images | [email protected] / [email protected] | 8.3 / 2.9 (Zoo3D0); 10.7 / 3.8 (Zoo3D1) | Zoo3D paper, Table 2 |
| ARKitScenes | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Point cloud and posed images | [email protected] / [email protected] | 24.4 / 11.0 (Zoo3D0); 34.2 / 24.2 (Zoo3D1) | Zoo3D paper, Table 15 |
| ARKitScenes | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Unposed multi-view images | [email protected] / [email protected] | 13.0 / 2.6 (Zoo3D0); 16.1 / 3.5 (Zoo3D1) | Zoo3D paper, Table 15 |
Values are attributed to the associated paper and the stated input and evaluation protocol.
Technologies and methods
- CLIP
- SAM 2.1
- MaskClustering
- DUSt3R
- Sparse 3D detection
Related M:3L research
Evidence boundary
Benchmark claims and numerical values are paper-reported results for the named protocol. Potential applications are not evidence of completed deployments.