# Zoo3D: Zero-Shot 3D Object Detection at Scene Level

> The first training-free 3D object detection framework. It constructs 3D bounding boxes through graph clustering of 2D instance masks and assigns open-vocabulary semantic labels.

## Publication record

- **Title:** Zoo3D: Zero-Shot 3D Object Detection at Scene Level
- **Authors:** Andrey Lemeshko, Bulat Gabdullin, Nikita Drozdov, Anton Konushin, Danila Rukhovich, Maksim Kolodiazhnyi
- **Year:** 2026
- **Status:** Accepted at CVPR 2026
- **Venue:** CVPR 2026
- **Conference ranking:** A*-ranked conference
- **Research area:** 3D Intelligence
- **First arXiv submission:** 2025-11-25

## Abstract / summary

Zoo3D introduces scene-level zero-shot 3D object detection. Its training-free Zoo3D0 mode constructs class-agnostic 3D boxes from 2D instance masks and assigns open-vocabulary labels; Zoo3D1 then trains a class-agnostic detector on Zoo3D0-generated pseudo-labels.

## Methodology

- 2D instance-mask extraction across scene views
- Graph clustering for class-agnostic 3D bounding-box construction
- Best-view selection and view-consensus mask generation
- Open-vocabulary visual-language semantic labeling
- Zero-shot Zoo3D0 and pseudo-label-based self-supervised Zoo3D1 modes
- DUSt3R-based support for posed and unposed image inputs

## Datasets and benchmarks

- ScanNet20
- ScanNet60
- ScanNet200
- ARKitScenes

## Principal results

- The paper reports state-of-the-art open-vocabulary 3D-detection results across ScanNet-based benchmarks at publication time.
- On ScanNet200 with point-cloud inputs, Zoo3D0 reports mAP@0.25/mAP@0.5 of 21.1/14.1 without training, while Zoo3D1 reports 23.5/15.2.
- On ARKitScenes, the paper evaluates point-cloud and unposed-image tracks and reports mAP at IoU thresholds of 0.25 and 0.5.

## Benchmark evidence

| Benchmark | Task | Input | Metric | Result | Evidence |
| --- | --- | --- | --- | --- | --- |
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D0) | Point cloud and posed scene images; training-free | mAP@0.25 / mAP@0.5 | 21.1 / 14.1 | Zoo3D paper, Table 2 |
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D1) | Point cloud and posed scene images; self-supervised pseudo-label training | mAP@0.25 / mAP@0.5 | 23.5 / 15.2 | Zoo3D paper, Table 2 |
| ScanNet200 | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Fully unposed multi-view images | mAP@0.25 / mAP@0.5 | 8.3 / 2.9 (Zoo3D0); 10.7 / 3.8 (Zoo3D1) | Zoo3D paper, Table 2 |
| ARKitScenes | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Point cloud and posed images | mAP@0.25 / mAP@0.5 | 24.4 / 11.0 (Zoo3D0); 34.2 / 24.2 (Zoo3D1) | Zoo3D paper, Table 15 |
| ARKitScenes | Open-vocabulary 3D object detection (Zoo3D0 / Zoo3D1) | Unposed multi-view images | mAP@0.25 / mAP@0.5 | 13.0 / 2.6 (Zoo3D0); 16.1 / 3.5 (Zoo3D1) | Zoo3D paper, Table 15 |

Metrics are reported exactly for the stated input and evaluation setting; rows from different settings should not be treated as directly interchangeable.

## Technologies and methods

- CLIP
- SAM 2.1
- MaskClustering
- DUSt3R
- Sparse 3D detection

## Related M:3L research

- [3D Intelligence](https://m3l.am/research/3d-intelligence.md)
- [CAD Intelligence](https://m3l.am/research/cad-intelligence.md)

## External resources

- [arXiv record](https://arxiv.org/abs/2511.20253)
- [DOI](https://doi.org/10.48550/arXiv.2511.20253)
- [Code repository](https://github.com/col14m/zoo3d)

## Evidence boundary

Benchmark claims and numerical values above are attributed to the associated paper and its stated protocol. Potential applications are not evidence of completed deployments.

## About M:3L

M:3L is the Mathematical Modeling & Machine Learning Laboratory, a research and engineering laboratory in Yerevan, Armenia, established at the Institute of Mechanics of the National Academy of Sciences of the Republic of Armenia.

## Authoritative URLs

- [Canonical publication page](https://m3l.am/publications/zoo3d)
- [All M:3L publications](https://m3l.am/publications.md)
- [M:3L research](https://m3l.am/research.md)