Kyle Vedder Neehar Peri Nathaniel Chodosh Ishan Khatri Eric Eaton Dinesh Jayaraman Yang Liu Deva Ramanan

Abstract
Scene flow estimation is the task of describing the 3D motion field between temporally successive point clouds. State-of-the-art methods use strong priors and test-time optimization techniques, but require on the order of tens of seconds to process full-size point clouds, making them unusable as computer vision primitives for real-time applications such as open world object detection. Feedforward methods are considerably faster, running on the order of tens to hundreds of milliseconds for full-size point clouds, but require expensive human supervision. To address both limitations, we propose Scene Flow via Distillation, a simple, scalable distillation framework that uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model. Our instantiation of this framework, ZeroFlow, achieves state-of-the-art performance on the Argoverse 2 Self-Supervised Scene Flow Challenge while using zero human labels by simply training on large-scale, diverse unlabeled data. At test-time, ZeroFlow is over 1000x faster than label-free state-of-the-art optimization-based methods on full-size point clouds (34 FPS vs 0.028 FPS) and over 1000x cheaper to train on unlabeled data compared to the cost of human annotation ($394 vs ~$750,000). To facilitate further research, we release our code, trained model weights, and high quality pseudo-labels for the Argoverse 2 and Waymo Open datasets at https://vedder.io/zeroflow.html
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| scene-flow-estimation-on-argoverse-2 | ZeroFlow 5x XL | EPE 3-Way: 0.049392 EPE Background Static: 0.013082 EPE Foreground Dynamic: 0.117688 EPE Foreground Static: 0.017406 |
| self-supervised-scene-flow-estimation-on-1 | ZeroFlow 5x XL | EPE 3-Way: 0.049392 EPE Background Static: 0.013082 EPE Foreground Dynamic: 0.117688 EPE Foreground Static: 0.017406 |
| self-supervised-scene-flow-estimation-on-1 | ZeroFlow | EPE 3-Way: 0.0814 EPE Foreground Dynamic: 0.2109 EPE Foreground Static: 0.0254 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.