DriveTrack
A Benchmark for Long-Range Point Tracking in Real-World Videos

[Paper]      [Code]      [Dataset]


DriveTrack is a new benchmark and data generation framework for long-range keypoint tracking in real-world videos. DriveTrack is motivated by the observation that the accuracy of state-of-the-art trackers depends strongly on visual attributes around the selected keypoints, such as texture and lighting. The problem is that these artifacts are especially pronounced in real-world videos, but these trackers are unable to train on such scenes due to a dearth of annotations. DriveTrack bridges this gap by building a framework to automatically annotate point tracks on autonomous driving datasets. We release a dataset consisting of 1 billion point tracks across 24 hours of video, which is seven orders of magnitude greater than prior real-world benchmarks and on par with the scale of synthetic benchmarks.

DriveTrack unlocks new use cases for point tracking in real-world videos. First, we show that fine-tuning keypoint trackers on DriveTrack improves accuracy on real-world scenes by up to 7%. Second, we analyze the sensitivity of trackers to visual artifacts in real scenes and motivate the idea of running assistive keypoint selectors alongside trackers.

Changelog

Demo

Paper

DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos

Arjun Balasingam, Joseph Chandler, Chenning Li, Zhoutong Zhang, Hari Balakrishnan.
To appear at CVPR 2024

Cite this work

@misc{balasingam-drivetrack,
      title={DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos}, 
      author={Arjun Balasingam and Joseph Chandler and Chenning Li and Zhoutong Zhang and Hari Balakrishnan},
      year={2023},
      eprint={2312.09523},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contact

If you have any questions, please file an issue on GitHub or reach out to us via email.