DriveTrack is a new benchmark and data generation framework for long-range keypoint tracking in real-world videos. DriveTrack is motivated by the observation that the accuracy of state-of-the-art trackers depends strongly on visual attributes around the selected keypoints, such as texture and lighting. The problem is that these artifacts are especially pronounced in real-world videos, but these trackers are unable to train on such scenes due to a dearth of annotations. DriveTrack bridges this gap by building a framework to automatically annotate point tracks on autonomous driving datasets. We release a dataset consisting of 1 billion point tracks across 24 hours of video, which is seven orders of magnitude greater than prior real-world benchmarks and on par with the scale of synthetic benchmarks.
DriveTrack unlocks new use cases for point tracking in real-world videos. First, we show that fine-tuning keypoint trackers on DriveTrack improves accuracy on real-world scenes by up to 7%. Second, we analyze the sensitivity of trackers to visual artifacts in real scenes and motivate the idea of running assistive keypoint selectors alongside trackers.
Arjun Balasingam, Joseph Chandler, Chenning Li,
Zhoutong Zhang, Hari Balakrishnan.
To appear at CVPR 2024
@misc{balasingam-drivetrack,
title={DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos},
author={Arjun Balasingam and Joseph Chandler and Chenning Li and Zhoutong Zhang and Hari Balakrishnan},
year={2023},
eprint={2312.09523},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
If you have any questions, please file an issue on GitHub or reach out to us via email.