PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object Tracking
- Aakash Kumar University of Central Florida
- Jyoti Kini University of Central Florida
- Ajmal Mian University of Western Australia
- Mubarak Shah Univerity of Central Florida
Abstract
In recent times, the scope of LIDAR (Light Detection and Ranging) sensor-based technology has spread across numerous fields. It is popularly used to map terrain and navigation information into reliable 3D point cloud data, potentially revolutionizing the autonomous vehicles and assistive robotic industry. A point cloud is a dense compilation of spatial data in 3D coordinates. It plays a vital role in modeling complex real-world scenes since it preserves structural information and avoids perspective distortion, unlike image data, which is the projection of a 3D structure on a 2D plane. In order to leverage the intrinsic capabilities of the LIDAR data, we propose a PointNet-based approach for 3D Multi- Object Tracking (MOT).
Video
Proposed Network Architecture
Our solution consists of a PointNet feature extractor, a component for exhaustive pairing permutation of features, and a compression network. Initially, we crop objects from the point cloud using 3D detections and pass these through PointNet to generate the features for each object. Exhaustive permutations of these feature vectors \( F_t \) and \( F_{t-n} \) are encoded in a tensor \( \in \mathbb{R}^{N_m \times N_m \times 1024} \), where \( N_m \) is the number of objects in each frame. For our experiments, we set \( N_m \) to 100 objects. Next, a compression network consisting of 5 convolution layers is used to map the encodings to \( M \in \mathbb{R}^{N_m \times N_m} \). The resultant matrix \( M \) is augmented to \( M_1 \) and \( M_2 \) matrices by appending an extra column and row respectively to account for the objects entering and leaving the scene. Thereafter, row-wise and column-wise softmax operations are performed to generate corresponding \( A_1 \) and \( A_2 \) matrices. These matrices, along with their column and row trimmed versions \( \hat{A}_1 \) and \( \hat{A}_2 \) respectively, are used for the loss computation. \( A_1 \) and \( A_2 \) matrices are used to predict the affinity between a pair of frames.
Qualitative Results
Quantitative Results
Quantitave Results on Kitt Dataset
Quantitivate Result on JackRabbot Dataset