Facial feature tracking is a key component of imaging ballistocardiography (BCG) where the displacement of facial key points needs to be accurately quantified to provide a good estimate of the heart rate. Skin feature tracking also enables video-based quantification of motor degradation due to dementia in Parkinson's disease.
Traditional computer vision algorithms like Scale Invariant Feature Transform (SIFT) or Speeded-Up Robust Features (SURF), and feature tracking, like the Lucas-Kanade method (LK) have long represented the state-of-the-art in efficiency and accuracy. However, common deformations, like affine local transformations or illumination changes, can cause them to fail. Over the past five years, deep convolutional neural networks have outperformed traditional methods for most computer vision tasks. We propose a pipeline for feature tracking that applies a convolutional stacked autoencoder to identify the most salient crops in an image and match them to a second image. The autoencoder learns to represent image crops into deep feature encodings specific to the object category it is trained on, making it better for application-specific cases. We train the autoencoder on facial images and validate its ability to track skin features using manually labeled face and hand videos. The tracking errors of distinctive skin features (moles) are so small that we cannot exclude that they stem from the manual labeling based on a χ2-test.
Our method achieved a lower mean error than the other methods in all but one scenario. More importantly, unlike all other tracking methods, our method was the only one to not diverge in any scenario. These results show that our method creates better feature descriptors for feature tracking, feature matching, and image registration than the traditional algorithms.
Jose Chang, Ph.D. Student at the Department of Mechanical Engineering, National Cheng Kung University.