Facial actions speak louder than words. Facial actions can reveal an individual’s emotion, intention, and physical condition making possible a variety of applications which include researching the market, human-robot interaction, sleepiness recognition, and clinical and developmental psychology research. Within this proposal, we investigate both supervised and without supervision methods to facial action discovery.

Supervised approaches aim to train and validate classifiers for facial action recognition. This is challenging, partly, for 2 significant reasons. First, classifiers must generalize to formerly unknown subjects that could differ markedly in behavior, facial morphology, and also the recording atmosphere. To deal with this issue, we advise Selective Transfer Machine (STM), a transductive learning way in which personalizes generic classifiers for facial expression analysis. By personalizing the classifier, STM has the capacity to generalize much better than condition-of-the-art methods to unseen subjects. Additionally, the STM framework can incorporate partially labeled data from the test subject.

Second, supervised learning typically uses hands-crafted, a priori features, for example Gabor, HOG and SIFT along with independent methods to classifier training (eg, SVM). Recent studies suggest that the alternative method of feature selection integrated by having an alternative learning paradigm (Deep Learning) may provide superior performance and help reduce or get rid of the problem of domain transfer. Given a restored number in excess of .5 million frames annotated within our GFT and BP4D+ datasets, this is actually the golden era with this exploration.

This thesis will test the hypothesis that Deep Learning enables greater precision in accordance with baseline SVMs and domain transfer approaches, normalizing by the amount of independent parameters.

A significant limitation of supervised approaches, including Deep Learning, may be the assortment of annotations, which may be time-consuming, error-prone, and restricted to recognition phenomena described in other contexts by observers. We explore the very first time, using without supervision methods for facial action discovery. Particularly, we introduce the most popular Event Discovery (CED) problem, which, within an without supervision manner, finds out correlated facial actions from some videos. A complete method of find such facial actions includes a quartic complexity in the size of videos, and therefore impractical. This thesis proposes a competent branch-and-bound (BB) way in which guarantees a worldwide optimal solution. We’ll evaluate CED in three human interaction tasks: video recorded three-person social interactions and parent-infant interaction, and motion taken movement. We hypothesize that CED may have moderate convergence with supervised approaches, and identify novel patterns in intra- and interpersonal actions occult to supervised approaches.

Thesis Committee
Fernando En Torre (Co-chair)
Jeffrey F. Cohn (Co-chair)
Simon Lucey
Deva Ramanan
Vladimir Pavlovic (Rutgers College)