▬
Research
Medical Cybernetics and Bioinformatics
Affective Computing
For machines, the capability to recognize
user state (mental and affective disposition of the user) and adapt
their response accordingly has the potential to improve communication
accuracy of these technologies, and promote an engaging user
experience, which in turn can improve user's adoption of the
technologies.
The objective of my PhD thesis was to identify motion and postural features most salient to affective expressions
and to exploit the identified features to develop computational models for affective movement recognition and
generation that are robust to kinematic, interpersonal, and stochastic variations inherent to bodily expression
of affect. I have adapted a stochastic graphical modeling approach based on the hidden Markov model to devise
a novel hybrid discriminative-generative representation of body movements, augmented with a quantitative
encoding of the Laban1
components of the movements. I have developed a new quantification for Laban components in a collaboration with
a certified Laban analyst2. The resulting hybrid movement encoding
was then used for the automatic estimation of
affective expressions from movements as well as the adaptation of pre-defined motion paths in order to overlay
affective content. Furthermore, I have conducted a series of perceptual user studies to explore the impact of
kinematic embodiment and the observer's gender on affective movement perception.
Affective Movement Recognition
The below figure shows a schematic of the proposed recognition approach. The proposed approach derives a
stochastic model of the affective movement dynamics using hidden Markov models (HMM)s. The
resulting HMMs along with the forward algorithm are used to derive a Fisher score representation
of the movements, which is subsequently used to optimize affective movement recognition using the
linear support vector machine classification. The Fisher score for a movement is the partial derivative of the
log-likelihood of the movement with respect to parameters of the class-specific HMMs.