Skip to Main content Skip to Navigation
Journal articles

Multimodal stress detection from multiple assessments

Abstract : Stress is a complex phenomenon that impacts the body and the mind at several levels. It has been studied for more than a century from different perspectives, which result in different definitions and different ways to assess the presence of stress. This paper introduces a methodology for analyzing multimodal stress detection results by taking into account the variety of stress assessments. As a first step, we have collected video, depth and physiological data from 25 subjects in a stressful situation: a socially evaluated mental arithmetic test. As a second step, we have acquired 3 different assessments of stress: self-assessment, assessments from external observers and assessment from a physiology expert. Finally, we extract 101 behavioural and physiological features and evaluate their predictive power for the 3 collected assessments using a classification task. Using multimodal features, we obtain average F1 scores up to 0.85. By investigating the composition of the best selected feature subsets and the individual feature classification performances, we show that several features provide valuable information for the classification of the 3 assessments: features related to body movement, blood volume pulse and heart rate. From a methodological point of view, we argue that a multiple assessment approach provide more robust results.
Document type :
Journal articles
Complete list of metadata

Cited literature [65 references]  Display  Hide  Download
Contributor : Gestionnaire Hal-Upmc <>
Submitted on : Wednesday, December 14, 2016 - 3:39:08 PM
Last modification on : Thursday, July 8, 2021 - 3:48:58 AM
Long-term archiving on: : Wednesday, March 15, 2017 - 1:36:44 PM


Files produced by the author(s)



Jonathan Aigrain, Michel Spodenkiewicz, Severine Dubuisson, Marcin Detyniecki, David Cohen, et al.. Multimodal stress detection from multiple assessments. IEEE Transactions on Affective Computing, Institute of Electrical and Electronics Engineers, 2016, PP (99), pp.1 - 1. ⟨10.1109/TAFFC.2016.2631594⟩. ⟨hal-01416517⟩



Record views


Files downloads