Keywords

deception, deception detection, credibility, credibility assessment, multimodal behavior analysis

Abstract

This article outlines an approach for automatically extracting behavioral indicators from video, audio, and text and explores the possibility of using those indicators to predict human-interpretable judgments of involvement, dominance, tension, and arousal. We utilized two-dimensional spatial inputs extracted from video, acoustic properties extracted from audio and verbal content transcribed from face-toface interactions to construct a set of multimodal features.Multiple predictive models were created using the extracted features as predictors and human-coded perceptions of involvement, tenseness, and arousal as the criterion. These predicted perceptions were then used as independent variables in classifying truth and deception. Though the predicted values for perceptions performed comparably to human-coded perceptions in detecting deception, the results were not satisfying. Thus, the extracted multimodal features were used to predict deception directly. Classification accuracy was substantially higher than typical human deception detection performance. Through this research, we consider the feasibility and validity of the approach and identify how such an approach could contribute to the broader community.

Original Publication Citation

"Automatic, Multimodal Evaluation of Human Communication", Group Decision and Negotiation, Edition 4, Volume 19, Pages 367-389, Springer, 2010

Document Type

Peer-Reviewed Article

Publication Date

2009

Publisher

Group Decision and Negotiation

Language

English

College

Marriott School of Business

Department

Information Systems Management

University Standing at Time of Publication

Full Professor

Download

Share

COinS