MONAH: Multi-Modal Narratives for Humans to analyze conversations
Joshua Y. Kim, Kalina Yacef, Greyson Kim, Chunfeng Liu, Rafael Calvo, Silas Taylor
Dialogue and Interactive Systems Long paper Paper
You can open the pre-recorded video in separate windows.
Abstract:
In conversational analyses, humans manually weave multimodal information into the transcripts, which is significantly time-consuming. We introduce a system that automatically expands the verbatim transcripts of video-recorded conversations using multimodal data streams. This system uses a set of preprocessing rules to weave multimodal annotations into the verbatim transcripts and promote interpretability. Our feature engineering contributions are two-fold: firstly, we identify the range of multimodal features relevant to detect rapport-building; secondly, we expand the range of multimodal annotations and show that the expansion leads to statistically significant improvements in detecting rapport-building.
NOTE: Video may display a random order of authors.
Correct author list is at the top of this page.