Aspects of gestural alignment in task-oriented dialogues
DOI:
https://doi.org/10.11649/cs.1640Keywords:
communicative alignment, intercultural communication, multimodal interactions, gesture function, gesture re-occurrence, gesture co-occurrenceAbstract
Interlocutors in a conversation influence each other in a number of dimensions. This process may lead to observable changes in their communicative behaviour. The directions and profiles of these changes are often correlated with the quality of interaction and may predict its success. In the present study, the gestural component of communication is scrutinised for changes that may reflect the process of alignment. Two types of task-oriented dialogues between teenagers are recorded and annotated for gestures and their features. We hypothesize that the dialogue task type (collaborative vs. competitive), as well as certain culture-specific properties of alignment that differ between German and Polish pairs, may significantly influence the process of communication. In order to explore the data and detect tendencies in gestural behaviour, automatised annotation mining and statistical exploration have been used, including a moving frame approach aimed at the investigation of co-occurring strokes as well as re-occurring strokes and their features. Significant differences between German and Polish speakers, as well as between the two dialogue types, have been found in the number of gestures, stroke duration and amplitude.
References
Beňuš, Š., Gravano, A., & Hirschberg, J. (2011). Pragmatic aspects of temporal accommodation in turn-taking. Journal of Pragmatics, 43(12), 3001-3027. https://doi.org/10.1016/j.pragma.2011.05.011 DOI: https://doi.org/10.1016/j.pragma.2011.05.011
Bonacchi, S., & Karpiński, M. (2014). Remarks about the use of the term "multimodality". Journal of Multimodal Communication Studies, 1, 1-7.
Brennan, S. E., & Clark, H. H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(6), 1482-1493. https://doi.org/10.1037/0278-7393.22.6.1482 DOI: https://doi.org/10.1037/0278-7393.22.6.1482
Campbell, N., & Scherer, S. (2010). Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity. In Proceedings of the 11th Annual Conference of the International Speech Communication Association. Retrieved from https://www.isca-speech.org/archive/interspeech_2010/index.html DOI: https://doi.org/10.21437/Interspeech.2010-689
Chartrand, T. L., & Bargh, J. A. (1999). The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76(6), 893-910. https://doi.org/10.1037/0022-3514.76.6.893 DOI: https://doi.org/10.1037/0022-3514.76.6.893
Chartrand, T. L., Maddux, W. W., & Lakin, J. L. (2006). Beyond the perception-behavior link: The ubiquitous utility and motivational moderators of nonconscious mimicry. In R. R. Hassin, J. S. Uleman, & J. A. Bargh (Eds.), The new unconscious (pp. 334-361). Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195307696.003.0014 DOI: https://doi.org/10.1093/acprof:oso/9780195307696.003.0014
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127-149). Washington, DC: American Psychological Association. https://doi.org/10.1037/10096-006 DOI: https://doi.org/10.1037/10096-006
Cleland, A. A., & Pickering, M. J. (2003). The use of lexical and syntactic information in language production: Evidence from the priming of noun-phrase structure. Journal of Memory and Language, 49(2), 214-230. DOI: https://doi.org/10.1016/S0749-596X(03)00060-3
Czoska, A., Klessa, K., Karpiński, M., & Nowikow-Jarmołowicz, E. (2015). Prosody and gesture in dialogue: Cross-modal interactions. In Proceedings of 4th Gesture and Speech in Interaction (GESPIN) Conference, Nantes, France (pp. 83-88). Retrieved from https://hal.archives-ouvertes.fr/hal-01195646/document
Fusaroli, R., Rączaszek-Leonardi, J., & Tylén, K. (2014). Dialog as interpersonal synergy. New Ideas in Psychology, 32(January-April), 147-157. https://doi.org/10.1016/j.newideapsych.2013.03.005 DOI: https://doi.org/10.1016/j.newideapsych.2013.03.005
Garrod, S., & Pickering, M. J. (2013). Interactive alignment and prediction in dialogue. In I. Wachsmuth, J. de Ruiter, P. Jaecks, & S. Kopp (Eds.), Alignment in communication: Towards a new theory of communication (pp. 193-204). Amsterdam: John Benjamins Publishing Company. (Advances in Interaction Studies, 6). https://doi.org/10.1075/ais.6.10gar DOI: https://doi.org/10.1075/ais.6.10gar
Giles, H., & Smith, P. (1979). Accommodation theory: Optimal levels of convergence. In H. Giles & R. N. St. Clair (Eds.), Language and social psychology (pp. 45-65). Baltimore, MD: University Park Press.
Giles, H., Taylor, D. M., & Bourhis, R. (1973). Towards a theory of interpersonal accommodation through language: Some Canadian data. Language in Society, 2(2), 177-192. https://doi.org/10.1017/S0047404500000701 DOI: https://doi.org/10.1017/S0047404500000701
Gravano, A., Levitan, R., Willson, L., Beňuš, Š., Hirschberg, J., & Nenkova, A. (2011). Acoustic and prosodic correlates of social behavior. In 12th Annual Conference of the International Speech Communication Association 2011 (Interspeech 2011). Retrieved from http://www.cs.columbia.edu/nlp/papers/2011/p95485.pdf DOI: https://doi.org/10.21437/Interspeech.2011-25
Guitar, B., & Marchinkoski, L. (2001). Influence of mothers' slower speech on their children's speech rate. Journal of Speech, Language, and Hearing Research, 44(4), 853-861. https://doi.org/10.1044/1092-4388(2001/067) DOI: https://doi.org/10.1044/1092-4388(2001/067)
Healey, P. G. T., Purver, M., & Howes, C. (2014). Divergence in dialogue. PLoS ONE, 9(6). https://doi.org/10.1371/journal.pone.0098598 DOI: https://doi.org/10.1371/journal.pone.0098598
Holle, H., & Rein, R. (2015). EasyDIAg: A tool for easy determination of interrater agreement. Behavior Research Methods, 47(3), 837-847. https://doi.org/10.3758/s13428-014-0506-7 DOI: https://doi.org/10.3758/s13428-014-0506-7
Jaecks, P., Damm, O., Hielscher-Fastabend, M., Malchus, K., Stenneken, P., & Wrede, B. (2013). What is the link between emotional and communicative alignment in interaction?: Towards a new theory of communication. In I. Wachsmuth, J. de Ruiter, P. Jaecks, & S. Kopp (Eds.), Alignment in communication: Towards a new theory of communication (pp. 205-224). Amsterdam: John Benjamins Publishing Company. (Advances in Interaction Studies, 6). https://doi.org/10.1075/ais.6.11jae DOI: https://doi.org/10.1075/ais.6.11jae
Karpiński, M. (2014). New challenges in psycholinguistics: Interactivity and alignment in interpersonal communication. Lingua Posnaniensis, 54(1), 97-106. DOI: https://doi.org/10.2478/linpo-2014-0006
Karpiński, M., Jarmołowicz-Nowikow, E., & Czoska, A. (2015). Gesture annotation scheme development and application for entrainment analysis in task-oriented dialogues in diverse cultures. In Proceedings of 4th Gesture and Speech in Interaction (GESPIN) Conference, Nantes, France (pp. 161-166). Retrieved from https://www.academia.edu/15517265/Karpi%C5%84ski_M._Jarmo%C5%82owicz-Nowikow_E._Czoska_A._2015._Gesture_annotation_scheme_development_and_application_for_entrainment_analysis_in_task-oriented_dialogues_in_diverse_cultures._Proceedings_of_GESPIN_2015_Conference_Nantes_France_161-166
Karpiński, M., Klessa, K., & Czoska, A. (2014). Local and global convergence in the temporal domain in Polish task-oriented dialogue. In N. Campbell, D. Gibbon, & D. Hirst (Eds.), Proceedings of the 7th International Conference on Speech Prosody (pp. 743-747). Retrieved from https://www.academia.edu/7151184/Karpinski_M._Klessa_K._Czoska_A._2014._Local_and_global_convergence_in_the_temporal_domain_in_Polish_task_oriented_dialogue._In_N._Campbell_D._Gibbon_and_D._Hirst_Eds._Speech_Prosody_7_2014_Dublin_pp._743-747 DOI: https://doi.org/10.21437/SpeechProsody.2014-137
Karpiński, M., & Klessa, K. (2018). Methods, tools and techniques for multimodal analysis of accommodation in intercultural communication. Computational Methods in Science and Technology, 24(1), 29-41. https://doi.org/10.12921/cmst.2018.0000006 DOI: https://doi.org/10.12921/cmst.2018.0000006
Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511807572 DOI: https://doi.org/10.1017/CBO9780511807572
Klessa, K., Karpiński, M., & Wagner, A. (2013). Annotation Pro - a new software tool for annotation of linguistic and paralinguistic features. In B. Bigi & D. Hirst (Eds.), Proceedings of TRASP (Tools and Resources for the Analysis of Speech Prosody) (pp. 51-54). Aix-en-Provence: Aix-Marseille Université.
Kousidis, S. (2010). A study of accommodation of prosodic and temporal features in spoken dialogues in view of speech technology applications (Doctoral thesis). Dublin Institute of Technology.
Kousidis, S., Dorran, D., Wang, Y., Vaughan, B., Cullen, C., Campbell, D., McDonnell, C., & Coyle, E. (2008). Towards measuring continuous acoustic feature convergence in unconstrained spoken dialogues. In Proceedings of Interspeech 2008 (pp. 1692-1695). DOI: https://doi.org/10.21437/Interspeech.2008-369
Kulesza, W. (2016). (Nie)świadomy kameleon: Analiza związku między stosowaniem niewerbalnej mimikry, uległością wobec tego procesu a (nie)świadomością. Psychologia Społeczna, 11(2(37)), 183-195. https://doi.org/10.7366/1896180020163705
Lausberg, H. (2013). NEUROGES - A coding system for the empirical analysis of hand movement behaviour as a reflection of cognitive, emotional, and interactive processes. In C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill, & S. Tessendorf (Eds.), Body-language-communication: An international handbook on multimodality in human interaction (Vol. 1, pp. 1022-1036). Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110261318.1022 DOI: https://doi.org/10.1515/9783110261318.1022
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago, IL: University of Chicago Press.
Mol, L., Krahmer, E., Maes, A., & Swerts, M. (2012). Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66(1), 249-264. https://doi.org/10.1016/j.jml.2011.07.004 DOI: https://doi.org/10.1016/j.jml.2011.07.004
Müller, C. (1998). Redebegleitende Gesten: Kulturgeschichte, Theorie, Sprachvergleich. Berlin: Berlin-Verl. Spitz. (Körper, Zeichen, Kultur, 1).
Pardo, J. S. (2006). On phonetic convergence during conversational interaction. The Journal of the Acoustical Society of America, 119(4), 2382-2393. https://doi.org/10.1121/1.2178720 DOI: https://doi.org/10.1121/1.2178720
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. The Behavioral and Brain Sciences, 27(2), 169-190. https://doi.org/10.1017/S0140525X04000056 DOI: https://doi.org/10.1017/S0140525X04000056
Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284-295. https://doi.org/10.1037/a0023419 DOI: https://doi.org/10.1037/a0023419
Richardson, D. C., Dale, R., & Tomlinson, J. M. (2009). Conversation, gaze coordination, and beliefs about visual context. Cognitive Science, 33(8), 1468-1482. https://doi.org/10.1111/j.1551-6709.2009.01057.x DOI: https://doi.org/10.1111/j.1551-6709.2009.01057.x
Sloetjes, H., & Wittenburg, P. (2008). Annotation by category: ELAN and ISO DCR. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008). Retrieved from http://pubman.mpdl.mpg.de/pubman/item/escidoc:60774:3/component/escidoc:60775/Sloetjes_2008_annotation.pdf
Truong, K. P., & Heylen, D. (2012). Measuring prosodic alignment in cooperative task-based conversations. In INTERSPEECH-2012 (pp. 843-846), Portland, OR. Retrieved January 31, 2018, from https://www.isca-speech.org/archive/archive_papers/interspeech_2012/i12_0843.pdf DOI: https://doi.org/10.21437/Interspeech.2012-190
Ward, A., & Litman, D. J. (2007). Automatically measuring lexical and acoustic/prosodic convergence in tutorial dialog corpora. In Workshop on Speech and Language Technology in Education. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.137.6046&rep=rep1&type=pdf
Ward, A., & Litman, D. J. (2007). Measuring convergence and priming in tutorial dialog. University of Pittsburgh.
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., & Sloetjes, H. (2006). ELAN: A professional framework for multimodality research. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006) (pp. 1556-1559). Retrieved from http://www.lrec-conf.org/proceedings/lrec2006
Downloads
Published
Issue
Section
License
Copyright (c) 2018 Maciej Karpiński, Agnieszka Czoska, Ewa Jarmołowicz-Nowikow, Konrad Juszczyk, Katarzyna Klessa
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
