The influence of input-modality and output-modality on multitasking during language processing, funded by the DFG

  • Funding agency: German Research Foundation, DFG
  • Applicant: Dr. Andrea M. Philipp
  • Responsible staff member: M.Sc. Simone Schäffner
  • Duration: 24 months, 2015-2017


Performing verbal tasks always requires a combination of input and output modalities. That is, speech perception is only possible via the auditory or visual input modality, for instance for the perception of spoken or written language, and speech production always involves the vocal or manual output modality, for instance for the production of spoken or written language. The aim of the present research program is to investigate the role of different input and output modalities as well as the influence of different input-output modality-combinations in monolingual and in bilingual situations. The present program is a continuation of the DFG-Projekts PH 156/3-1, in which we could already find modality-specific influences during language processing. On the one hand, there were input-output-modality compatibility-effects during modality-switching experiments. Switching between compatible modality combinations, that is auditory-vocal and visual-manual, led to lower switch costs than switching between incompatible combinations, that is auditory-manual and visual-vocal) during language processing. On the other hand, there were modality-specific effects during language-switching experiments. We found first indications of a bimodal advantage in terms of better performances during language switching, whenever there was an assignment of the different languages to separate output modalities. In the present research program, we deepen both areas of modality-specific influences during language processing. In a series of modality-switching experiments, we examine the question of whether there are quantitative or qualitative differences of modality-compatibility effects dependent on different kinds of language-specific tasks. On the other hand, we investigate the framework conditions of the bimodal advantage in a series of language-switching experiments. Furthermore, both areas will be combined. For example, we investigate modality-compatibility effects during language-switching experiments as a first transfer of the knowledge about modality compatibility from monolingual into bilingual situations.

Selected references

Schaeffner, S., Koch, I., & Philipp, A. M. (2015). The role of sensory-motor modality compatibility in language processing. Psychological Research. Advance online publication.

Schaeffner, S., Koch, I., Philipp, A.M. (submitted). Semantic effects on sensory-motor modality switching: Evidence from two language-related modality-switching experiments.

Selected conference contributions

Schaeffner, S., Koch, I., Philipp, A.M. (2015). Explicit and implicit modality effects in language processing. 19th Meeting of the European Society for Cognitive Psychology (ESCOP 2015), Paphos, Cyprus.

Schaeffner, S., Fibla, L., & Philipp, A.M. (2015). Is there a bimodal advantage in language switching? 57. Tagung experimentell arbeitender Psychologen (TeaP), Hildesheim, Deutschland.

Schaeffner, S., Philipp, A.M., & Koch, I. (2014). Input-output modality compatibility in language processing. 56. Tagung experimentell arbeitender Psychologen (TeaP), Gießen, Deutschland.

Philipp, A.M., Schaeffner, S., Stephan, D.N., & Koch, I. (2013). Modality compatibility in a semantic categorization task. Embodied and Situated Language Processing Conference (ESLP), Potsdam, Deutschland.