Perceptions of temporal synchrony in multimodal displays

Abstract:

Current multimodal interfaces make use of several intra-modal perceptual judgements that help users "di-rectly perceive" information. These judgements help users organize and group information with little cog-nitive effort. Cross-modal perceptual relationships are much less commonly used in multimodal inter-faces, but could also provide processing advantages for grouping and understanding data across different modalities. In this paper we examine whether individuals are able to directly perceive cross-modal auditory and tactile temporal rate synchrony events. If direct perception is possible, then we would expect that indi-viduals would be able to correctly make these judgements with very little cognitive effort. Our results indi-cate that individuals have difficulty identifying when the temporal rates of auditory and tactile stimuli in a monitoring task are synchronous. Changes in workload, manipulated using a secondary visual task, resulted in changes in performance in the temporal synchrony task. We concluded that temporal rate synchrony is not a perceptual relationship that allows for direct perception, but further investigation of cross-modal per-ceptual relationships is required. Copyright 2011 by Human Factors and Ergonomics Society, Inc. All rights reserved.

Año de publicación:

2011

Keywords:

Fuente:

scopus