Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface - Sorbonne Université Accéder directement au contenu
Article Dans Une Revue ACM Transactions on Applied Perception Année : 2016

Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface

Résumé

Audio, visual, and proprioceptive actions are involved when manipulating a graphic tablet musical interface. Previous works suggested a possible dominance of the visual over the auditory modality in this situation. The main goal of the present study is to examine the interferences between these modalities in visual, audio, and audiovisual target acquisition tasks. Experiments are based on a movement replication paradigm, where a subject controls a cursor on a screen or the pitch of a synthesized sound by changing the stylus position on a covered graphic tablet. The experiments consisted of the following tasks: (1) a target acquisition task that was aimed at a visual target (reaching a cue with the cursor displayed on a screen), an audio target (reaching a reference note by changing the pitch of the sound played in headsets), or an audiovisual target, and (2) the replication of the target acquisition movement in the opposite direction. In the return phase, visual and audio feedback were suppressed. Different gain factors perturbed the relationships among the stylus movements, visual cursor movements, and audio pitch movements. The deviations between acquisition and return movements were analyzed. The results showed that hand amplitudes varied in accordance with visual, audio, and audiovisual perturbed gains, showing a larger effect for the visual modality. This indicates that visual, audio, and audiovisual actions interfered with the motor modality and confirms the spatial representation of pitch reported in previous studies. In the audiovisual situation, vision dominated over audition, as the latter had no significant influence on motor movement. Consequently, visual feedback is helpful for musical targeting of pitch on a graphic tablet, at least during the learning phase of the instrument. This result is linked to the underlying spatial organization of pitch perception. Finally, this work brings a complementary approach to previous studies showing that audition may dominate over vision for other aspects of musical sound (e.g., timing, rhythm, and timbre).
Fichier principal
Vignette du fichier
TAP-2016-a10-Perrotin-dAlessandro.pdf (661.23 Ko) Télécharger le fichier
Origine : Publication financée par une institution
Loading...

Dates et versions

hal-01672241 , version 1 (23-01-2018)

Identifiants

Citer

Olivier Perrotin, Christophe d'Alessandro. Seeing, listening, drawing: interferences between sensorimotor modalities in the use of a tablet musical interface. ACM Transactions on Applied Perception, 2016, 14 (2), pp.1 - 19. ⟨10.1145/2990501⟩. ⟨hal-01672241⟩
114 Consultations
120 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More