Audiovisual Perception

6 Intermodal Analogy

In addition to modal qualities that occur exclusively for just one sense (for example, pitch for the sense of hearing, color for the sense of sight), there are also amodal (or intersensory)[12] qualities that are perceived by multiple senses. The psychologist Heinz Werner examined such phenomena in detail as early as the 1960s. When we say that a tone is strong or weak, that a pressure is strong or weak, we are no doubt referring in all cases to a property that is the same in all these sensory domains. Recent research has now shown that there are doubtless many more properties than psychology previously assumed that, like intensity, can be called intersensory.[13] Werner listed the following properties with which it is possible to establish analogies across the boundaries between the senses: intensity, brightness, volume, density, and roughness. According to Michel Chion, these and other amodal qualities are in fact at the center of our perception.[14]

Using these dimensions, it is possible to relate sensory impressions of widely different modalities to one another — that is, to create intermodal analogies. The process used, in contrast to multimodal integration, occurs consciously and actively, since one is searching for a criterion of comparison that is usually found in one of the amodal dimensions. For example, the question ‘What sound goes with this color?’ can be decided on the basis of brightness. The formation of such analogies is influenced by the context, since the color or loudness of an object are not absolute values but can only be assessed in relation to their surroundings. Intermodal analogies have features that tend to be regular from person to person (small interpersonal variation), whereas synesthetic correspondences are very different (large interpersonal variation). In general, the dimensions of brightness and intensity seem to be of central importance to the formation of intermodal analogies.[15]

The psychologist Albert Wellek examined similar connections in the 1920s. Experimental testing enabled him to compile a list of six correspondences — so-called primeval synesthesias), which in his view could be found among all peoples at all times and hence were fixed in human perception: thin forms correspond to high pitches, thick forms to low pitches, and so on. The historically demonstrable universality of these simplest sensory parallels go so far that everyone, even today, will consider all six correspondences valid and intelligible, at least in one of the given forms.[16] Our Western notation clearly corresponds to these primeval synesthesias, since, for example, the pitches are depicted, by visual analogy, as high or low, which makes intuitive sense to us.

In my view, the terms amodal and intersensorial are used synonymously in the literature.  
The association of pitch with color via the dimension of brightness is probably the only meaningful analogy that can be made between these two domains. A familiar procedure used by subjects seeking to correlate different sensory domains via the dimension of intensity is ‘cross-modality matching.’ — Trans. N. W..  
1
2
3
4
5
6
7