MULTIMODALITY

Imagine the pre-Babel tongue in which every word was perfect vocal expression for the thing it named, the thing behind the play of appearances and shared illusions. Following our race’s arrogance to the deity, so the story goes, our pillar was shattered and with it the language of Adam.

The Good Book compels us to decry the loss of this noumenal tongue for the communicative difficulties which ensued, yet we might instead laud the world’s linguistic diversity as a remarkable demonstration of our brains’ capabilities. In any case, the real devastation in the confusion of languages is that words became mechanisms of expression, rather than the magically expressive objects we feel them to be.

Another story goes that Cratylus, friend of Aristotle and lover of words, recognized this duplicitous quality of language and became so disillusioned that he gradually descended into lifelong muteness. I suppose in some way unknown to me this passes for wisdom, but in my own estimation the man was a fool. He need not have been so distraught. There are fortunately still extant elements of the pre-Babel speech in our contemporary languages.

SOUND SYMBOLISM: REMNANTS OF THE PRE-BABEL TONGUE OR BRAIN ARCHITECTURE?
Shirou Kunihira of Loma Linda University in California presented American college students with Japanese antonym pairs and their English translations. Wide-Narrow, Sour-Sweet, Fat-Thin, for example. He found that using nothing but their ears and their instincts, the cohort could distinguish their meanings at rates significantly greater than chance (p < .01).

These findings suggest that there are non-arbitrary relations between a word’s sound and its semantics—that the sound of a word hints at its meaning. Lynne Nygaard of Emory University replicated Kunihira’s findings with other languages. Charting the phonetic qualities of all these antonym pairs, he unearthed an odd fact: across languages,  similar descriptive words share phonological characteristics.

Thus words meaning big, or fast, or round on average show phonological consistencies across languages. Furthermore, these phonological features reliably predicted monolingual subjects’ judgments of these words’ meanings. Experientially, these words have a certain inner bigness or fastness or roundness to them, which allows even Americans to distinguish them from their antonyms above chance when experimentally prodded to do so. This cross-linguistic sound symbolism hints at non-arbitrary links between sound and meaning. Take that, de Saussure!

DEFINITELY BRAIN ARCHITECTURE
This sound symbolism, being cross-cultural, can come from nowhere else but what we humans all share in common—that is, brain architecture. Furthermore, it inherently spans sense modalities in a not entirely intuitive way. For this reason, the existence of cross-modal, non-arbitrary sound symbolism can shed light on the evolution of language.

To demonstrate this concretely, two shapes:


One is named Kiki and the other Bouba. You should have been able to guess which was which (regardless of your native tongue). To explain why, we must delve into the wrinkles of grey matter that fill the skull, to a region called the Left Angular Gyrus. This valuable cortical device plays an important role, being at the crossroads of centers for processing vision, hearing and touch. It integrates these various senses and presents them to our consciousness in a cohesive way. Thus, it might be illuminating to examine what goes awry when lesions muck up this region’s functioning.

One neuroscientist, V.S. Ramachandran, did just that. He gathered four patients with speech aphasia and asked them to identify Kiki from Bouba. They could not do it. Neither could these subjects grasp any but the literal meanings of metaphors. To them, reaching for the stars was a conceivable yet futile task. The proverb “the grass is always greener” struck them as sheer fallacy. Clearly, then, the Kiki/Bouba effect is grounded in multimodal integration in the brain, and plausibly metaphors are too. This would mean that metaphors are not merely linguistic quirks, or tricks of word play, but actually reflect the integration of different senses at the neural level. One cannot think Kiki/Bouba away; it is a function of our brain architecture and it forcefully raises the question of how multimodal integration might structure our cognition in other fundamental ways.

Ramachandran then engaged this question with another question. At the whiff of some rotting odor, he reasoned, we primates scrunch our faces in disgust. This is quite a natural response, and perhaps conveys to the community at large that this substance is distasteful, maybe poisonous (avoid it!). But then, he notes, cross-culturally, humans reacting to “disgusting” behaviors contort their faces in the same fashion as they would with despoiled food. The burning questions: why is the physiological disgust reaction the same as the social one? Furthermore, this is reflected in human speech the world round. In metaphor so implicit as to be unnoticed, languages—even those entirely unrelated to each other—often refer to sordid individuals engaged in immoral behavior as disgusting, or rotten. The existence of this metaphor hints at a universal explanation, to be found in brain structure. Ramachandran speculates that the orbito-frontal cortex, which handles disgust from the nose and tongue, also manages moral disgust. If this is true, then perhaps the icky we feel at the mention of pederasty stems from the need to avoid food poisoning.

Kiki and Bouba do not make sense, except from the perspective of multimodal cognition. I have quizzed people of different philosophical stripes on these shapes, even nihilists, and they all fit the pattern. I then ask them why. Postmodernists hastily refer to culture. They might say: “Do not our K’s have the angles of Kiki’s starburst? And does not the circle of the O and the oblongity of the U mimic Bouba’s roundness?” When I query general responders as to why they pick certain chapes, they say “Well, Bouba sounds rounder, and Kiki sounds sharper.”

Ramachandran thinks the effect might come about from the fact that these shapes mimic the ones our mouths take in making them. In saying Bouba, the labia orbiculates, and with Kiki, the tongue adopts angular formations on the palate. But to stick with the main point: vision, sound, and proprioception, the conscious experiences of which are vastly different and generally do not seem to bleed into each other are actually deeply and unconsciously linked in our brains in ways that fundamentally structure consciousness. Because of this, “Kiki” can sound sharp in a way that can be drawn, and “Bouba” can sound round. Very similar multimodal connections in the brain might be responsible for the capacity for metaphorical thinking, as evidenced by the fact that a single lesion can impair the ability to do both.

It has been asserted that metaphors arising from embodied experience fundamentally structure the way we think culturally. However, insights from the neuroscience of cross-modal cognition hint that cognition itself might be inherently metaphorical from the start—that even a single moment of thought presupposes a synthesis and integration of different sense modalities.

SANDEEP NAYAK B’12 eats papist babes.

One Comment

  1. yeah right says:

    i’ll believe when i see it

    Reply

Leave a Comment

Powered by WordPress | Deadline Theme : An AWESEM design