Imagery in language understanding
While Cognitive Linguists have long been proponents of a role for mental imagery in linguistic meaning (Casad 1982; Lakoff 1987; Langacker 1987), only recently has this notion accumulated systematic support from behavioral and neural imaging experiments, and started to become fully articulated in a formal and computationally implementable model. The basic idea is that in order to understand an utterance, a language comprehender mentally simulates or imagines its perceptual or motor content. For example, comprehending a sentence like “John kicked the ball over the fence” might entail motor imagery - like what it feels like to kick a ball - as well as visual imagery - such as what it looks like for a ball to be kicked over a fence. On this view, language is meaningful when it effectively evokes mental simulations of this type - when it recreates experiences of “being there” in the mind of a language user. Studies of various types are increasingly demonstrating that language comprehenders automatically and often unconsciously activate both perceptual and motor imagery in order to understand the content of language. Recent work has shown that people make use of particular parts of the visual field (Richardson et al 2003, Bergen To Appear) when a subject is processing simple declarative sentences whose meaning includes objects in those same areas, and that this visual imagination is reflected by actual eye movements (Spivey and Geng 2001). These linguistically evoked visual images include details about the orientation of objects (Stanfield and Zwaan 2001) and their shape (Zwaan et al 2002).
Motor imagery - imagining what it would be like to perform actions described is also activated by language (Glenberg and Kaschak 2002; Bergen et al. 2003). Several recent studies show that parts of motor and pre-motor cortex areas associated with specific body parts become active in response to motor language referring to those body parts. Using behavioral and neurophysiological methods, Pulvermuller et al. (2001) and Hauk et al. (2004) found that verbs associated with different effectors were processed at different rates and in different motor cortex regions. In particular, when subjects perform a lexical decision task with verbs referring to actions involving the mouth (chew), leg (kick), or hand (grab), the motor cortex areas responsible for mouth versus leg versus hand motion received more activation, respectively. Tettamanti et al. (ms.) have also shown through an imaging study that passive listening to sentences describing mouth versus leg versus hand motions activates different parts of pre-motor cortex.
Behavioral methods (Glenberg and Kashak 2002; Bergen et al. 2003) have provided convergent evidence for the role of motor structures in understanding language that encodes actions.The findings from Glenberg and Kaschak’s approach show that when subjects are asked to perform a physical action in response to a sentence, such as moving their hand away from or toward their body, it takes them longer to perform the action if it is incompatible with the motor actions described in the sentence. This suggests that while processing language, we perform motor imagery, using neural structures dedicated to motor control. Similarly, Bergen et al’s approach has shown that subjects have more trouble deciding that a verb is not a good descriptor for a picture of an action when the action it describes uses the same effector (hand, foot, or mouth) as the action depicted. This finding demonstrates that detailed motor knowledge constitutes part of word meaning.
If, as the various studies described above indicate, understanding language entails performing visual and motor imagery, then this suggests that a complete understanding of how language is used for any given purpose, including its formal configurations, is only possible in a model that takes into account the human cognitive system that linguistic knowledge is embedded in. This system includes motor and perceptual knowledge, based on experiences interacting with the world. An approach to language that can accommodate this constraint along with the finding that grammatical constructions are bound to associated pragmatics, must thus be embodied it must allow the linguistic system to be shaped by the bodily, physical, and social environment in which it is embedded.