The best way to achieve the flexible interpretative capacity needed for frame- shifting may be to adopt an empirically inspired architecture that is based on dynamic internal imagery (Bergen and Coulson 2006). In such models, language interpretation involves the creation of internal simulations of events that include sensory, motor, and affective dimensions. Barsalou (1999) has suggested that background knowledge is stored as perceptual symbols, schematic representations of perceptual experience stored around a common frame that promotes schematized simulations. Perceptual symbols are thus grounded in experience as the brain captures states across modalities and integrates them into a multimodal representation stored in memory. This multi-modal representation is later reactivated to simulate relevant aspects of perception, action, and introspection.
Whereas frames have traditionally been understood as amodal knowledge structures that result from a distillation of experience, perceptual symbols are understood as having modal characteristics. Perceptual symbols recruit brain areas involved in the acquisition of the relevant concepts, and have some characteristics of analogue representations. However, perceptual symbols have also been argued to be schematic enough to implement standard symbolic functions, such as type-token relationships, recursion, and inference (Barsalou 1999).
As in traditional implementations of frames, perceptual symbols capture causal and relational information needed for frame-shifting. Moreover, because they are the product of neural learning mechanisms, perceptual symbols incorporate many of the features of sub-symbolic frames, such as partial and probabilistic activation patterns. Perhaps most importantly, the analogue character of perceptual symbols allows for novel combinations based on the affordances of the constituent concepts (Glenberg and Robertson 2000).
This model is supported by research indicating that the neural systems responsible for performing actions or perceiving percepts are also recruited for linguistically inspired simulations (Barsalou 2008). Consistent with the embodiment and elaboration assumptions in the space structuring model, recent findings suggest that language processing utilizes the perceptual and motor systems as internal models that allow for the construction of subjective experiences in the absence of motor action or perceptual input (Pecher and Zwaan 2005).
Bergen and Coulson (2006) argue that a simulation based model might account for the joke about the pool in (14). Because our experiences with pools almost without exception include water, water will automatically be activated in mental simulations that involve pools. Our experience with diving, by contrast, presumably involves some cases of landing on a solid surface, thus enabling us to viscerally imagine diving into a pool with water.