Home Language & Literature COGNITIVE APPROACH TO NATURAL LANGUAGE PROCESSING
The Reverse Association Task
Free word associations are the words human subjects spontaneously come up with upon presentation of a stimulus word. In experiments comprising thousands of test subjects, large collections of associative responses have been compiled. In previous publications, it was shown that these human associations can be resembled by statistically analyzing the co-occurrences of words in large text corpora. In this chapter, we consider the reverse question, namely whether the stimulus can be predicted from the responses. We call this reverse association task and present an algorithm for approaching it. We also collected human data on the reverse association task, and compared them with the machine-generated results.
Word associations have always played an important role in psychological learning theory, and have been investigated not only in theory, but also in experimental work where, for example, such associations were collected from human subjects. Typically, the subjects are given questionnaires with lists of stimulus words, and were asked to write down for each stimulus word the spontaneous association which first came to mind. This led to collections of associations, the so-called association norms, as exemplified in Table 4.1. Among the best known association norms are the Edinburgh Associative Thesaurus (EAT) [KIS 73], the Minnesota Word Association Norms [JEN 70, PAL 64] and the University of South Florida Free
Chapter written by Reinhard Rapp.
Association Norms [NEL 98]. More recently, attempts have been made to use crowd sourcing methods for collecting associations in various languages (Jeux de mots1 and Word Association Study ). In this way, researchers are able to collect much larger datasets than was previously possible.
Association theory, which can be traced back to Aristotle in ancient Greece, has often stated that our associations are governed by our experiences. For example, more than a century ago, William James [JAM 90] formulated this in his book, The Principles of Psychology, as follows:
“Objects once experienced together tend to become associated in the imagination, so that when any one of them is thought of, the others are likely to be thought of also, in the same order of sequence or coexistence as before. This statement we may name the law of mental association by contiguity.”
This citation is talking of objects, but the question arose whether for words the same principles might apply, and with the advent of corpus linguistics, it was possible to verify this experimentally by looking at the distribution of words in texts. Among the first to do so were [CHU 90], [SCH 89] and [WET 89].
Their underlying assumption was that in text corpora, strongly associated words often occur in close proximity. This is actually confirmed by corpus evidence: Figure 4.1 assigns to each stimulus word position 0, and displays the occurrence frequencies of its primary associative response (most frequent response as produced by the test persons) at relative distances between -50 and +50 words. However, to give a general picture and to abstract away from idiosyncrasies, the figure is not based on a single stimulus/response pair, but instead represents the average of 100 German stimulus/ response pairs as used by Russell and Meseck [RUS 96]. The effect is in line with expectations: the closer we get to the stimulus word, the higher the chances that the primary associative response occurs. Only for distances of plus or minus one, there is an exception, but this is an artifact because content words are typically separated by function words, and among our 100 primary responses there are no function words. In addition, test persons typically select content words only.
While such considerations are the basis underlying our work, in this chapter, the focus is on whether it is possible not only to compute the responses from the stimulus, but also to compute the stimulus from the responses. To the best of our knowledge, this has not been attempted before in a comparable (distributional semantics) framework, and therefore we are not aware of any directly related literature.
However, this task is somewhat related to the computation of associations when given several stimulus words simultaneously, which is sometimes referred to using the terms multi-stimulus- or multiword associations [RAP 08], or remote association test (RAT). A recent notable publication on the RAT, which gives pointers to other related works, is Smith et al. [SMI 13]. It applies this methodology on problems that require consideration of multiple constraints, such as choosing a job based on salary, location and work description. Another one is Griffiths et al. [GRI 07], which assumes that concept retrieval from memory can be facilitated by inferring the gist of a sentence, and using that gist to predict related concepts and disambiguate words. It implements this by using a topic model.
Table 4.1. Top 10 sample associations for three stimulus words as taken from the Edinburgh Associative Thesaurus. The numbers of subjects responding with the respective word are given in brackets
Our approach differs from this previous work in that it focuses on a related but different and particularly well-defined task. In our approach, we have eliminated all (for this particular task) unnecessary sophistication, such as Latent Semantic Analysis (which we used extensively in previous work) or Topic Modeling, resulting in a simple yet effective algorithm. For example, [GRI 07] reports 11.54% correctly predicted first associates. Rapp [RAP 08] presents a number of evaluations using various corpora and datasets, but with all results below 10%. The above-mentioned paper by Smith et al. [SMI 13] gives no such figures at all. In comparison, the best results presented here are at 54% (see section 4.3.3). It should be emphasized, however, that all comparisons have to be taken with caution, as there is no commonly used gold standard for this, and hence all authors used different test data, and different corpora. Note also that, in contrast to the related work, our focus is on the novel reverse association task, which gives us test data of unprecedented quality and quantity (as any word association norm can be used), but for which the previous test data is unsuitable as it relates to a somewhat different task.
Figure 4.1. Occurrence frequency f of a primary response at distance d from a stimulus word, averaged over 100 stimulus/response pairs [RAP 96]
This paper, which is a substantially extended version of [RAP 13], is structured as follows: we first look at how we compute associations for single stimulus words. This lays the basis for the second part where we reverse our viewpoint and compute the stimulus word from its associations. In the third part, we want to find out how humans perform on the reverse association task. For this purpose, we conducted a reverse association experiment where human responses were collected. Finally, we compare the performances of man and machine.
|< Prev||CONTENTS||Next >|