Home Language & Literature COGNITIVE APPROACH TO NATURAL LANGUAGE PROCESSING
Discussion, conclusions and outlook
Reverse associations by a human
Associating with several given words, as required in the reverse association task, is not easy. This has been remarked by several test persons and is also confirmed by a considerable number of omissions in the test sheets. For several reasons, it is difficult to come up with an expected word:
Owing to these difficulties, we expected low-performance figures, and this expectation was confirmed.
As mentioned in section 4.4.2, we had not disclosed the nature of the dataset to the test subjects, i.e. did not tell them that the underlying idea is the reverse association task. Alternatively, we could have informed people about this and asked them to come up with a word with which they would associate each of the five given words. But this would probably have been conceived as an even more sophisticated task, potentially blocking spontaneity. We also tend to think that, although associations can occasionally be asymmetric, assymetry is likely not to have a decisive effect in our scenario. However, this is certainly a question which requires further investigation
While in previous studies involving multi-stimulus associations, human performance had always been much better than what simulation programs produced (see e.g. [RAP 08]), this is not the case here. In the CogALex shared task, a number of teams had tested their algorithms, which were mostly based on the analysis of word co-occurrences in large text corpora, on the very same dataset, and achieved performances of up to 30.45%. In the current paper, on this dataset, we presented a very similar result. These results are much better than the human performance of our non-native speakers shown in Table 4.6. Although native speakers can be expected to do better, we think that it will be challenging to outperform the automatic results. This might well mean that in one of its core disciplines, namely association, human intelligence is not better than a machine, although of course further investigation is required to confirm this finding.
|< Prev||CONTENTS||Next >|