Desktop version

Home arrow Philosophy

  • Increase font
  • Decrease font


<<   CONTENTS   >>

Ecological Validity of Learning Tasks

A consequence of using eye movement and brain imaging methods is that this research is almost always conducted in a lab setting. With regard to eye movement research, screen-based eye-trackers or eye-tracking glasses can be used. A limitation of using eye movements with a screen-based eye tracker (i.e., the eye tracking equipment is integrated into the computer screen) to examine students’ processing strategies, is that one is only able to investigate students’ mental or covert processing strategies. Overt processing strategies produce physical records, such as text notes, summaries, mind maps, while covert strategies do not produce this kind of record and refer to internal mental learning processes (Kardash & Amlund, 1991; Merchie & Van Keer, 2014). Eye movement glasses are worn as normal glasses and, thus, allow the students to freely inspect and interact with all kinds of learning materials. However, there is a downside on using eye movement glasses because the data is more complex to analyze than with screen-based eye trackers, and the eye movement glasses have a lower sampling frequency than screen-based eye trackers (Holmqvist & Andersson, 2017). The computer overlays the gaze data onto the scene video and shows with a marker where the participant is looking. Even if there is a data file, the coordinates of the data refer to the positions in the video and not to positions in the text. Each dataset for each participant will thus be different, which makes it less straightforward to analyze the data afterwards (Holmqvist & Andersson, 2017). Another important aspect of eye movement research is the eye tracker calibration. During the calibration procedure, a dot is moving on the screen and the participant needs to follow the dot with his/her eyes. After this procedure, it can be verified whether the eye gaze can be recorded on different points on the screen. It is important that the eye-tracker be calibrated before and during longer reading processes in order to assure good data quality (Holmqvist et al., 2011). As a consequence, either shorter learning tasks are used or longer learning tasks are used but need to be interrupted for a calibration procedure. Therefore, questions can be raised on the ecological validity of the learning tasks used in eye movement research.

Concerning brain imaging methods, even greater issues arise with regard to the ecological validity of learning tasks. A typical characteristic of fMRI research is the highly controlled environment (Varma et al., 2008). The magnetic resonance imaging (MRI) scanner is a very noisy environment in which subjects have to lie still and are not allowed to move (De Smedt, 2014; Huettel et al., 2014). During an experiment, participants see stimuli projected on a small hanging mirror and are mostly asked to respond by pressing buttons (Varma et al., 2008). These practical constraints result in the use of restricted paradigms in which very elementary tasks, such as learning words, are used (De Smedt, 2014; Howard-Jones, Ott, van Leeuwen, & De Smedt, 2014; Willems, 2015). Thus, fMRI research is limited on what it can tell us about the contextual aspects that are crucial for learning (Varma et al., 2008). Some studies have already moved into studying differences in reading strategies at the text level (Moss & Schunn, 2015; Moss, Schunn, Schneider, & McNamara, 2013; Moss, Schunn, Schneider, McNamara, & Vanlehn, 2011). However, in our opinion fMRI may be a bridge too far at the moment to examine differences in processing strategies at the text level. We believe this first because, in the studies of Moss and colleagues, participants were not able to look back at the short paragraphs (Moss & Schunn, 2015; Moss et al., 2011, 2013). Short paragraphs of two to four sentences were presented only at one point in time. However, there is a vast tradition of eye movement research that shows that look back behavior is a crucial aspect for strategic and deeper cognitive processing when processing words, sentences and texts (Ariasi et al., 2017; Holmqvist et al., 2011; Jarodzka & Brand-Gruwel, 2017; Penttinen et al., 2013; Rayner, 2009). Especially in text comprehension, it is not only crucial to look back within words or sentences but also within and between paragraphs (Hyona et al., 2003; Jarodzka & Brand-Gruwel, 2017). However, this is not (yet) possible in fMRI research. A second reason why we believe it is a bridge too far, is that we first need to gain more insight into how processing strategies are reflected in behavioral measures before we move to neuroscientific research. Processing strategies were mostly measured with self-report questionnaires at a more general level by which the learning task was often neglected (Dinsmore & Alexander, 2012; Fryer, 2017; Vermunt & Donche, 2017). It is only more recently that think-aloud protocols (Dinsmore & Alexander, 2016; Dinsmore & Zoellner, 2018) and eye movement registration have been used to investigate differences in processing strategies during text learning (Catrysse et al., 2016,2018; Catrysse et al., 2018). We agree with the suggestion of Varma et al. (2008) that, in order to move beyond the practical constraints of the MRI scanner, more contextual rich learning tasks could be given outside the scanner. However, in our opinion, fMRI will become an online outcome measure and not an online learning process measure. If students get more complex tasks outside the scanner and are then scanned during memory tests as suggested by Varma et al. (2008), it is not a pure online measure that captures learning during the learning process.

Analysis of Complex Data

A characteristic of psychophysiological measures, such as eye movement registration and brain imaging, is that these techniques sample information with a high temporal precision (Holmqvist & Andersson, 2017; Huettel et al., 2014). This results in huge datasets per subject in comparison with the more traditional methods used in educational sciences. This calls for using other analytical techniques in order to take the complexity of the data into account. With regard to analyzing the eye movement data on learning from a text, we want to emphasize the strengths of applying mixed effects models. Although this analytical technique was described in good practices by different researchers in 2008 (Baayen, 2008; Baayen, Davidson, & Bates, 2008; Quene & van Den Bergh, 2008), it is only very recently that eye tracking researchers have adapted this analysis technique to examine eye movement data on reading/learning from texts (Ariasi et al., 2017; Catrysse et al., 2018; Catrysse et al., 2018). Mixed effects models offer several advantages in comparison with other techniques, such as repeated measures ANOVA (Baayen, 2008; Baayen et al., 2008; Quene & van Den Bergh, 2008). A first advantage is that mixed effects models offer more statistical power by conducting analysis on the sentence level instead of on the subject level. Furthermore, mixed effects models have a lower risk of capitalization on chance, i.e., type I error (Quene & van Den Bergh, 2008). Other advantages of mixed effects models include, among others, better methods for treating continuous responses and better methods for modeling heteroscedasticity and non-spherical error variance. In addition, by treating subjects, sentences and texts (if applicable) as crossed items, results can be jointly generalized over similar subjects, sentences and texts (Baayen, 2008; Quene & van Den Bergh, 2008). Subjects, sentences and texts are sampled from a larger population and it is thus important to take this into account in the analysis. Thus, we highly recommend applying mixed effects models for the analysis of eye movement data.

 
<<   CONTENTS   >>

Related topics