Desktop version

Home arrow Philosophy

  • Increase font
  • Decrease font

<<   CONTENTS   >>

Remaining issues in the qualitative analysis of verbal report data

As reviewed in this chapter, and in relation to our own research, qualitative analysis of verbal report data has many challenges. These challenges, to name a few, include identifying and accounting for the contextual influences of tasks on verbal reporting, and acknowledging the sometimes ambiguous boundaries within which verbal report data can describe a particular type of strategic processing of interest, but beyond which the inference of strategy or process is not warranted. Finally, the complex argumentative work on a claim made for strategic processing ought to be substantiated by evidence of verbal data and theoretical reasoning to bond the claim and evidence, which should result in a robust understanding of the particular phenomenon being studied. However, we counter that these issues can be addressed with detailed accounts of the variables and their influences involved in research task. We note that qualitative analysis, as opposed to quantitative work, embraces the complications and interweaving nature of strategic processing across a series of critical moments. Sophistication about such nuances and complexities and revelation of possible factors that might have affected the process by active use of contextual knowledge and situated reasoning is one of the powerful virtues that a rigorous and sophisticated qualitative analysis can achieve.

We support that verbal protocol analysis is a system of methods that is useful for the scientific investigation of what strategic processing is and how it works. The purpose of verbal protocol studies of reading is to know what the reader is doing and thinking while performing a cognitive task, which can be achieved through situated inferences that are made from the real-time language data collected through carefully designed procedures for verbal reporting. Such analysis can challenge and help us to detail the nature and process of decision making and thinking engagement in the comprehension of readers with different tasks (e.g., Hartman, 1995; Wolfe & Goldman, 2005), texts (e.g., Eva-Wood, 2004; Gottlieb & Wineburg, 2012), and environments (e.g., Azevedo, Guthrie, & Seibert, 2004; Coiro & Dobler, 2007). This stance is undergirded by accumulated methodological work suggesting verbal reports as a helpful source of information (e.g., Fox et al., 2011) by which we can look into not only readers’ conscious, goal-directed, and strategic acts of reading (e.g., Pressley & Afflerbach, 1995; Suh & Trabasso, 1993) but also the mental representations built through such processing (e.g., Chi, 1997; Tenbrink, 2015).

We note that reliable inferences made from verbal data have significant contributions to exploring, building, and refining theoretical models of and perspectives on reading in complex tasks and contexts (Cho & Afflerbach, 2018; Pressley & Afflerbach, 1995). What must be reiterated here is that what could be inferred from a participant’s verbal data depends on our acuity with theoretical confidence in response to the subtleties, nuances, and complexities of what verbal utterances mean and represent; all of such inferences are contingent on the task, the context, and the participant. For example, a conclusion drawn from what verbal data tell us about reading can be therefore bolstered in association with, and is often disrupted by, a method of triangulation (Denzin, 1970) that interrelates other complementary data sources such as behavioral, biometric, and outcome data that are sensitive to detect the influences of how reading works in that complex coordination of relevant variables on the substance of the verbal data (Braten, Magliano, & Salmeron, in press; Magliano & Graesser, 1991).

At the same time, we also note that central to the credibility of a qualitative study is transparency in data collection and analysis. A robust explanation of the procedures and factors involved in verbal protocol analysis increases the trustworthiness of findings (Lincoln & Guba, 2000). Reporting contextual information about both the participants and the researcher, and the procedures they have been through, is critical to a publishing of qualitative work in general, and verbal data analysis especially. Both verbal protocol experts (Afflerbach, 2000; Chi, 1997) and qualitative methodologists (Cresswell, 2014; Denzin & Lincoln, 2005) address the importance of disclosure in qualitative work. Qualitative data produces a rich understanding of particular phenomena, which is closely tied to both the participants and the context in which the data is produced. Thus, it is critical to disclose that information which directly relates to verbal data analysis when reporting the results of a study.

More important is disclosure related to researcher positionality, which is shaped by theoretical approaches that inform the design and purpose of the study and securing data sources that best demonstrate the focal aspects of the study (Denzin, 1986; Miles, Huberman, & Saldana, 2014). The theoretical approaches to qualitative verbal data analysis serve to disclose the orientations and perspectives used by the researchers to both design their study and inform their data analysis. Establishing a clear position within the existing literature is an important contribution to the credibility of reports of qualitative verbal data analysis. Therefore, theoretical positioning and orientation of the researcher must be completely disclosed, due to the potential influence that these elements have not only on the process of data collection and analysis but also on the consequent claiming and reasoning about strategic processing in reading.

<<   CONTENTS   >>

Related topics