Qualitative Approaches to the Verbal Protocol Analysis of Strategic Processing
In this chapter, we examine qualitative approaches to verbal protocol analysis and their role in constructing detailed accounts of strategic processing. To that end, we draw from diverse studies in the area of reading in which qualitative protocol data is extensively used to catalog and describe the readers’ strategies engaged with a variety of texts, tasks, and goals. Thus, our discussion is situated within the domain of reading in an effort to illustrate the potential of qualitative verbal protocol analysis to a wider range of domains of knowledge, problem, and learning through examining the versatility of strategic processing in reading. In the sections that follow, we first argue for the importance of investigating strategic processing in reading. Then, we examine the scientific merits of qualitative verbal protocol analysis in examining such strategic processing. Finally, we discuss critical issues in the qualitative analysis of verbal protocol data and suggest possible means of addressing the issues to undertake rigorous analyses of strategic processing.
The importance of analyzing strategic processing to understanding the construct of reading
Our conception of reading is grounded at the intersection of research literature on readers (e.g., knowledge, skills, attitudes, identities), texts that they access and process (e.g., genres, structures, purposes, modes), and the contexts in which reading takes place (e.g., social influences, task environments) (McNamara & Magliano, 2009; Snow, 2002). We note that these variables interact distinctively in accordance with readers’ goals. When reading is successful, the goals that readers establish help guide strategies—as text, task, and context comprise the problem space that is demanding of strategic processing. Reading goals are not static. They may change as readers gain knowledge from constructing meaning from text, and as readers reflect on their needs and opportunities related to goals. That is, acts of reading are situated with the specific goals that are planned, revisited, and reformulated along the course of making meaning (Brown, Collins, & Duguid, 1989; Kintsch, 1988).
Readers use cognitive and metacognitive strategies to achieve goals. This process is dynamic, as readers select, adjust, and coordinate different strategies to construct meaning. The use of specific strategies is not a stand-alone cognitive activity. Rather, strategies are interrelated and mutually supportive as reading progresses under control of reader metacognition (McNamara, 2007; Pressley, 1995; Veenman, 2015). Hence, reading is better understood when we identify and describe how strategies are chosen and orchestrated by readers. Verbal protocol analysis is well-suited to the task of providing detailed data related to such strategic reading.
Strategic reading, in particular, involves chains of intentional cognitive actions focused on building a coherent understanding of text (Alexander, Graham, & Harris, 1998; Cho, 2014). Of course, not all reading behaviors are necessarily strategic, neither are all strategies successful (Afflerbach, Pearson, & Paris, 2008). However, cognitive work must be initiated and assessed by the reader for utility and effectiveness in each iteration of strategy use (Afflerbach, Cho, & Kim, 2015). Due to this situational intentionality, determining the nature of strategies in specific reading situations is a challenging task that demands contextualized inferences about the enactment of those processes.
That said, qualitative studies provide a common ground for research that seeks to describe the types and sequences of strategic processes in reading. Research on accomplished reading describes some common features that may distinguish strategic actions from non-strategic behaviors. In their meta-analytic work, Pressley and Afflerbach (1995) reexamined the numerous cognitive and metacognitive strategies taken from the verbal protocols of accomplished readers (e.g., librarians, historians, scientists) observed in 38 primary research studies. Their analysis yielded a compendium of reading comprehension strategies, which were grouped into three categories: (a) identifying and learning text content, (b) self-monitoring of thinking processes, and (c) evaluating different aspects of reading.
For example, Pressley and Afflerbach (1995, pp. 31-62) describe the array of strategic actions that accomplished readers carry out for identifying and learning text content in the following manner:
both literal and inferential reasoning using text information, analyze and integrate different parts of text, and interpret text ideas and hidden meanings.
• After the initial reading, accomplished readers reread selected parts of text with an eye out for particular information. Rereading and reflection leads them to construct a cohesive summary of text (i.e., situated mental representation). They self-question over text content, reflecting on text with the possibility of reflection leading to shifts in text interpretation. Consequently, these readers continually evaluate and reconstruct an understanding of text, change their responses to text as the understanding is reconstructed as a result of reading and reflection, and think further on the mentally represented text in anticipation of using it later for real-life purposes.
Strategic processing is a window on our understanding of how reading works. Verbal protocol studies of reading describe successful readers who employ a set of strategic thoughts and processes (italicized in the above descriptions from Pressley & Aftler-bach, 1995). Although not all strategies are equally prominent in their text processing, these accomplished readers regulate their use of strategies in relation to the construction of meaning and the goals they seek to achieve. In stark contrast, ineffective strategy use is observed in novice readers. These readers might intend to be strategic, but their actions may not be goal-relevant due to multiple challenges such as a lack of domain knowledge, ineffective self-reflection, unspecific (or no) goals, a shortage of metacognitive awareness, and uninformed perspectives on what reading “is.” Therefore, observing and evaluating one’s strategy use, such as that inferred from verbal reports, is a litmus test for the performance and accomplishment of the reader.
We note that the bullet points above offer only a tangential summary of many strategies for identifying and learning text content—Pressley and Afflerbach (1995) documented a total of 204 interrelated strategies in this category only. That is, a considerable degree of variety, complexity, and uniqueness must be anticipated in the examination of how readers choose and organize among these strategies in different, extended, or novel tasks. We also reiterate that reading is understood in a particular coordination of reader, text, and context, and it is important to consider how manifold interactions among these components may play out in the analysis of readers’ goal-directed engagement in strategic processing.
In short, our stance toward reading supports the importance of investigating the nature of strategic processing situated in this complex coordination that must count as an essential consideration for that investigation. It also calls for attention to qualitative analysis of process-oriented data, specifically reader-generated verbal reports, from which detailed accounts of readers’ strategies, and their relationship to text, context, and goal that interplay in meaning making, are constructed. Such work can contribute to increasingly fine-grain understandings of the complexities and nuances in strategic reading which might not be revealed otherwise.