Desktop version

Home arrow Philosophy

  • Increase font
  • Decrease font

<<   CONTENTS   >>

Argumentation and evidentiary reasoning in qualitative verbal protocol analysis

Verbal protocol analysis involves an argument process. It requires a logical reasoning concerned with the experiences of human subjects and the processes they engage in to navigate and make sense of the world (Bogdan & Biklen, 2007). Chi (1997) elaborated on this idea within the context of verbal report data regarding the potential of qualitative methods to capture participants’ natural behaviors in authentic settings to gain a “richer and deeper understanding of a situation” (p. 280). Central to developing claims in qualitative research are the data analysis procedures used to identify evidence of a particular phenomenon. However, this process is not without challenges because “coding of verbal reports is an interpretive act” and “the richness of language and the constructive nature of understanding language” present both the promise and the challenge of making a legitimate claim substantiated by verbal reports (Pressley & Afflerbach, 1995, p. 122).

Because qualitative research is primarily inductive and grounded in data, rather than deductive or beholden to theory, there is particular importance placed upon the conceptual frameworks that ground the study and contribute to the inferences made from the data (Merriam, 1998). Thus, the coding of qualitative data typically includes a robust explanation of both previous research, related theory, and their contributions to the emerging data coding and analysis schemes. Further, as in quantitative research, the methods of analysis are discussed at due length; however, the involvement of the researcher as an analytical instrument (Patton, 1990) often engages additional discussion on the methodological decisions made (i.e., rationale that supports why a particular method, approach, and procedure is chosen, employed, or modified within the context of its use).

We present the path of argumentation undertaken in our verbal protocol analysis concerning epistemic processing for online reading (Cho et al., 2018). As outlined in Figure 23.4, the diagram (informed by Toulmin’s (1958) argument model) unpacks the reasoning process into the following components:

  • • Claim: The assertions made about the focal aspect of investigation about epistemic processing in online reading.
  • • Data: The verbal report data collected from a suitably designed research task and the results from the qualitative verbal protocol analysis in relation to codes, categories, and themes of epistemic processing in online reading.
  • • Warrant: Theoretical framework that describes what epistemic processing means in online reading, why the studies of the topic matter, and how it can be investigated.
  • • Backing: Empirical studies that use verbal protocol analysis to examine epistemic processing in online reading and therefore elaborate the theoretical framework.
  • • Qualifiers: Conditions that limit the claim made from evidentiary reasoning.
  • • Rebuttals: Counterclaims, alternative explanations, or limitations that could be suggested in response to the claiming and reasoning.

We first sought to examine why epistemic processing matters in online environments through an overview of the theoretical literature in the areas of personal epistemology and internet reading, which serves as the warrant for our study. We drew on previous work that established theories for understanding how epistemic beliefs influence learning processes (Hofer & Pintrich, 1997), the relationship between epistemic beliefs and cognitive processing (Hofer, 2004), and established what we meant by epistemic processing and addressed relevant concerns regarding measurement issues related to personal epistemology (Cho, 2014; DeBacker, Crowson, Beesley, Thoma, & Hestevold, 2008; Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012; Greene, Azevedo, & Torney-Purta, 2008). Our exploration of theoretical warrant extended to a discussion of the perspectives informing our understanding of epistemic processing in online reading, which included recognizing the intertextual nature of the internet (Landow, 1992), complexities of identifying sources of information (Lankes, 2008; Metzger & Flanagin, 2013), and the sophisticated and critical mindset needed to seek and gain knowledge from reading online (Greene et al., 2014; Lankshear, Peters, & Knobel, 2000; Ulyshen, Koehler, & Gao, 2015).

We then drew on additional research to expand these theoretical understandings to other studies which sought to utilize verbal reporting methods to investigate the role of epistemic beliefs in online reading, as backing for the warrant established in theoretical overview. We discussed the contributions and limitations of the selected empirical studies related to the particular dimensions of epistemic beliefs (Hofer, 2004), how individual differences of readers could interact with epistemic beliefs when reading online (Mason, Boldrin, & Ariasi, 2010a), the relationship among sophisticated epistemic beliefs, searching processes, and understanding (Mason, Boldrin, & Ariasi, 2010b), the influence of a particular topic on the enactment of epistemic beliefs (Ferguson, Braten, & Stromso, 2012), the role of self-regulation and epistemic processing in learning from online sources (Greene et al., 2014), and the processing related to

A Simple Representation of Argument Reasoning Engaged in Verbal Protocol Analysis of Epistemic Processing in Online Readingviews that readers may hold about the nature of knowledge

Figure 23.4 A Simple Representation of Argument Reasoning Engaged in Verbal Protocol Analysis of Epistemic Processing in Online Readingviews that readers may hold about the nature of knowledge (Barzilai & Zohar, 2012). Having presented the previous warrant and backing, we argued that in order to build on the existing understanding of epistemic beliefs and online reading, it is also important to understand the epistemic processes readers employ in an unconstrained digital space and which particular processes are undertaken by more and less successful readers. In one way, we hope to build a theoretical net which is capable of capturing and then explaining the phenomena reported in participants’ verbalizations.

We used readers’ concurrent verbal protocols as data that is particularly well-suited for capturing the moment-to-moment processes of readers as they navigate a multisource text environment. We first identify the context of data collection and detail the types of data and methods used to collect it. Then, we describe how the data was prepared for analysis, which included transcripts of participants’ verbal reports along with the screen capture of their actions. The transcripts were then open coded using semantic segmentation of data (Chi, 1997), which allowed for particular thoughts or actions to be contextualized within a larger chain of processing. This initial coding focused on a small sample of data to identify all possible instances of epistemic processing. Subsequent coding involved comparison of the researcher-generated codes to develop superordinate code categories which were then integrated into a coherent coding scheme. The remaining data was analyzed in a process of axial coding with the initial coding scheme to refine and improve it throughout the data analysis process. Because a primary purpose of our study was to investigate the epistemic processes that emerged from the data, the explanation of the categories and related dimensions and actions were the first major claim. In addition to descriptions of epistemic judgment, monitoring, and regulation, verbal report data was presented to exemplify each element of the three larger categories (Cho et al., 2018, pp. 209-210).

Our findings were then discussed relative to the nomological network provided by prior theory and research. This also allowed us to qualify the context in which our claims were grounded and recognized that our study did not capture students who were not likely to be proficient at thinking aloud, nor the knowledge that is gained when seeking to understand as part of a collaborative effort. However, the study did yield evidence about how the individual epistemic processes of students experienced more and less success during their reading task. Yet, there are other potential limitations of (or counterclaims against) our study that must be addressed in any discussion of our findings. These include the critical questioning task that may not reflect the ways in which students learn from online sources in more personal spaces, the nature of the task design to be situated within a school context, and the non-verbalized beliefs that learners may hold. Further, research on domain expertise indicates that lack of domain knowledge and understanding of disciplinary practices (Maggioni, Fox, & Alexander, 2010; Warren, 2011) may also shape the process of online reading.

In summary, argumentative reasoning is an essential part of the verbal protocol analysis that seeks to achieve a methodological rigor and theoretical contribution to generating new knowledge and insights. Such evidentiary process requires connecting the research work to a robust conceptual framework built upon previous studies of key constructs to be examined, and to a thoughtful consideration of counter claims and alternative explanations. Through the process, researchers may be able to seek to not only establish a clear ground for the claims that emerge from the data but also to clarify the importance of our findings through acknowledging additional and alternative perspectives.

<<   CONTENTS   >>

Related topics