III: Open questions and new approaches

Challenges in exploring individual’s conceptions of knowledge and knowing: examples of research on university students

Introduction

Individuals’ conceptions of knowledge and knowing are considered to be a challenging research topic due to their complexity and multidimensional character. As noted in Chapters 3 and 4, these conceptions have been investigated in the context of different disciplines, such as philosophy, psychology, and education. Although previous research in the fields of education and psychology has frequently leant on self-reporting methods, the reliability and adequacy of these methods have recently been questioned, and it has been suggested that they do not capture the complexity of phenomena. The present chapter extends the discussion on epistemological development by introducing and evaluating empirical methods used in investigating individuals’ conceptions of knowledge, focusing on the methodological and theoretical challenges related to them. The chapter also presents methodological alternatives for future research, and highlights the need for dialogue between theoretical, methodological, and empirical perspectives to advance research on epistemic conceptions. The chapter particularly focuses on research into university students’ conceptions of knowledge from the perspective of developmental and educational psychology.

A Short history of investigating individuals' conceptions of knowledge and knowing

A variety of research methods and materials have been applied when investigating epistemic conceptions within developmental and educational psychology (see Chapter 3). From the 1950s onward, Perry used open-ended interviews, questionnaires, and problem-solving tasks to examine epistemological beliefs - or in

Perry’s own words “forms of intellectual and ethical development” (Perry, 1970). However, the next generation of researchers has quite systematically applied different questionnaires, and it was not until the turn of this millennium that qualitative methods became increasingly common.

In his research project. Perry investigated college students at Harvard University and followed their intellectual and ethical development during their four college years by interviewing them at the end of each academic year. The participants were randomly selected white male students who had answered a questionnaire designed by Perry and his colleagues, and entitled “the Checklist of Educational Views” (Perry, 1970; Schommer-Aikins, 2004). Interviewing was the main research method because, according to Perry, the use of open-ended interviews enabled the examination of individual development paths during college years. The research group detected a nine-phase development process during which the students’ conceptions of the nature of knowledge changed and deepened and their worldviews and self-conceptions changed (Perry, 1970; Schommer-Aikins, 2004).

From interviews to questionnaires and problem-solving tasks

After Perry, research has expanded to include female students (e.g., Belenky, Clin-chy, Goldberger, & Tarule, 1986) and the different societal and cultural backgrounds of students (Helsing et al., 2001). Like Perry, Belenky and colleagues also used open-ended interviews in their research project on Women’s Ways of Knowing (Belenky et al., 1986). On the basis of the interviews, they described the process of cognitive development comprising of five knowledge positions, from reliance on authorities to constructed knowledge by integrating voices. These five positions follow quite closely Perry’s development model, though his model is more detailed at the more developed end.

To conclude, both Perry and Belenky created their development models on the basis of open-ended interview data, focusing on students’ experiences, evaluations, and narratives. Both research groups used several raters in analysing the data, and were aware of the limitations of their research methods, in particular the possibility of subjective bias during the analysis process. Furthermore, both groups were aware of the limited generalisability of their findings (Baxter Magolda, 2001; Belenky et al., 1986; Perry, 1970).

Many researchers were concerned about the use of interviews as the main data collection method and began to apply quantitative research methods to explore epistemic conceptions. Interviews were considered slow and expensive, and both data collection and analysis required high expertise compared to questionnaires, which were much faster and cheaper to use, and expertise was not required in scoring students’ answers. Moreover, teachers, tutors, and counsellors were all able to use the questionnaires.

Ryan (1984, p. 250) developed a Dualism Scale based on Perry’s model. The scale consisted of seven items, with statements such as "For most questions there is only one right answer once a person is able to get all the facts" and “ If professors would stick more to the facts and do less theorising, one could get more out of college”. These items were designed to measure the level of students’ epistemological development in the “dualism-relativism continuum”. A five-point Likert scale was used. An average score lower than 3.0 reflected a relativist conception of knowledge, whereas an average score above 3.0 was judged to reflect a dualist conception of knowledge.

Schommer-Aikins presented a theoretical model (Schommer, 1990) which suggests that epistemological beliefs consist of several more or less independent beliefs which form the individual’s epistemological belief system. This broadened view of epistemological beliefs led to a more systematic use of questionnaires in this research field. Schommer-Aikins (2004) has, however, emphasised the difficulties and challenges in measuring epistemological beliefs by using questionnaires. These difficulties were also apparent in her Epistemological Questionnaire (EQ) (Schommer, 1998). Researchers using this questionnaire reported various factor solutions, because in different datasets, the original scales failed to appear. Therefore, Schommer-Aikins shortened the Epistemological Questionnaire from 63 items first to a 34-item and later to a 28-item questionnaire. However, the scale reliabilities remained quite low. Indeed, the key challenge in the use of questionnaires has been to find a way to capture the complex nature of the phenomenon in questionnaire items (e.g., Baxter Magolda, 2001; Schommer-Aikins, 2004).

Besides questionnaires and interviews, additional methods have also been applied in examining individuals’ conceptions of knowledge and knowing. Ryan (1984, p. 251) developed a Comprehension Monitoring Probe, in which the students were given 15 minutes to answer the following questions:

• 1) How do you determine (when you have completed a reading assignment or when you are reviewing the material) whether you have understood the material well enough?
• 2) What specific information do you use to assess the degree to which you have understood the material you have read in a chapter?
• 3) On what basis would you decide that you need to go over the chapter again or to seek help in figuring it out?

Ryan analysed each student’s responses to the comprehension-monitoring probe to determine the specific comprehension criteria employed. An effort was made to score each response for as many different comprehension criteria as possible in order to capture the full range of each student’s comprehension monitoring capabilities. The comprehension monitoring criteria (Ryan, 1984) were classified as knowledge criteria or comprehension/application criteria. Ryan showed that it was possible to statistically analyse epistemic conceptions and to demonstrate a correlation between students’ conceptions of knowledge and the comprehension criteria they used.

Baxter Magolda and Porterfield developed the Measure of Epistemological Reflection (MER) (Baxter Magolda, 2001) on the basis of Perry’s model. The

MER contains six short open-ended tasks on the basis of which it is possible to evaluate different areas of epistemological development. These tasks measure students’ skills in drawing conclusions, their perceptions of the roles of themselves, teachers and peers in learning, their views of how learning should be evaluated, and how students make decisions about what to believe. The tasks are open-ended to avoid leading or confining the students’ responses, and follow-up questions are made to clarify students’ perspectives. Because of the use of open-ended tasks, evaluation of students’ answers is a demanding process. After the development of MER, Baxter Magolda developed the MER constructivist interpretation process on the basis of which it is possible to analyse open-ended interviews on students’ epistemological development. The aim of this method is not to evaluate the development phase of individual students, but instead to steer and support the researcher’s analysis process.

Based on an extensive review of contemporary research findings and theoretical frameworks, Hofer and Pintrich (1997) proposed that although the number and nature of dimensions of individuals’ conceptions of knowledge vary across different theoretical frameworks, they all include some commonalities, such as certainty of knowledge (i.e., an absolutist versus a relativist view), simplicity of knowledge (i.e., simple and concrete versus complex and context-dependent), source of knowledge (from external authorities versus from personal construction), and justification for knowing (criteria for making knowledge claims, use of evidence and reasoning (Hofer & Pintrich, 1997; see also Chapter 3). Around these theoretical findings, Hofer (2000) built the Discipline-focused Epistemological Belief Questionnaire (DEBQ), which has become one of the most widely used quantitative measures (Muis, Trevors, Duffy, Ranellucci, & Foy, 2016). Hofer’s DEBQ questionnaire is designed to focus on these four commonalities of conceptions of knowledge (Hofer, 2000).

Mixed-method approaches, e.g., combining questionnaire and interview data, have recently become increasingly common in exploring personal epistemology (e.g., Hofer & Sinatra, 2010; Hyytinen, Clancy, Teviotdale, & Postareff, 2016; King & Kitchener, 2004; Muis et al., 2016). For example, King and Kitchener (2004) created the Reflective Judgment Model (see Chapter 4). To evaluate the level of students’ cognitive development, King and Kitchener used problemsolving tasks. They developed the Reflective Judgment Interview in which trained interviewers ask open-ended questions to evaluate the students’ cognitive development, the quality of argumentation and their conceptions of knowledge and knowing. The results of the Reflective Judgment Interviews can further be analysed statistically. Kuhn and her colleagues (e.g., Kuhn, Cheney, & Weinstock, 2000; Kuhn, Katz, & Dean, 2004) have also used different problem-solving tasks combined with questionnaires to examine cognitive development from childhood to adulthood. Thus, the methodological approach is similar to that of King and Kitchener (2004).