Desktop version

Home arrow Philosophy

  • Increase font
  • Decrease font


<<   CONTENTS   >>

Commentary: Analyzing Strategic Processing: Pros and Cons of Different Methods

The authors in this section reviewed variable-centered (e.g., regression), personcentered (e.g., cluster analysis), and qualitative approaches to analyzing a variety of types of data collected on learning strategies. The authors discussed data collected from questionnaires, think-aloud protocols, eye tracking, and computer log files. Research on strategy use has also considered student discourse in dyads (e.g., Miller, Cromley, & Newcombe, 2015) and coding of gestures as indicative of strategy use (e.g., Alibali & Goldin-Meadow, 1993). The authors clearly laid out the different types of research questions that different analyses are suited for and various assumptions about the data that are required for the analyses. For most analyses, when data meet the assumptions of the test (e.g., normal distribution) the parametric tests are used, and when data fail one or more assumptions, nonparametric tests are used. Although mathematical transformations or bootstrapping are other options, they are not commonly used in education research, so I do not discuss them here.

In this chapter, I lay out some different data analysis options for the researcher working with strategy data. I lay out the options, in each case taking into account a number of considerations: (1) Do the data meet the assumptions for a candidate data analytic approach (e.g., normality, linearity)? (2) Are sample sizes large enough for the candidate data analytic approach? (3) Is data reduction needed (e.g., combining fine-grained codes from think-alouds, creating factor scores)?, and (4) Do large sample sizes need to be planned for in advance (e.g., full structural equation modeling)? Taking these considerations into account allows the analyst to maximize statistical power, while preserving the validity (i.e., trustworthiness) of the results from each analysis.

Variable-centered analyses

Chi Squared Test

In addition to a long tradition of t test and ANOVA analyses to compare strategy treatment groups to control groups, some early work in strategy use during learning used chi squared tests to look for disproportionate use of certain strategies in think-aloud studies between novice/expert (e.g., Ennis & Safrit, 1991) or tutored/untutored participants (e.g., Azevedo & Cromley, 2004). One advantage of the chi squared test is that it takes account of different learners being more or less talkative—some people are simply more chatty than others (see Table 24.1). It is important to take account of these differences, because analyses of raw counts of many types of data will not show the differences that researchers are trying to test; they will simply show effects of talkativeness, and talkativeness is rarely related to learning. One disadvantage of the chi squared test is that, as a non-parametric test, it has low statistical power to detect an association, if an association between variables is indeed present. Another disadvantage is that while a chi squared test may seem to show associations between codes, it merely shows co-occurrence across an entire transcript (or log file, etc.).

Table 24.1 Summary of Uses for, Pros of, and Cons of the Statistics Reviewed in this Chapter

Statistic

Used for

Pros

Cons

chi squared

Associations between 2 categorical variables

EasyAccounts for base rate differences without any extra calculations

Few assumptions

Low power

Just an association

ttest

Comparing means of 2 groups on a continuous outcome

Easy

Must meet 4 assumptions No ‘covariate’

ANOVA

Comparing means of 2+ groups on a continuous outcome

Easy

Must meet 4 assumptions No ‘covariate’

correlation

Associations between 2 continuous variables

Easy

Must meet 4 assumptions

Just an association

regression

Effect of one or more independent variables, simultaneously, on a single, continuous dependent variable

EasyEffect of each independent variable takes account of all other independent variables Can test moderation

Must meet 4 assumptions

path analysis

Effect of one or more independent variables, simultaneously, on continuous dependent variable(s)

Can test mediation

Account for measurement error in ‘DVs’

Multiple DV s simultaneously

Specialized software, steep learning curve Must meet 4 assumptions

structural equation modeling

Effect of one or more independent factors, simultaneously, on continuous dependent factor(s)

‘Pure’ factors, account for measurement error Can test mediation

multiple DV s simultaneously

Specialized software, steep learning curve Multiple variables per construct Must meet 4 assumptions

Statistic

Used for

Pros

Cons

GCM

Shape and rate of change over time on a variable Predictor(s) of intercept and slope(s), outcome(s) of intercept and slope(s)

Power

Flexible shape(s) of growth Growth on growth (i.e., multiple DVs simultaneously)

Specialized software, steep learning curve Multiple variables per construct Must meet 4 assumptions Multiple time points High reliability needed measurement invariance over time needed

transition analysis

Sequence of strategy moves during learning

Analyzes the actual sequence

Low power

LCA

Subgroups of participants Predictor(s) of subgroup membership, outcome(s) of subgroup membership

‘Pure’ factors, account for measurement error

Can combat stereotyping

Specialized software, steep learning curve No unique solution Multiple choice points

GMM

Subgroups of trajectories Predictor(s) of intercept and slope(s), outcome(s) of intercept and slope(s)

Can combat stereotyping Multiple routes to similar outcomes

See GCM plus No unique solution Multiple choice points

LTA

Predictors of transitions between levels of a categorical outcome

‘Pure’ factors, account for measurement error

Can combat stereotyping

See GCM plus No unique solution Multiple choice points

Quai TA

Sequences during actual learning

Illuminate how a process works

Good data collection practice is especially important Coding approach needs to be somewhat flexible Coding is effort intensive

 
<<   CONTENTS   >>

Related topics