General Linear Models
The general linear model (GLM) is a term used to encapsulate models that rely on the notion that the relationships between a dependent or outcome variable and independent or predictor variables can be described as a linear function (Rutherford, 2011; Tabachnick & Fidell, 2013). GLM not only includes traditional linear models based on continuous data, like regression, but also incorporates models that utilize categorical data as predictors, such as analysis of variance (ANOVA).
GLM analysis techniques can be used to understand relations between strategy use or strategic processing and other variables of interest, including group membership (e.g., Students t-test or ANOVA) as well as other continuous variables such as motivation either on their own (i.e., correlation) or in relation to multiple variables (i.e., multiple linear regression). In terms of GLM analyses, strategic processing variables can be either the predictor variable (e.g., how does frequency of strategic processing predict academic achievement?) or the criterion variable (e.g., how do men and women differ in their strategic processing?; what is the relationship between motivation, emotions, and strategic processing?).
Student’s t-Test. Students t-test is a common method used to determine if the difference between the means of two independent samples is statistically significant. However, Students t-test functions under the assumption that samples are normally distributed and have equal variances. If t-tests are performed on data that do not adhere to these assumptions, the risk of erroneously reporting that the means are statistically significantly different (Type 1 error), and erroneously reporting that the means are statistically equal (Type II error), both increase (Gibbons & Chakraborti, 1991).
Ruffing, Wach, Spinath, Briinken, and Karbach (2015) used t-tests to determine if the use of learning strategies differed based on students’ gender. They administered a 77-item inventory to educational science students to collect their self-reported use of learning strategies. The instrument was designed to produce 11 scales of learning strategy use, including effort, attention, time management, literature, learning environment, resource-management, organization, relationships, critical evaluation, cognitive strategies, and metacognition. Then, the data was grouped by gender and mean scores were calculated for each learning strategy scale. The gender-specific scale means could be compared using a t-test to determine if differences between the average male and female strategy use levels were statistically significant. They found female students reported more frequent use of effort, time management, organization, cognitive strategies, and metacognition, whereas male students reported performing the learning strategies of critical evaluation and relationships more often.
Analyses of Variance. ANOVA is used to determine if the differences between means (i.e., dependent variable) of three or more categorical groups (i.e., independent variable) are statistically significant (Rutherford, 2011; Tabachnick & Fidell, 2013). Variance due to additional predictor variables can be controlled by using analysis of covariance (ANCOVA; Rutherford, 2011). When researchers are interested in group differences across a number of dependent variables, a multivariate analysis of variance (MANOVA) should be used to evaluate the mean differences in composites of those dependent variables across independent variable categories (Tabachnick & Fidell, 2013). Finally, when dependent variables are captured from the same participants more than once, repeated-measures ANOVA or MANOVA can be used (Rutherford, 2011).
Anmarkrud, McCrudden, Braten, and Stroms© (2013) used ANOVA to evaluate university students’ think-aloud judgments of text relevance while reading conflicting documents about the scientific evidence regarding the health concerns of cell phone use. Text segments within the documents were coded as containing more or less relevant information and students’ comments while reading them were coded as being either positive or negative judgments. To compare the frequency of positive and negative judgements across more and less relevant text segments a 2 (judgment type: positive or negative) x 2 (segment type: more relevant or less relevant) within-subjects ANOVA was performed. Anmarkrud et al. found that while reading more relevant text segments, students expressed a greater number of judgments and those judgments were more frequently rated as positive, whereas the judgments expressed when reading less relevant text were more frequently negative.
Vasilyeva, Laski, and Shen (2015) utilized a MANOVA to determine whether groups based on gender and age differed across multiple outcome measures. The outcomes measured for their study focused on first-graders’ answer accuracy and use of four classifications of strategies (retrieval, counting, decomposition, other) while solving addition problems involving single-, mixed-, and double-digit numbers. Vasilyeva et al. found no differences between groups on the five outcome measures, allowing them to exclude those demographic variables from further analysis of the study’s data.
Correlation. The bivariate correlation coefficient is used to measure size and directionality of the linear association between two variables. Most commonly reported as Pearsons r, correlation represents the degree to which two variables are related to each other. This measure of relationship ranges from -1 to 1, depending on the degree and valence of their correlation. Measures of variable correlation are omnipresent in reports of statistical findings and provide a foundation for more complex analytical methods, which has led to an underappreciation of their own utility and explanatory value (Lee Rodgers & Nicewander, 1988).
Using bivariate correlation, Askell-Williams, Lawson, and Skrzypiec (2012) evaluated survey responses to study the relationship between Australian secondary school students’ self-reported use of cognitive and metacognitive strategies and their selfassessment of how they coped with their homework. Askell-Williams et al. found statistically significant, positive correlations between students coping status and the use of both cognitive and metacognitive strategies, indicating that “students who reported using higher levels of cognitive and metacognitive strategies were more likely to report that they were coping well with school work” (p. 442). After identifying these positive relationships, the researchers used ANOVA to compare students grouped at the extreme ends of the “coping with homework” scale and found that students who reported coping very well with homework were statistically more likely to report using metacognitive strategies than those students who were not coping well with homework.
Multiple Linear Regression Models. Unlike t-tests and bivariate correlation measures, multiple linear regression (MLR) allows researchers to test multiple predictors at once and look at the unique relationship of each predictor with the criterion variable, over the combined relations of the others. MLR models are used to predict criterion variable values by calculating coefficients for the included predictor variables that minimize the sum of the squared differences between them. Often included in the reported results of a regression is the proportion of variation in the criterion variable that is explained by the model (i.e., R2; Rutherford, 2011; Tabachnick & Fidell, 2013).
Roelle, Schmidt, Buchau, & Berthold, (2017) described how the use of MLR allowed them to measure an unexpected effect of their experimental intervention that was not initially apparent when they analyzed their data using ANCOVA. In the third experiment discussed in their article, Roelle et al. tested the effects of providing high school students with either information about regulation strategies or the dangers of making overconfident judgments of learning (JOLs), or both. Although the results of ANCOVA indicated that only providing students with information about the use of regulation strategies had a statistically significant positive effect on the number of elaborations students made, the treatment did not have a statistically significant effect on posttest results. In contrast, a statistically significant positive effect on posttest results was found for providing information about the dangers of making overconfident JOLs and additionally the interaction of the two treatments also showed a significant positive effect on posttest scores. To further understand the relationships between their two treatments, the frequency of students’ elaboration use, and posttest results, Roelle et al. used MLR. Its ability to test multiple predictors and their interactions made MLR particularly well-suited for this situation.
Using posttest scores as the dependent variable, Roelle et al. (2017) produced an MLR model that included pretest scores, the two treatment groups (i.e., information about regulation strategies and information about the dangers of making overconfident JOLs), an interaction term for the two treatments, the number of elaborations performed, and finally an interaction term between being informed about making overconfident JOLs and elaborations performed. The resulting MLR model produced statistically significant positive coefficients for pretest scores, elaboration use, and the interaction of the JOLs treatment on elaboration use. Though at first an ANCOVA indicated that the overconfident JOLs treatment had no effect on the use of elaboration, the results of an MLR revealed that the JOLs treatment had a statistically significant effect on the use of elaboration. In sum, conducting an MLR revealed that understanding the dangers of making overconfident JOLs was not enough to have a statistically significant effect in this learning situation, and that participants also benefitted from learning regulation strategies in order to produce more effective elaborations. Thus conducting an MLR gave more details of the relationship between having knowledge about overconfident JOLs, strategic processing, and learning outcomes.