Desktop version

Home arrow Education arrow Handbook of Test Development



Thanks to Randy Bennett, Bob Mislevy, Heather Buzick, Caroline Wylie, Leslie Nabors Olah, and the editors of this volume for their reviews of earlier versions of this manuscript, and to Jim Fife for suggested revisions. Their efforts are appreciated and any errors are the sole responsibility of the authors.


1. Classification can also be pursued with the PCM, but it does not follow directly from the model parameters. It can be done by using the averages of the item-specific level transitions as cutoffs. A comparison of this approach with the classification obtained with the CPCM reveals an 84% exact agreement and quadratically weighted kappa of 0.93 (SE = 0.03).


American Educational Research Association, American Psychological Association & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561—573.

Arieli-Attali, M. (June, 2011). Linear Functions Revised Model. Presented at the Advisory Panel of the CAMA project on developmental models, Princeton, NJ.

Arieli-Attali, M., Wylie, E. C., & Bauer, M. I. (2012, April). The use of three learning progressions in supporting formative assessment in middle school mathematics. Paper presented at the annual meeting of the American Educational Research Association (AERA), Vancouver, Canada.

Bennett, R. E. (2010). Cognitively based assessment of, for, and as learning: A preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspectives, 8, 70—91.

Bennett, R. E. (2015). The changing nature of educational assessment. Review of Research in Education, 39(1), 370-407.

Bennett, R. E., & Gitomer, D. H. (2009). Transforming K-12 assessment: Integrating accountability testing, formative assessment, and professional support. In C. Wyatt-Smith & J. Cumming (Eds.), Educational assessment in the 21st century (pp. 43-61). New York, NY: Springer.

Black, P, Wilson, M., & Yao, S.Y. (2011). Road maps for learning: A guide to the navigation of learning progressions. Measurement: Interdisciplinary Research & Perspectives, 9(2-3), 71-123.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29-51.

Briggs, D. C., & Alonzo, A. C. (2012). The psychometric modeling of ordered multiple-choice item responses for diagnostic assessment with a learning progression. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 293-316). Rotterdam, The Netherlands: Sense Publishers.

Briggs, D. C., Alonzo, A. C., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple- choice items. Educational Assessment, 11(1), 33-63.

Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical Thinking and

Learning, 6(2), 81-89.

Clements, D. H., Wilson, D. C., & Sarama, J. (2004). Young children’s composition of geometric figures: A learning trajectory. Mathematical Thinking and Learning, 6(2), 163-184.

Common Core State Standards Initiative. (2010). Common core state standards for mathematics. Retrieved from

Confrey J., Maloney, A., Nguyen, K., Mojica, G., & Myers, M. (2009, July). Equipartitioning/splitting as afoundation of rational number reasoning using learning trajectories. Paper presented at the 33rd Conference of the International Group for the Psychology of Mathematics Education, Thessaloniki, Greece.

Corcoran, T, Mosher, F. A., & Rogat, A. (2009). Learning progressions in science: An evidence-based approach to reform (Research Report No. RR-63). Philadelphia, PA: Consortium for Policy Research in Education.

Daro, P, Mosher, F. A., & Corcoran, T. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction (CPRE Research Report No. RR-68). Philadelphia, PA: Consortium for Policy Research in Education.

Deane, P (2011). Writing assessment and cognition (ETS Research Report No. RR-11-14). Princeton, NJ: Educational Testing Service.

Deane, P, Odendahl, N., Quinlan, T, Fowles, M., Welsh, C., & Bivens-Tatum, J. (2008). Cognitive models of writing: Writing proficiency as a complex integrated skill (ETS Research Report No. RR-08-55). Princeton, NJ: Educational Testing Service.

Deane, P, Sabatini, J., & O’Reilly, T. (2012). The CBAL English language arts (ELA) competency model and provisional learning progressions. Retrieved from

De Boeck, P, Wilson, M., & Acton, G. S. (2005). A conceptual and psychometric framework for distinguishing categories and dimensions. Psychological Review, 112(1), 129.

Duschl, R., Maeng, S., & Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis. Studies in Science Education, 47(2), 123-182.

Graf, E. A. (2009). Defining mathematics competency in the service of cognitively based assessment for grades 6 through 8 (ETS Research Report No. RR-09-42). Princeton, NJ: Educational Testing Service.

Graf, E. A., & Arieli-Attali, M. (2014). Developing and validating a learning progression for an assessment of complex thinking in mathematics for the middle grades. Manuscript submitted for publication.

Graf, E. A., Harris, K., Marquez, E., Fife, J., & Redman, M. (2009). Cognitively Based Assessment of, for, and as Learning (CBAL) in mathematics: A design and first steps toward implementation (Research Memorandum No. RM-09-07). Princeton, NJ: Educational Testing Service.

Graf, E. A., Harris, K., Marquez, E., Fife, J., & Redman, M. (2010). Highlights from the Cognitively Based Assessment of for, and as Learning (CBAL) project in mathematics. ETS Research Spotlight, 3, 19-30.

Haberman, S. J., & von Davier, M. (2006). Some notes on models for cognitively based skills diagnosis. In C. R. Rao & S. Sinharay (Eds.), Handbook of statistics: Vol. 6. Psychometrics (pp. 1031-1038). Amsterdam, The Netherlands: Elsevier North-Holland.

Kalchman, M., Moss, J., & Case, R. (2001). Psychological models for development of mathematical understanding: Rational numbers and functions. In S. M. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress, (pp. 1-38). Mahwah, NJ: Erlbaum.

Kane, M. T (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.

Kieran, C. (1993). Functions, graphing, and technology: Integrating research on learning and instruction. In T A. Romberg, E. Fennema & T P Carpenter (Eds.), Integrating research on the graphical representation of functions (pp. 189-237). Hillsdale, NJ: Erlbaum Associates.

Leighton, J. P, Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s Rule space approach. Journal of Educational Measurement, 41(3), 205-237.

Liu, L., Rogat, A., & Bertling, M. (2013). A CBAL science model of cognition: Developing a competency model and learning progressions to support assessment development (ETS Research Report No. RR-13-29). Princeton, NJ: Educational Testing Service.

Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York, NY: American Council on Education.

Mislevy, R. J., Almond, R. G., & Lukas, J. F. (2003). A brief introduction to evidence-centered design (ETS Research Report No. RR-03-16). Princeton, NJ: Educational Testing Service.

Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3-62.

Mislevy, R. J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55(2), 195-215.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.

National Research Council. (2006). Systems for state science assessment (M. R. Wilson & M. W Bertenthal, Eds.). Washington, DC: National Academies Press.

National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8 (R. A. Duschl, H. A. Schweingruber & A. W. Shouse, Eds.). Washington, DC: National Academies Press.

O’Reilly, T, & Sheehan, K. (2009). Cognitively Based Assessment of, for, and as Learning: A framework for assessing reading competency (ETS Research Report No. RR-09-26). Princeton, NJ: Educational Testing Service.

Pellegrino, J. W, Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

Rittle-Johnson, B., Matthews, P G., Taylor, R. S., & McEldoon, K. L. (2011). Assessing knowledge of mathematical equivalence: A construct-modeling approach. Journal of Educational Psychology, 103(1), 85.

Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464.

Sfard, A. (1991). On the dual nature of mathematical conceptions: Reflections on processes and objects as different sides of the same coin. Educational Studies in Mathematics, 22(1), 1-36.

Sfard, A. (1992). Operational origins of mathematical objects and the quandary of reification-the case of function. In E. Dubinsky & G. Harel (Eds.), The concept of function: Aspects of epistemology and pedagogy (Vol. 25, pp. 59-84). Washington, DC: Mathematical Association of America.

Shavelson, R. J. (2009, June). Reflections on learning progressions. Paper presented at the Learning Progressions in Science (LeaPS) Conference, Iowa City, IA.

Sheehan, K., & O’Reilly, T. (2011). The CBAL reading assessment: An approach for balancing measurement and learning goals (ETS Research Report No. RR-11-21). Princeton, NJ: Educational Testing Service.

Simon, M. A. (1995). Reconstructing mathematics pedagogy from a constructivist perspective. Journal for Research in Mathematics Education, 26(2), 114-145.

Smith, C., Wiser, M., Anderson, C., Krajcik, J., & Coppola, B. (2004). Implications of research on children' learning for assessment: Matter and atomic molecular theory. Paper commissioned by the Committee on Test Design for K-12 Science Achievement, Center for Education, National Research Council.

Steedle, J. T., & Shavelson, R. J. (2009). Supporting valid interpretations of learning progression level diagnoses. Journal of Research in Science Teaching, 46(6), 669-715.

van Rijn, P W, & Graf, E. A. (2013). Measurement models for establishing learning progressions with applications to mathematics and English language arts. Manuscript in preparation.

van Rijn, P W, Graf, E. A., & Deane, P (2014). Empirical recovery of argumentation learning progressions in scenario- based assessments of English language arts. Psicolog a Educativa, 20(2), 109-115.

van Rijn, P W, Wise, M., Yoo, H., & Cheung, S. (2014). Statistical report: Summary statistics, local dependence, and differential item functioning in the CBAL 2012 Mathematics study. Manuscript in preparation.

Vinner, S., & Dreyfus, T. (1989). Images and definitions for the concept of function. Journal for Research in Mathematics Education, 20(4), 356-366.

West, P, Rutstein, D. W, Mislevy, R. J., Liu, J., Levy, R., DiCerbo, K. E., et al. (2012). A Bayesian network approach to modeling learning progressions. In A. C. Alonzo and A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 257-292). Rotterdam, The Netherlands: Sense Publishers.

Wilmot, D. B., Schoenfeld, A., Wilson, M., Champney, D., & Zahner, W. (2011). Validating a learning progression in mathematical functions for college readiness. Mathematical Thinking and Learning, 13(4), 259—291.

Wilson, M. (1989). Saltus: A psychometric model of discontinuity in cognitive development. Psychological Bulletin, 105(2), 276-289.

Wilson, M. (2005). Constructing measures. Mahwah, NJ: Lawrence Erlbaum.

Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46(6), 716-730.

Wilson, M., & Sloane, K. (2000). From principles to practice: An embedded assessment system. Applied Measurement in Education, 13(2), 181-208.

Zalles, D., Haertel, G., & Mislevy, R. J. (2010). Using evidence-centered design to state large-scale science assessment (Technical Report No. 10). Menlo Park, CA: SRI. Retrieved from TR10_Learning_Progressions.pdf

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >

Related topics