Assessing English Language Proficiency in U.S. K–12 Schools - Assessing English Language Proficiency in U.S. K

English Learners in U.S. K-12 Schools Roles of Standards-Based K-12 ELP Assessments Key Research Areas for Supporting Best Practices of ELP Assessment Defining an ELP Construct Sustaining Validation Research Using Technology in ELP Assessments Connecting ELP Assessment and Instruction Concluding Remarks References SECTION I. Contexts and Fundamental Considerations in the Development of K–12 ELP Assessments Standards-Based K–12 English Language Proficiency Assessments in the United States: Current Policies and Practices Significant Court Cases and Legislation Relevant to ELP Assessment Lau v. Nichols Castaneda v. Pickard No Child Left Behind Act of 2001 Common Core State Standards Every Student Succeeds Act Characteristics of Current ELP Assessments Assessment Content Language Domains Standardized Content Content That Reflects Language Development/Growth within and across Grades Assessment Consortia and Independent States Consortia Independent States Variation in Approaches to Challenges in Assessment Administration Assessment of Very Young ELs Computer-Based Assessment One-on-One Assessment Concluding Remarks Notes References The Construct of English Language Proficiency in Consideration of College and Career Readiness Standards The Federal Policy Context of ELP Standards and Assessments Role of Language Development Theory in Exploring the ELP Construct Understanding Contexts of Language Use Understanding Language Functions Understanding Linguistic Complexity Integrated Language and Literacy Skills Understanding Individual Differences and Learners' Characteristics Evolving ELP Standards and Constructs in Relation to Academic Content Standards Identifying Language Knowledge and Skills in CCR Standards and ELP Standards Examples from CCR Content Standards Examples from ELP Standards Implications for Determining the Construct(s) to Be Measured by ELP Assessments Concluding Remarks Acknowledgments Note References Innovations and Challenges in K–12 English Language Proficiency Assessment Tasks Overview of Three Major K-12 ELP Assessments in the United States Innovations in K-12 ELP Assessment Tasks The Inclusion of Technology-Enhanced Items The Provision of Scaffolding in Tasks Universal Tools Embedded in Assessment Challenges and Suggestions for Innovative K-12 ELP Assessment Tasks Developing Age-Appropriate Innovative Tasks Developing Appropriate Test Administration Procedures for Innovative Tasks Scoring Innovative Constructed-Response Tasks Conclusion References A Review of Validity Evidence on K–12 English Language Proficiency Assessment: Current State and Future Direction Background and Theoretical Frameworks for ELP Assessment Validation Validity Evidence Collection and Regulatory Guidance Evidence Based on Test Content Evidence Based on Response Processes Evidence Based on Internal Structure Evidence Based on Relations to Other Variables Evidence for Validity and Consequences of Testing Reliability and Other Technical Quality Particular Challenges for Validating ELP Assessment Systems Validation Research on Reclassification Decisions Attention to Consequences of ELP Assessment Use Assessing Young English Learners Inclusion of ELs with Disabilities in ELP Assessments Conclusion References SECTION II. Empirical Research and Practical Considerations on the Development of K–12 ELP Assessments Collaborating with Educators on the Development of English Language Proficiency Assessments Key Parties Involved in the Design and Development of K-12 ELP Assessments Classroom Educator Involvement in the ELPAC Pilot Study of Task Types Item Writing Workshops Item Review Panels Sample Selection to Support Constructed-Response Scoring Support for Score Reporting: Performance Level Descriptor Reviews and Standard Setting Performance Level Descriptors Standard Setting Impact of the Involvement of Educators Recommendations for Successful Involvement of Educators in Practice Note References Examining Students’ Response Processes in a Computer-Based English Language Proficiency Assessment Relevant Literature Advantages of Computer-Based ELP Assessments Validity Evidence Based on Response Processes for Computer-Based ELP Assessments Study Contexts and Research Questions Method Participants Study Instruments Item Types Included in the Study Tutorial Development for the Study Procedure Analysis Findings Clarity of Directions and Types of Difficulties Students' Interaction with TE Features Recommendations and Actions Taken Based on Findings Conclusion and Future Research Notes References APPENDIX: Cognitive Lab Interview Questions and Rating Scales Generating Score Reports: Psychometric Issues to Consider The Summative ELPAC Test Structure and Score Report Psychometric Analyses to Generate Score Scales Overview of Dimensionality Analysis Overview of Vertical Scaling Analysis Analysis of the Summative ELPAC Field Test Results Study One: Dimensionality Analysis Study Two: Vertical Scaling Analysis Discussion References Validating Threshold Scores for English Language Proficiency Assessment Uses Contexts of the Present Study: Timing for Setting and Validating Performance Levels ELPAC Standard-Setting Design of the Standard-Setting Workshop Standard-Setting Results and Recommendation for Validation Studies Study 1: Threshold Score Validation Study Using the Contrasting Groups Method Study Sample Study Findings Study 2: Analyses of the State ELP Assessment in Relation to the State ELA Assessment Data Sources Methods Distribution of EL students by Overall Performance Level on ELPAC: Two Rounds of Analyses EL students' Performance on the State's ELA Assessment by Overall ELP Performance Level: Two Rounds of Analyses Results from Descriptive Box Plots: Two Rounds of Analyses Summary of Findings from Contrasting Groups and Two Rounds of ELP/ELA Analyses Comparing Performance on the State's ELA Assessment of EL students with ELPAC Overall PL 4 to Similarly Situated EO Students and New RFEP Students Summary and Discussion Note References Accessibility Considerations for English Learners with Disabilities in English Language Proficiency Assessments English Learners with Disabilities ELP Assessment Context for ELSWDs Assets-Based Perspective Variability Designing Accessible Assessments Accessibility in ELP Assessment in Current Practice Assessment Accommodations Complex Interactions between Construct, Disability, and Accommodations Validity and Fairness of ELP Assessments for EL students with Disabilities: An Example from One State An ELPAC Cognitive Lab Study Validity and Fairness Issues Enhancing Validity and Fairness of ELP Assessments for ELSWDs Recommendations to Improve Accessibility of ELP Assessments Clearly Define the Constructs to Be Measured in an Inclusive and Universally Accessible Manner Identify the Range of Test Taker Characteristics from Disabilities to Accommodations in Need Include a Multidisciplinary Design Team Consider Appropriateness, Efficacy, and Opportunity to Learn in Accommodation Assignment Determine Whether Domain Exemptions Will Be Needed Develop Guidance for Which Students May Need an Alternate Assessment Attend to Accessibility at Every Stage of the Assessment Design and Development Process Implications for Future Research, Policy, and Practice Note References SECTION III. Some Future Considerations in K–12 ELP Assessment Using Automated Scoring in K–12 English Language Proficiency Assessments Benefits and Challenges of Automated Scoring Automated Scoring System Components Preprocessing Feature Extraction Scoring Model Filtering Model Automated Scoring Use Cases Extended Written Response Scoring Spoken Response Scoring Practical Considerations and Recommendations Assessment Construct and Task Design Considerations Scoring and Score Reporting Considerations Automated Scoring Model Development and Test Delivery Considerations Conclusion Note References Teachers’ Perspectives on the Use of Summative English Language Proficiency Assessments for Instructional Purposes Current Summative ELP Assessments Educators' Views and Uses of Summative ELP Assessments Assessment Timing Classroom Practitioners' Use of the Summative ELP Scores Purpose of the Study and Research Questions Methods Participants Procedures Analysis Findings Research Question 1: How Did the Content and Delivery Mode of a Summative ELP Assessment Influence Teachers' Classroom Practices?Content of the ELP Assessment Delivery Mode of the ELP Assessment Research Question 2: What Types of Instructional Decisions Did Teachers Make Based on Their EL students' Scores on the Summative ELP Assessment?To Inform Assessment Practices To Make Placement and Grouping Decisions To Measure EL students' Progress over Time To Inform Other Teachers in the School about How to Support EL Students Research Question 3: What Factors Affected Teachers' Use of Student Scores on the Summative ELP Assessments for Instructional Purposes?Timing of the Score Reports Lack of Detail in the Score Reports Score Reports Not Meaningful to Students Validity Concerns about the Speaking Scores Discussion Implications for Score Reporting Limitations and Directions for Future Research References Toward a Model of Validity in Accountability Testing Reform Policies and Accountability Testing Validity in the Standards: Sources of Evidence Kane's Argument-Based Validity Approach Bachman and Palmer Assessment Use Argument Validation and Accountability Consequences and Accountability Testing: A Shared Responsibility Accountability Testing and Anticipatory Research: Social Impact Analysis and Theories of Action Concluding Remarks Note References