The 35th Language Testing Research Colloquium
Untitled Document
Title: Developing English Language Proficiency Tests: An Integrated Framework for Understanding the Steps
Date: July 1, 2013
Instructor: Dorry M. Kenyon, Center for Applied Linguistics
The development of an English language proficiency test is an interdisciplinary effort requiring attention to the technical requirements of language testing and measurement specialists (among others), as well as to the needs and requirements of various stakeholders, including test-takers, test scores users, policy setters, and researchers. Such a large effort requires a principled approach in order to be carried out successfully.
In this workshop, participants will be introduced to the integrated steps, or stages, in the development of an English language proficiency test. Within each stage, the outcome of that stage will be clearly delineated, as well as a discussion of various state-of-the-art approaches (that is, the what that needs to be done) to achieving each outcome and the roles and qualifications of the participants (that is, the who is qualified to do the work in that stage). But above all, particular attention will be paid to understanding the why of each stage, particularly within a framework of building an assessment use argument to support the validity of the use of the test scores. Providing such an overarching framework will enable language testers to understand their work in each stage as part of an integrated whole. Particular emphases will be placed on the stages preceding the operationalization of the language test: laying the foundations, achieving consensus, developing specifications, creating prototypes, conducting pre-operational testing research, and making the test operational.
The workshop will be conducted as a series of short presentations on each stage, within which exercises will make clear to participants the critical issues to be addressed in each stage. The role and contributions of language testers to each stage will be highlighted. Illustrations will be provided by the presenter's experiences in developing large-scale English language proficiency assessments for students in grades K-12 used by over 28 states for the WIDA Consortium ( and the new ACCESS 2.0 assessments being developed by the ASSETS Consortium ( At the end of the workshop participants will be able to describe the outcomes and articulate a rationale for each step involved in an integrated framework for language test development.
Dr. Dorry M. Kenyon has considerable experience in all aspects of designing, developing, validating, and operationalizing language tests through many large projects at the state and national level since joining the Center for Applied Linguistics (CAL) in Washington, DC in 1987. He currently serves as CAL's Vice President for Programs and Director of CAL/WIDA Partnership Activities. At CAL, he currently directs a variety of projects related to the large-scale, high-stakes assessment of the English language skills of language learners from pre-school to adult. With a doctorate in Educational Measurement, he has also served as CAL's chief psychometrician. Active in research on language testing, Dr. Kenyon is particularly interested in the application of new technology to language assessment problems. Prior to joining CAL, he taught German and English as a Foreign/Second Language for seven years in the United States and abroad.
9:00 a.m. - 9:30 a.m. Introduction: The need for an integrated framework to developing English language proficiency tests
9:30 a.m. - 10:30 a.m. Stage 1: Laying the foundations
10:30 a.m.- 11:30 a.m. Stage 2: Achieving consensus
11:30 a.m.- 12: 15 p.m. Stage 3: Developing specifications
1:00 p.m. - 1:30 p.m. Stage 3: Developing specifications (con't)
1:30 p.m. - 2:30 p.m. Stage 4: Creating prototypes
2:30 p.m. - 3:30 p.m. Stage 5: Conducting pre-operational testing research
3:30 p.m. - 4:30 p.m. Stage 6: Making the test operational
4:30 p.m. - 5:00 p.m. Wrap-Up: Putting it altogether

Untitled Document
Untitled Document