Thinking about Your Audience in Designing and Evaluating Score Reports
National Council on Measurement in Education (NCME) Annual Meeting
April 7-11, 2016
Location: Washington, D.C.
Co-presenters: Priya Kannan (Chair), April Zenisky (Discussant), Ikkyu Choi, Amy Clark, Amanda L. Clauser, Meagan Karvonen, Neal Kingston, Emily A. Leibowitz, Spyridon Papageorgiou, Richard J. Tannenbaum, and Diego Zapata-Rivera
Session Handout: click here
Description: The information presented in score reports is often the single-most important point of interaction between a score user and the outcomes of an assessment. Score reports are consumed by a variety of score users (e.g., test takers, parents, teachers, administrators, policy makers), and each of these users have different levels of understanding of the assessment and its intended outcomes. The degree to which these diverse users understand the information presented in score reports impacts their ability to draw reasonable conclusions. Recent score reporting frameworks have highlighted the importance of taking into account the needs, pre-existing knowledge, and attitudes of specific stakeholder groups (Zapata-Rivera & Katz, 2014) as well as the importance of iterative design in the development of score reports (Hambleton & Zenisky, 2013). The papers in this session employ a variety of methods to identify and understand the needs of diverse stakeholder groups, and studies highlight the importance of sequential and iterative approaches (i.e., assessing needs – prototyping – evaluating usability and accuracy of understanding) to the design and development of audience-focused score reports. These collection of studies demonstrate how a focus on stakeholder needs can bring substantive gains for the validity of interpretations and decisions made from assessment results.