Over-the-Counter Data’s Impact on Educators’ Data Analysis Accuracy
Roundtable Presentation to Be Given at the International Society for Technology in Education (ISTE®) Conference
June 29, 2014 (4:15-5:15 P.M. EST)
Event Information: https://isteconference.org/2014/program/
Objectives of Research Being Presented: The purpose of this experimental quantitative study, conducted in a laboratory environment, was to facilitate causal inferences concerning the degree to which including different forms of data usage guidance within a data system reporting environment can improve educators’ understanding of the data contents, much like including different forms of usage guidance with over-the-counter medication is needed to properly communicate how to use its contents. Independent variables included brief, cautionary verbiage in report footers, report-specific abstracts, and report-specific interpretation guides. The dependent variable was accuracy of data analysis-based responses. The researcher explored three data analysis supports provided by a data system, each framed in two different formats, by presenting 211 elementary and secondary educators in ethnically and culturally diverse southern California with different versions of the same two student achievement data report environments. Each of these report sets fit into one of the following treatment categories (a) no added analysis support; (b) analysis support by way of footers directly on the reports, which were offered in two different framing styles; (c) analysis support by way of abstracts, which were 1-page reference sheets that accompanied the reports and were offered in two different framing styles; and (d) analysis support by way of interpretation guides, which were 2- to 3-page reference guides that accompanied the reports and were offered in two different framing styles. The researcher then compared the results of educators using data system reports embedded with data analysis guidance in the varied formats noted above (a-c). Participant responses were collected in person while using a web-based questionnaire crafted and administered in Google Docs, taking advantage of the Google Form feature, and involved groups of no more than 30 respondents at each administration time at each participant’s school site. Data was collected at one point in time for each participant within a one-month research window. Findings from this research are suited to identify whether data systems used by educators can help prevent common analysis mistakes by providing analysis support within the interface and the reports they are used to generate.