Guide to Early Literacy Benchmarks and Use
On August 22, 2017, the Department provided all Iowa districts with their 2016-2017 early literacy screening data. These data were calculated using the benchmarks originally set in 2015-2016, which we are using as the baseline year to measure progress for the state.
You may have questions about why the 2015-2016 benchmarks are being used to measure progress over time, and you may get questions from others, as well. This document provides a set of questions and answers to help AEAs, districts, and schools work with the recently released data.
Question 1: Why are you using the 2015-16 benchmarks instead of the 2016-17 benchmarks?
Longitudinal data - which allows us to keep track of student performance and growth over time -- is essential to our ability to evaluate the impact of Iowa’s early literacy initiative. This is why we’re tracking long-term growth on the early literacy screening assessments using benchmarks set during the 2015-16 school year. 2015-2016 was the first year during which we had complete data for the K-3 early literacy work, so the benchmarks set that year represent the baseline for measuring statewide progress in the years to come.
The benchmarks we use to judge our progress against need to remain the same over time. Otherwise it would be like moving the bar while trying to improve.
Imagine we are trying to improve our students’ performance on the high jump and in the first year we set the bar at 4 feet - and call that proficient. We can measure the percent of our 3rd graders who can jump over that bar. If we want to know how well our PE teachers are doing over time at improving students’ high jump skills, we cannot set the bar at 41/2 feet in the second year and accurately judge whether we made improvement. We have to hold the bar at 4 feet each year, so we have a fair comparison.
Question 2: Isn’t that holding districts to a different standard for making decisions about kids?
No. They are different benchmarks for different decisions.
One is to track state progress over time, the other is to determine if an individual student needs additional support.
Assessment vendors often alter benchmarks over time to try to decrease the error associated with the measures. FAST literacy measures already have very low error. Decreased error is nice, but not likely to substantially change which students get additional support, on balance. This is particularly true since in Iowa we have defined persistently at risk to be below benchmark for two screening periods in a row.
Small changes in the benchmarks do add up to big changes in state-level data, however. This is why it is important to keep a single set of benchmarks for measuring progress over time. This is true for districts, as well, but the effects are amplified even more at the state level.
Question 3: Will the benchmarks in the FASTBridge system continue to change every year?
Possibly. The benchmarks have been set for the 2017-2018 school year, and they have not changed from the 2016-2017 school year. The vendor may choose to alter the benchmarks in future years based on their research. We will keep districts informed of any changes and how they may affect local decision-making.
Question 4: How did the benchmarks change from 2015-2016 to 2016-2017?
There were very slight changes in several of the benchmarks, with the largest change being an increase in the benchmark at first grade on the aReading measure. As stated in the response to Question 2, above, these small changes have small effects on which students receive additional progress monitoring or intervention, but add up to large effects at the state level. Similarly, they will have larger effects in the aggregate in larger districts versus smaller districts.
Question 4: How can I see the difference in my district’s data with the 2015-2016 benchmarks versus the 2016-2017 benchmarks?
You could make this comparison by getting the results from the Iowa TIER data you downloaded in the Spring - which would have used the 2016-2017 benchmarks - and comparing it to the data provided to you recently by the Department. We strongly caution against making this comparison, since it is not a fair comparison over time when benchmarks are altered.
Question 5: Will the Department continue to provide the data using the 2015-2016 benchmarks so districts can track progress over time?
Yes, at the end of each year these data will be provided statewide. During the year districts should continue to use the data within the FastBridge system (if using the FAST measures) to track progress and make decisions for individual students.
Question 6: Nonpublic school data were not provided, how can I get those data?
We did not have immediate access to the nonpublic school data, but will provide a revised list as soon as it is available.