Tuesday, September 3, 2013

Report Cards that Offer Zero Useful Information

Years ago, my niece gave a colorful report on her aggravating and disappointing trip to see the pyramids with the title “I Went to Egypt so that You Don’t Have To.” My trip through the labyrinth to try to understand the new school accountability system was similarly aggravating and disappointing. No, I did not have people trying to pick my pockets throughout, but I ended feeling just as ripped off and disgusted. Here is my report. Do not expect much local color.

In compliance with federal Race to the Top legislation, Michigan has now implemented a new Accountability Scorecard for every school and district, giving them color-coded ratings ranging from red (terrible!) to green (terrific!). The great majority (more than two-thirds) were rated yellow (caution: red is imminent!). This has been confusing to all, since many educational entities appeared to be doing well in every sub-category, yet they did not achieve a green rating overall. How can that be?

The short answer is that the system is rigged to produce failure. And that is exactly what it did.

The system requires 85% of children to become proficient in every subject by the end of the 2021–22 school year, mandating incremental progress toward that goal in every subgroup every year. The subgroups include students with disabilities, English Language–learners, economically disadvantaged students, the bottom 30% (in terms of proficiency), and various ethnic groups (white, black, Hispanic of any race, Asian, Native Hawaiian or Pacific Islander, American Indian or Alaska Native, and multiracial). In addition, there are requirements regarding test participation rate, attendance rate, graduation rate, and compliance with school improvement, educator effectiveness, and other kinds of reporting.

The state’s database lists 5,816 schools and districts to be rated. Of that number, 1,144 (almost 20%) are given no color/rating, mostly because they are too new to have the requisite historic data trail. The lowest, red, rating was given to 692 (14.8% of those given a rating). Another 263 (5.6% of rated entities) were rated orange. A whopping 3,180 or 68% were rated as yellow. None were given the lime green rating. And a tiny 135 (2.9% of the total rated) got the coveted green designation. To summarize, then, more than 97% of schools and districts rated are considered “failing.”

But the news is actually worse than that. Of the 135 green schools, 41 (nearly one-third) got “zero of a possible zero points” — so how is it that they are rated green? Most are so designated on the basis of three-year participation rates (how many children actually took the tests) and “compliance factors” (planning and reporting requirements that earn no points). They have no student test scores because all their grade cohorts are under 30 pupils — a prime indicator that they are likely to be charter schools. Other “green” schools with very low point totals got them for such factors as student attendance. You will note that student attendance, test participation rates, school improvement planning and teacher evaluation reporting are ALL FACTORS THAT HAVE NOTHING TO DO WITH STUDENT ACHIEVEMENT.

Looking more closely at the data, the news is even worse. Here is an example: the Jalen Rose Academy is Green on the strength of Student Attendance (2 of a possible 2 points) and Compliance Factors (no points). Its only assessment participation rate is a three-year average for social studies — which made me realize it is just three repetitions of the sixth-grade participation rate they brought with them to this high school. How is that at all relevant to this school?

This exposes the glaring hole in the rating system — and the reason nearly every entity will be “failing” in short order: it requires incremental improvement in student test scores every year, but not all tests are given every year. So, students who were not proficient in social studies in sixth grade are unable to show improvement until they take a social studies test again in ninth grade. Students who were not proficient in science in fifth grade will be unable to show improvement until the eighth grade test. For three years they WILL fail to achieve the required incremental progress. Students who are not proficient at writing in seventh grade will not be able to demonstrate progress for four years, until they take the ACT in eleventh grade. Any one such failure mandates a yellow rating, at best, for the school and district.

The system also requires a steady progress (equal increments over each of the next ten years) toward proficiency targets. The farther away a group is in a particular subject, the larger those increments will be. In the subject in which the state’s students do most poorly overall, science, the improvement targets will be almost impossible to reach. If 15.9% of eighth graders statewide were considered proficient in 2012 (they were), and 85% must achieve proficiency by 2022, then the proficient group must increase by 6.9 points [(85 - 15.9)/10 years] every year. If the target is missed one year, it just gets higher the next. Improvements of that magnitude are almost never achieved anywhere, let alone consistently every year for a decade. If not one of a particular subgroup was proficient in 2012, as was true in many places for special education or bottom 30% groups in science, then their improvement target will be even higher: 8.5% per year — which simply cannot be done for ten years straight. It will be impossible to achieve green status once student achievement data is used for all schools, unless those schools already have elite, selective populations that are universally high-achieving. The only way to guarantee that is to exclude special education, English Language–learning, and economically disadvantaged students — which is exactly what is occurring at charter schools today.

Another wrinkle — or perhaps “monkey wrench” is more evocative here — in the system is that cyber schools can now provide one-third or more of educational classes for grades five through 12, but the home school district will be held accountable for student achievement results. Let me repeat: traditional school districts will be punished for the failures of on-line schools over which they have zero control.

Presumably, the point of the rating system is to give parents the information they need to exercise the vaunted “choice” that is supposed to be making all schools more effective. If the basis for the ratings is so hard to understand, and if all are rated as failing — as they soon will be, how is this helpful to anyone?