----
Dear Friends,
For all the angst he has caused and the retirements he has motivated with his tactics, Tony Bennett’s record on raising test scores and graduation rates is essentially on par with the record of his predecessor, Suellen Reed.
In two campaign commercials and in his “State of Education” address on September 25th, he has touted raising both graduation rates and test scores. Let’s look at the precise record.
Graduation Rates: A Continuation of Steady Improvement
Graduation rates have been the fodder for political campaign debates since the Frank O’Bannon-David McIntire race for Governor in 2000. After great squabbles over the formula for determining graduation rates, the Indiana General Assembly finally authorized a count based on giving every student a number and then tracking every student who stays in that school to see how many graduate in four years. If they take five or six years to graduate, as in the case of many special education students, they do not count in the graduation rate.
The formula was clarified and students tracked from 9th to 12th grades beginning with the Class of 2006. Here are the results of the six years IDOE has reported the revised method:
Year % Graduating in Four Years % Drop Outs
Class of 2006 76.1 11.2
Class of 2007 76.4 11.9
Class of 2008 77.8 10.3
Class of 2009 81.5 8.7
Class of 2010 84.1 6.4
Class of 2011 85.7 6.1
Data for the Class of 2012 is not expected until December. The columns above do not add up to 100% because the new method of tracking every student revealed that an unexpectedly large number of students, as many as 7 or 8 percent, stay in school for a 5th or 6th year. Special education students have the right to stay in high school until their 22nd birthday. Others leave school without graduating and without dropping out by completing a special education certificate or by completing a course completion certificate, given to those who did not pass the required high school graduation tests.
Tony Bennett has complained that too many schools are granting graduations based on the waiver law. Waiver graduations occur when students do not pass the high school graduation tests in math and English but do meet the alternative list of criteria set up by the General Assembly for graduation. The General Assembly passed legislation authorizing waiver graduations after parents appealed to legislators to not allow a single test to nullify a four-year record of successful course work and achievement. The percentage of diplomas based on waivers has ranged between 5% and 8% since 2006. Waiver diplomas are the subject of much debate, but the focus here is on the total graduation figures listed above because those are the figures Dr. Bennett has cited in his campaign.
The next question is how the above scores should be divided between Dr. Reed and Dr. Bennett. This is a matter of judgment. Since Dr. Reed was in office for seven of the eight semesters for the Class of 2009, I include 2009 data with Dr. Reed’s record. Since Dr. Bennett took office in January of 2009 before the Class of 2009 graduated in June, I also include the Class of 2009 with Dr. Bennett’s record. Using these guidelines, the comparison comes out as follows:
Dr. Reed (2006-2009):
- Graduation rate 76.1% in 2006
- Graduation rate 81.5% in 2009, up 5.4% in 3 years after her baseline year.
- Drop out rate 11.2% in 2006
- Drop out rate8.7% in 2009, down 2.5% in 3 years after her baseline year.
- Graduation rate 81.5% in 2009
- Graduation rate 85.7% in 2011, up 4.2% in 2 years after his baseline year.
- Drop out rate 8.7% in 2009
- Drop out rate 6.1% in 2011, down 2.6% in 2 years after his baseline year.
That leaves an important question: What did all of Tony Bennett’s comparative turmoil and controversy accomplish if his record is about the same as that of Suellen Reed?
Test Scores: A Conveniently Timed Adjustment
The record on ISTEP+ scores is, as always, more complex.
The Indianapolis Star took Tony Bennett to task in his “State of Education” address for saying that the ISTEP+ scores went up by 8 percent, from 63% to 71%, for those passing both math and English parts of the test. He was, the Star said, supposed to say the scores went up by 8 percentage points, not 8 percent. A change, they said, from 63% to 71% was a 12% increase.
That was the first fact-checking on Tony Bennett I had seen in the Star in a long, long time. I wish, however, they had discovered a deeper problem.
The deeper concern was buried in the fine print of the 2011 ISTEP+ results. Many of you have seen the report I produce every year, currently showing 22 years of year-by-year data on improvement in Indiana’s public schools. When I was preparing the most recent report last spring, which is attached for your reference, the Spring 2011 results carried a surprising advisory, saying that the 2011 results could not be compared to the Spring 2010 results. The reason given was that in 2011, the “Undetermined” scores were removed from the analysis.
At this point, you are asking: What does all this mean?
Undetermined scores are for students who have only partially completed the test. This could be due to students getting sick before the test is completed or to any number of calamities that could occur when students are tested: weather problems, floods, family emergencies. Schools have seen it all.
Until the Spring 2011 tests, the long term practice throughout Dr. Reed’s time in office was to include undetermined scores in the denominator when percent passing was calculated. In effect, that meant undetermined scores were treated as failing scores. Suddenly, in 2011, a different decision was made. Undetermined scores were set aside and excluded from the analysis. They were left out of the denominator. The net effect of that decision was to lower the denominator and raise the percentage passing totals across the board by about 2%. That is why the advisory said that comparisons between 2010 and 2011 should not be made. It was a “one-time in history” 2% bump up in the scores.
Now fast forward to 2012. The new results issued last summer again excluded the undetermined scores. Therefore, 2012 results can appropriately be compared with 2011 results. However, Tony Bennett in his speech compared the 2012 results (“71%”) with the 2009 results (“63%”). That would be an inappropriate comparison due to the change in IDOE policy on undetermined scores. Instead of a change of 8 percentage points, it should be reported as a probable change of 6 percentage points, since changing the policy on undetermined scores lifts all passing percentage figures by approximately 2 percentage points.
Like I said, the ISTEP+ story is more complex.
The interesting question for speculation is why the change was made. I have no inside information. I don’t even know if Tony Bennett is aware of this nuance produced by his testing staff. All I can do is observe that the change was made in one of the middle years of his term so Tony Bennett would not have to see words saying “comparisons should not be made to the previous year” in 2012. It was timed just right to show a slightly bigger gain (about 2% bigger) since the 2009 tests.
Website Changes
After telling you the story of undetermined scores, I planned to tell you the story of my independent efforts to verify the gains claimed on test scores. My framework was to be the 22-year review, which is attached. The analysis in that document counts, grade by grade, the number of valid year-by-year comparisons that went up, the number that went down, and the number that stayed the same.
To my dismay, when I went to the IDOE website to get the 2012 data to bring my tables up to date, the data I have used for years was not on the website. I have used ISTEP+ reports that were statewide grade by grade disaggregated totals. They are no longer available on the IDOE website.
IDOE has changed their website several times, starting in 2010. I don’t know anyone who thinks the changes have been in the direction of more transparency and more data available. Instead, there is less available.
Then when I went to the archive of previous ISTEP years, the archive files did not open. It could be a problem with the IDOE website or it could be a problem with my trusted computer, which has opened everything else. Draw your own conclusions.
Summary
My quest to verify graduation rate and test score gains during Tony Bennett’s years in office has ended in two conclusions:
- Graduation rate increases during Dr. Bennett’s term are about on par with graduation rate increases while Dr. Reed was in office.
- The switch in 2011 to excluding undetermined scores from the calculations provided an approximate 2% boost to all percentage passing levels.
A key question that must be asked is whether the gains in test scores and graduation rates during Tony Bennett’s four-year term have come at a cost. Did he sacrifice in other areas to bring these gains, areas that Suellen Reed did not sacrifice? What were the opportunity costs of his intense focus on math and English tests?
I will explore that important question in the next “Notes.”
A Strong Endorsement for Glenda Ritz
The Fort Wayne Journal Gazette strongly endorsed Glenda Ritz in their October 14th editorial entitled “Time to Halt Bennett’s Ambitions.” I agree with everything they wrote, from their opening words, “Indiana public schools are struggling under the leadership of Tony Bennett” to their closing words, “Voters should compare the leadership styles and results of a politically ambitious superintendent versus his two widely respected predecessors. Ritz promises to serve more in the mold of Evans and Reed; she’s the easy choice for Indiana’s top education post.”
Please talk with neighbors and friends who are not familiar with education issues to enlist their support for Glenda Ritz. That is a vital step to counteract the enormous advantage in TV advertising held by Tony Bennett and his campaign that has been so well funded by out-of –state interests. Your actions will make all the difference if Glenda Ritz is going to win in this grassroots campaign.
Best wishes,
Vic Smith
“Vic’s Election Notes on Education” is not linked to any organization and is not being distributed by me to any organization. It is only being distributed to those who have previously sent personal requests for my commentaries. If you want to pass it along to others, you do not need to ask my permission. If you want to be taken off the distribution list, just let me know. If you know of others who want to be added to the list, just send me an email.
