If I told you that North Carolina had one of America’s top-ranked systems of public education, there are at least three ways you could respond.
You could question what I mean by “top-ranked system.” You could challenge the factual accuracy of my claim. Or you could contest the policy implications of the claim — does it suggest that the education reforms implemented by a mostly Republican legislature are succeeding, or does it suggest that they were never warranted in the first place?
Definitions, premises, and logical conclusions are the three elements of any argument, political or otherwise. Often, we jump to debating the third element too soon, alleging that the other side is being “illogical” or refusing to let facts get in the way of dogmatic insistence on preexisting beliefs. But what are the facts, anyway? And are we even talking about the same things?
I’ll apply these distinctions to the topic at hand. When I say that North Carolina has a top-ranked system of public education, what I mean is that, according to measures of student outcomes adjusted for student background, North Carolina’s schools seem to add more value than do schools in most other states.
If you look at test scores, graduation rates, or other outcomes without adjusting for student background, you’ll find, not surprisingly, that states with relatively low rates of poverty, students or parents with native languages other than English, and single-parent families have relatively high academic outcomes.
The circumstances in which kids grow up have a huge effect on their academic achievement and attainment. Schools matter, too, of course — or else we ought to reconsider all the time and resources we devote to them! But if we want to have a meaningful conversation about education, we must at least try to isolate the effects of schooling from the effects of other influences.
With that definition in mind, here’s one piece of evidence I’ll cite for my factual claim. Every two years, the National Assessment of Educational Progress tests a representative sample of students across the country in 4th- and 8th-grade reading and math. Using the Urban Institute’s handy tool for adjusting the 2017 NAEP results for age, race or ethnicity, native language, disability, and poverty, I determined that only four states ranked in the top 10 on all four tests: Massachusetts, Florida, New Jersey, and Indiana. Three other states — Virginia, Georgia, and North Carolina — were in the top 10 on three of the four tests.
If we treat that three-of-four standard as the entrance requirement for the club of leaders in educational value-added, North Carolina’s membership may strike some observers as incredible. If you want to argue that consistently high test scores aren’t meaningful, or that being in the top 10 in America isn’t impressive by international standards, be my guest.
In the meantime, however, I’ll proceed to the third element of the argument. Can we draw any logical policy conclusions? When I ran the same analysis for the 2011 NAEP scores, North Carolina was in the same club, with at least three top-10 rankings, along with almost all of the same members. The only change was that by 2017, Maryland had dropped out while Indiana had surged in — the latter having had no top-10 rankings in either subject in 2011.
These facts suggest a couple of reasonable hypotheses for further testing. First, Indiana may have enacted some critical reforms in the 2000s that significantly improved the effectiveness of its schools. And second, North Carolina’s relative high performance has neither been caused by Republican-enacted policies nor been injured by it.
It takes many years for even sweeping education reforms to affect student performance. Actually, since the late 1990s, the mix of leading states in educational value-added has changed dramatically. Astoundingly, not a single member of the 2017 club was a top-ranked state in adjusted NAEP scores in the late 1990s, including North Carolina.
In policy debates, as in so many other settings, patience is a virtue — and, alas, a rarity.
Editor’s Note: The John William Pope Foundation supports the work of EdNC.
