Hard Lessons from the New NAEP Results

It was the worst of times

Scores from the National Assessment of Educational Progress (NAEP) were released on January 29. How were they? Headlines used terms like “disheartening,” “new low,” and “even worse.”

It was that bad. Average performance dropped in reading and barely improved in math.

Here’s what we learned:

#1. The education depression continues

I wrote in October that we have been mired in shrinking academic outcomes for more than a decade, exemplified by a surge in students with very low performance levels. From 1990 to 2013, we made massive progress in that area. By 2022, most of it had disappeared.

The latest scores confirm that the depression rolls on. Below is an updated version of a graph I made in the fall. For each racial group, we have just as many eighth graders scoring Below Basic as we did in 2022 and far more than we had in 2013.

All data taken from the NAEP data explorer, https://www.nationsreportcard.gov/ndecore/xplore/NDE

Five years after the onset of the pandemic, there has been virtually zero recovery. We’ve seen this across assessments. It’s not an illusion.

#2. Mediocrity rules

Massachusetts and Wisconsin tied for the top score in 8th grade math at 283. They are justifiably proud of their standing. But let’s put their performance in perspective. In 2013, that same score of 283 would have placed 30th. Let that marinate for a moment. Math achievement has fallen so far in 11 years that what would once have been an unremarkable score now leads the nation.

Reading is similar. The 2024 fourth grade leader - again, Massachusetts - would have tied in 2013 with Indiana, Washington, and Maine… for 14th.

Another troubling sign is a lack of positive outliers. In 2013, only five states were within 10 points - which is approximately one year’s learning - of Massachusetts in 8th grade math. It stood head and shoulders above the pack. This year, 24 states were within 10 points of Massachusetts and Wisconsin.

There is a golden opportunity for a handful of focused, aggressive states to remake the top of the leader board by improving just a handful of points. Start your engines.

#3. If it seems too good to be true, it probably is

Two years ago, Los Angeles Unified posted a shocking gain of nine points on the eighth grade reading test. This was unexpected for several reasons. First, pandemic school closures lasted longer in L.A. than most cities. Second, reading scores nationally dropped by three points in 2022 and not a single state improved. Yet, L.A. didn’t just improve - it soared.

When skeptics wondered if something might be amiss with the results - an error in selecting the sample of students to be tested, perhaps - Superintendent Alberto Carvalho publicly compared them to flat-earthers. He maintained that the gains were real and attributed them to distribution of technology during pandemic shutdowns, tutoring, and teacher training.

There was some awkwardness last week when 2024 NAEP results showed that LAUSD’s eighth grade reading score dropped by eight points, wiping out the 2022 improvement almost entirely. Carvalho, apparently with a straight face, told the Los Angeles Times that “sampling methods can lead to anomalies.”

#4. Money matters, but it’s not mattering enough right now

Most research finds that providing more funding for schools leads to better outcomes such as higher test scores and more college-going. Just last summer, two separate studies found that the federal infusion of $190 billion for COVID recovery was helping.

Along comes these new NAEP results. We spent an unprecedented amount of money. And results are very discouraging.

Cue Edunomics Lab, based at Georgetown University. Marguerite Roza and her team examined each state’s return on investment (ROI) by comparing changes in spending since 2013 to changes in academic outcomes.

It’s ugly.

Consider Oregon. Had it merely kept pace with inflation, it would have increased school spending by about 35 percent from 2013 to 2023. In actuality, it raised spending by 80 percent. Over the same period, math and reading performance tanked, with math posting a remarkable 16-point decline - the equivalent of 1.5 grade levels. Oregon is spending much more and achieving much less.

Oregon is not alone. You can view all 50 states here. Washington and Vermont are pretty scary. Mississippi has bucked the trend with smaller spending increases and real gains in reading.

Here’s the bottom line: The case for investing more in schools is very weak right now because it’s not clear that new resources will yield better results. All of us who care about education have a stake in changing that storyline.

#5. Baghdad Bob is back and better than ever

Baghdad Bob was easily the best part of the Iraq War. Real name Mohammed Saeed al-Sahhaf, he served as Saddam Hussein’s press chief. While American soldiers and tanks streamed into the capital city right behind him, he calmly reassured television cameras that no such invasion was happening. It was PR performance art.

I suspect Bob is now writing press releases for state education agencies. In the face of dismal national NAEP results, almost every state claimed victory and did a goofy, unconvincing touchdown dance.

Wyoming’s performance dropped on three of the four NAEP tests and was flat in the other. Its official statement, which makes no mention of this, says Wyoming “continues to stand out nationally.”

It was a popular move. The logic? Because other states did poorly, our state is doing awesome by comparison - even if our students are learning less than before. Shout outs to Illinois and Washington, among others, for going all in.

Florida, which had particularly bad reading results, goosed the ante by having its state chief, Manny Diaz, write a letter saying NAEP’s methodology is wrong and calling for the Department of Education to be abolished.

The only thing more disappointing than the NAEP results themselves has been the state response. Truly a low point for our sector. We are hiding from the truth about what we are providing for our students. Let’s do better, friends.

#6. Demographic adjustments change the picture

Fueled by coffee and chutzpah, Matt Chingos and Kristin Blagg from the Urban Institute published demographically adjusted NAEP results the same day as the national release. It’s a great resource.

The method here is to compare a state’s actual NAEP scores to what would be expected given that state’s student population. Factors considered include gender, age, race/ethnicity, and status for free lunch, special education and English language learner services.

It produces a very different leader board. Mississippi, for example, finished 13th in fourth grade math and 7th in fourth grade reading in the official NAEP rankings. Not bad! But adjusting for demographics, they finished first in both.

On the other end, New Hampshire’s fourth grade reading score was the fourth highest in the country. Once adjusted, though, they dropped to 24th.

This is yet another indication that some states are facing and meeting greater challenges while others might be achieving less than meets the eye.

What now?

The new test results suggest we’ve been chasing the wrong ghost for years. This isn’t about recovery from deficits that festered during COVID Zoom school.

The bigger picture is a prolonged slide dating to 2013 - especially among lower performing students. We’ve always had an achievement gap. It was getting worse prior to the pandemic. Since 2019, it has exploded and we can’t tell yet if it’s done growing.

Some districts and states have no plan to reverse the sad patterns because they barely acknowledge the patterns exist.

Patience is waning. But it can’t fall on system leaders alone to fix this. They need help. They need better insights on what caused our depression and better ideas for reinvigorating instruction. We’ve all got a role to play.

For now, our shared job is to be honest: This isn’t good enough.

---

Bonus reading and watching

• Watershed Advisors created a very handy dashboard that allows users to review individual state scores and changes over time, with filters. Highly recommended.

• In June 2022 - before the 2022 NAEP results had even come out - Mike Goldstein argued that learning declines would continue and NAEP 2024 scores would be even worse than 2022. He was exactly right. More importantly, Mike knew why. In speaking with school leaders on the front lines, he was hearing a stream of concern about low student agency, labor shortages, and rising disorder. The quality of teaching and learning was not high enough to reverse learning losses.

• I discussed the NAEP results on LinkedIn with

  Andrew Rotherham
  and
  Denise Forte
  . Really enjoyable conversation with two folks who know the evolution of state and national policy as well as anyone. View here.