A Decade of PISA Scores Shows Alarming Achievement Gap Growth

This is the second in a series of two posts, the first of which can be found here.

Yesterday I showed that, during the pandemic, changes in achievement gaps for US 15 year olds on Program for International Student Assessment (PISA) were mixed, depending on the subject. In short, between 2018 and 2022, science scores on PISA were flat, but the achievement gap widened substantially. In math, scores fell substantially but the achievement gap was relatively stable. And in reading, scores didn’t change much, nor did the achievement gap. Given the expanding US achievement gaps evident on tests in younger grades, these uneven PISA results were a surprise to me. Unfortunately, however, although PISA achievement gaps over the course of the pandemic tell a mixed story, looking over a longer period suggests that widening absolute achievement gaps are a significant concern for 15-year-olds—and for younger students. In other words, the issue doesn’t just hinge on the pandemic.

To show how achievement gaps have grown over the last decade, I have graphed PISA scores for each percentile group relative to that percentile group’s 2012 scores (see charts below). I do this by subtracting each percentile group’s 2012 score from its score in a given year. The resulting graphs are similar to my graphs in the last post (they use the same data) but center scores on 2012 instead of 2018. This makes it easy to see how PISA scores have changed not just since the pandemic, but over the last decade.

Looking at the graph below, we can see that the achievement gap in science scores was dramatically widening even before the pandemic. Between 2012 and 2022, the score gap between the 75th and 25th percentiles grew by 25 points—which translates to more than a year’s worth of progress in learning for the average student—and the score gap between the 90th and 10th percentiles grew by 40 points—which is roughly equivalent to two years of learning for the average student. Growth in these gaps for the Organization for Economic Co-operation and Development (OECD) countries’ average were half as big as those in the US over the same period.

(Site: https://public.flourish.studio/visualisation/16041238/)

The “bright spot” in US pandemic PISA scores was in reading, where achievement was flat and achievement gap growth was modest. However, scores over the last 10 years paint a dimmer picture of both achievement and growth in the achievement gap. Between 2012 and 2022, the gap between the 75th and 25th percentiles in reading grew 30 points—roughly a year and a half of average progress—and the gap between the 90th and 10th percentile gap grew by a whopping 56 points—over half a standard deviation. The comparative growth in achievement gaps for the OECD average were half as large.

(Site: https://public.flourish.studio/visualisation/16041150/)

Both during the pandemic and since 2012, PISA math scores have not followed the same patterns as PISA reading and science scores. Perhaps most significantly, math scores fell significantly for all percentile groups both over the pandemic years and on net over the past decade. This should be a major point of concern: since 2012, 50th, 75th, and 90th percentile US PISA scores went up for students in reading and science, but math scores went down. No percentile group is doing well in math.

Also of interest, although between 2012 and 2022 the US achievement gap didn’t grow nearly as much in math as it did in science and reading, the growth in the US math achievement gap over the past decade was still greater than it was in many other nations. For instance, the US gap between the 25th and 75th percentiles in math grew ten points over the past decade. This growth in the math achievement gap is modest compared to the growth in the science and reading achievement gaps (24 and 30 points, respectively), but pronounced compared to the 1-point growth in the math achievement gap for OECD countries during the same period.

(Site: https://public.flourish.studio/visualisation/16041310/)

It is hard to identify the cause of these changes in test scores or test score gaps, but the absolute achievement gap in these PISA scores reflect those seen in recent National Assessment of Educational Progress (NAEP) scores for younger students. Since 2012 test scores have fallen or been stagnant, but the absolute achievement gaps between the highest and lowest performers have grown, in some cases growing dramatically, and PISA suggest grew more in the US than many other countries. These trends in achievement gaps predate the pandemic, which should be cause for alarm. Test results have repeatedly shown that the floor is falling for the lowest-achieving US students, and there is little reason to expect that it will magically stop falling now that the pandemic is over.