The rumors of our sucking at math have been greatly exaggerated

Question authority.

Whenever I hear authoritative statements made that don’t fit with the world I see around me, I try to follow up.

How many times have we been told that the U.S. is just terrible in math, we are falling behind educationally, China and India are eating our lunch – deservedly so, because our students are all fat, lazy pie-eating, Wii-playing slackers, we’re far behind where we used to be and therefore our teachers need to be fired, unions disbanded and everyone who came to this country after 1918 sent back to wherever the hell they came from?

And yet, I wonder how that can be when many countries from China and India to Ethiopa and Uruguay have a much higher rate of poverty than the U.S.

I asked a very pompous businessman who told me how poorly we are doing in international comparisons if the 80% of the world’s population living on less than $10 a day, including the one-quarter of the children in the developing world who are underweight, the 1.6 billion people who don’t have electricity, THEY’RE all doing better than our kids in Santa Monica. He said,

“Yes! They’re studying by candlelight without enough to eat, living in a hut and they’re still doing better than our kids at math!”

I thought I would check. So, I looked at the Trends in International Mathematics and Science Study report and here is what I found:

The 2007 study only included 36 countries for Grade 4 and 48 for Grade 8. Neither India nor China were included. Only one country from central America was in the study, one from South America and almost no African countries outside of northern Africa / middle east nations.

In short, it was a non-representative sample that was definitely skewed toward the more affluent nations, I presume because those are the ones that chose to participate.

Among this group of predominantly fairly well off Asian and European countries, the U.S. average was above the median mathematics score both in fourth grade and eighth grade (which were the two grades tested). The median for both grade levels was 500. The average for U.S. fourth graders was 529 and for U.S. eighth graders, 508.

In eighth grade, only Korea, Japan, Singapore, Hongkong and Taiwan score significantly higher than the U.S. No disrespect to the people of Hongkong, but it is, after all, a city. So, four actual countries scored higher than the United States in  a sample of relatively affluent countries. Four. Rather than being in the bottom of the national rankings, we are in about the top 10% of a selective sample. The reports of our sucking are seeming a bit questionable.

At fourth grade, Korea was not included in the study. Again, Singapore, Hongkong and Taiwan scored higher than the U.S., as did England and Russia. There were two eastern European countries that also scored higher but they did not meet the TIMSS criteria for national target population. In short, if you have sampled from, say only 50% of your schools, then the results are called into question because that may be the higher performing half of the population. Even if we include those two, the U.S. came in tied for ninth out of 36 – so, in the top quarter of a group of countries that are richer and more industrialized by far than the world population.

But our students DROPPED in test scores from fourth to eighth grade, didn’t they? How about THAT? They did. So did  Hongkong, Singapore, England and Russia (four countries that were ahead of us in the fourth grade).  In fact, in eighth grade we are tied for fifth out of 48, in international rankings, we had moved UP.

In the twelve years the study has been conducted, from 1995 t0 2007, the U.S. average mathematics score has increased 11 points for fourth graders and 16 points for eighth grade students. In 1995, U.S. eighth graders’ average mathematics score was 8 points below the median while in 2007 it was 8 points ABOVE the median.

So, to recap, even among a sample of comparatively well-off countries, the U.S. comes out above average on all overall measures, and, in fact, above 3/4 or more of the other countries. The ‘countries’ that consistently score above the U.S. are Taiwan (population 23 million), Singapore (population 5 million) , Hongkong, which is actually a city (population 7 million) and Japan (127 million). South Korea (48 million) only had data for 8th graders. By comparison, the U.S. has a population of 307 million. So, all of those countries put together add up to about two-thirds the size of the U.S. and about 3% of the world population of 6 billion and rapidly increasing.

This is NOT to argue against looking at what these countries are doing to try to see if it could work in the United States, or to see what other explanation there could be for the differences. Yes, there may be other countries not included that score higher than us. Yes, the U.S. does need to try to do better, especially in Geometry, which was the one area of the test where we were below average.

However, this picture is a far cry from we’re near dead last and racing toward the bottom. Next time someone tells you how terrible mathematics education in the U.S. is, point him or her toward the TIMSS report and suggest looking at the data.

After all, our eighth-graders scored an average of 531, compared to an international mean of 500, in the area of data and probability.

NOTES

As did the TIMSS researchers, I considered countries tied if the difference in scores were not statistically significant. You can download the TIMSS report here.

By the way, statistics on poverty, etc. came from the Poverty Facts and Statistics page.