What I’m Learning at NCES

I’m currently at a seminar in Washington, D.C.,  sponsored by the National Center for Education Statistics. I’ve seen the notices for these a lot of times over the years, and always thought, “Well, that looks interesting” but never applied to go, primarily because although your expenses are paid, there is no stipend, so that is a week of consulting hours I’m giving up.

This latest one, though, was on the National Indian Education Statistics and it was right up my alley. My former business partner and co-founder of Spirit Lake Consulting, Inc. (which I left to form The Julia Group) was also going.

So, what have I learned my first day? Well, that some of our assumptions were wrong. We had assumed, for example, that American Indian/ Alaska Native children were probably about equal to the total population average up through fourth grade or so and then the mathematics achievement gap would show up as students began attempting more complex problems.

At 9 months, according to the Early Childhood Longitudinal Study, Birth Cohort (which we learned about today), the two groups are about equal:

However, the gap between American Indian/ Alaska Native children and the general population emerges far sooner than we had expected .. by age two years, even in early precursors to mathematics like matching and counting we could see a difference:

We had thought that perhaps there would be some areas of mathematics where American Indian / Alaska Native children would be relatively closer to the national average. Well, I thought that, Erich didn’t and he turned out to be correct.

As you can see here, the deficits for AI/ AN children in mathematics were both persistent and pervasive, extending across all domains measured and through the eighth grade, which was as far as we got today (hey, we’ve only been here a day, what did you expect?)

Actually, there was more that we learned today, a lot more, particularly about factors potentially explaining these achievement gaps. We came up with a whole list of positive and negative correlates and used the NCES web tools to test univariate relationships, but, of course, we are interested in much more than that.

Erich and I met for a couple of hours after the last session, drafting out a research design which we intend to apply to the NIES database that we expect to get our hands on tomorrow afternoon.

BUT .. it’s three hours later in D.C. and I have to be up early enough to drink at least three cups of coffee to stay awake in the 8:30 a.m session in the morning. So, I’ll talk about that tomorrow.

I should add, though, that not only am I learning a lot but they are also very generous with the swag. Being a statistician, of course, my idea of swag is not the designer cosmetics they put into the Gucci bag, but rather things like a book, Status and Trends in American Indian/ Alaska Native statistics, and software they are going to give us tomorrow to analyze the NAEP / NIES.

The latter is very exciting because the sample is based on Item Response Theory and I thought I was going to have to use SAS or SPSS for multiple imputation, neither of which are cheap. (SPSS is far less expensive but multiple imputation is one of those add-on modules. As a faculty member, I qualify for the SPSS faculty pack – an awesome deal, and I have access to SAS on the university computers – but not all my colleagues are in the same position).

NCES made this software sound like the greatest thing since sliced bread, so, I guess we’ll see if it stands up.

Speaking of Item Response Theory, that’s another good thing about this seminar. I grew jaded early in my career about the latest statistic being the answer to everything. It was factor analysis, then path analysis, then structural equation model, Rasch, IRT, MI and who knows what. Very often, an unneeded level of complexity was added just to show how smart and cutting edge the researcher was when a simpler design would have worked very nearly as well and been a whole lot easier to explain to the school board members who have to foot the bill, make policy and give us permission (or not) to come back and collect data next year.

All of that being said, the current study was the perfect situation to which Item Response Theory is the answer and Dr. Sikali did a brilliant job explaining exactly why.

I know we’re all busy teaching summer school, writing grants, publishing articles, conducting research and meeting with clients but if you see one of these NCES seminars that fits your research interests, I really recommend you go. I finally came because not one, but THREE of my friends who had attended previous seminars told me it was really worthwhile.

They were right. Your mileage may vary. But I doubt it.