Statistics is statistics

While I love teaching and am looking forward to be working in a completely new environment – teaching an online course to masters students – I was initially concerned that teaching a course on biostatistics in public health might draw too much time away to my work for The Julia Group. I really should have known better. Statistics is statistics, and as I’m reading the textbook preparing for my lectures in November, it is actually very relevant to the research design we are implementing now.

Randomized controlled trials are not an option for us. We work either with schools, testing our 7 Generation Games educational software, or with social service programs that are mandated to provide an intervention for those who qualify. On the other hand, the controlled part is very relevant to our work. For example,

“… investigators must be sure that participants are taking the assigned drug as planned and not taking other medications that might interfere with the study medications…”

You might wonder how this relates to educational research …
When we collect data on our games, we have a timestamp recorded with each answer. This was extremely useful when we were looking at the different classrooms that used our game, because some had better outcomes than others. We were able to estimate, from the timestamp on the first problem answered by a student in the class to the last, about how long students in that classroom had the opportunity to play the game. Not all were ‘taking the program as planned’. There might have been an early dismissal on several days because of hazardous weather (our initial testing occurred in North Dakota in the fall and winter), so students in that class only played the game 2/3 of the time students in the other classes did, because their math class was the last period of the day, and others had math before lunch.

What was not mentioned in the text, but is equally important, and I’ll address in lecture, is whether it is a drug study or an educational intervention, you also need to be sure you assess your DEPENDENT variable correctly.

Recently, I was talking to someone from a school that used a mathematics program, a competitor to us, that had demonstrated very good results. The test scores at the school had risen dramatically, and yet, to my surprise, the math department was very interested in having us come install our program. I found out that in the year before they used the program, students were given a set amount of time for their standardized tests. The year they began using the program, students were allowed unlimited time to complete the same tests. They were even allowed to come back the following day and finish up where they had left off. The math teachers working with these students were very aware that the students’ skills needed major help, regardless of what the tests might say.

While this has to do with education and not health care, the same applies. If your dependent variable is blood pressure, cholesterol or blood sugar, you need to make sure that it was measured accurately AND UNDER THE SAME CONDITIONS, for both groups and at pretest and posttest.

I’m amused when people make comments like, “I’ve forgotten more statistics than you’ll ever know.”

Personally, I try NOT to forget, which is why teaching a masters level class every now and then is a good reminder of the basic principles. Just so you know that I practice what I preach, I’m on a flight to North Dakota right now where I will be meeting with principals and teachers of our intervention schools to discuss the importance of collecting data from all of the students at the same time in the same way.