I was wrong

Last week, teaching my statistics class, I gave an example of regression using actual data. I had hypothesized that given the greater discrimination and poverty on the reservation 40 or 50 years ago, there would be a negative relationship between age and educational attainment within the adult population. That is, people over 60 would be less likely to have obtained a high school diploma, associate’s degree or bachelor’s degree than those under 40.

To get to the point – I was wrong. I deleted the outliers on both age and education. I was still wrong. The students were waiting for me to do the next analysis to prove that I was right but there was no next analysis. It seems that, at least for the population I had sampled, I was simply wrong.

I try to do this at least two or three times during the semester with every course I teach.  The first time, the students are always surprised. We present research in textbooks in such a neat, linear fashion – you have a hypothesis, you collect data, analyze data, reject the null hypothesis, write up your conclusions and either publish your results or pick up a fat speaker fee to talk about your brilliant study.

It doesn’t always work that way. One advantage of being a small company with multiple clients is that we aren’t so tied to anyone that we feel that we MUST produce certain findings. I’ve worked for pharmaceutical companies, educational institutions – you name it. All of my clients are really smart, competent people. If I didn’t believe that, I wouldn’t be working with them – you only get one life and why waste it with people who aren’t rewarding to work with? No matter how smart, educated, experienced and hard-working you are, sometimes the results don’t come out the way you expected.

The profoundest lesson my advisor, Dr. Richard Eyman, taught me was,

“The data show what they show.”

Sometimes they show that you are wrong. Not only do you have to accept that, you even have to expect it.