Scoring tests with SAS: What a difference array makes

Day eight of the 20-day blogging challenge was to write about a professional read – a book, article or blog post that has had an impact on me. To be truthful, I would have to say that the SAS documentation has had a profound impact on me. SAS documentation is extremely well-written (to be fair, so is SPSS) in contrast to most operating system documentation which is written as if feces-flinging monkeys were somehow given words instead, which they flung onto a page which then became a manual. But I digress – more than usual. It’s not reasonable to suggest to someone reading the entire SAS documentation which is several thousand pages by now. Instead, I’d recommend Jennifer Waller’s paper on arrays and do-loops. This isn’t the paper where I first learned about arrays – that was before pdf files and I have met Jennifer Waller and she was probably barely in elementary school at the time. It’s a good paper though and if you are interested in arrays, you should check it out.

Here is what I did today, why and how. I wanted to score a dataset that had hundreds of student records. I had automatically received the raw score for each student, percent correct and what answer they gave for each multiple choice question. I wanted more than that. I wanted to know for each question whether or not they got it correct so that I could do some item analyses, test reliability and create subtests. This is a reasonable thing for a teacher to want to know – did my students do worse on the regression questions, say, than the ones on probability, or vice-versa?  Do the data back up that the topics I think are the hardest are the ones that my students really score worst on?  Of course, test reliability is something that would be useful to know and most teachers just assume but don’t actually assess. So, that’s what I did and why. Here is how.
filename sample “my-directory/data2013.csv”;
libname mydata “mydirectory” ;
data mydata.data2013 ;
infile sample firstobs = 2 dsd missover ;
input group_type $ idnum $ raw pct_correct qa qb qc q1- q70 ;

** These statements read in the raw data, which was an Excel file I had saved as csv file ;
** The first line was the header and I forgot to delete it so I used FIRSTOBS = 2 ;
*** That way, I started reading at the actual data. ;
*** The dsd specifies comma-delimited data. dlm=”,” would have worked equally well ;
*** Missover instructs it to leave any data missing if there are no values, rather than skipping to the next line ;

Data scored ;
set mydata.data2013 ;
array ans{70} q1- q70 ;
array correct{70} c1 – c70 ;
array scored{70} sc1 – sc70 ;

*** Here I created three arrays. One is the actual responses ;
*** The second array is the correct answer for each item ;
*** The third array is where I will put the scored right or wrong answers ;

if _N_ = 1 then do i = 1 to 70 ;
correct{i} = ans{i} ;
end ;

*** If it is the first record (the answer key) then c1 – c70 will be set to whatever the value for the correct answer is ;

else do i = 1 to 70 ;
if ans{i} = correct{i} then scored{i} = 1 ;
else scored{i} = 0 ;
end ;

**** If it is NOT the first record, then if the answer = the correct answer from the key, it is 1 , otherwise 0 ;

Retain c1 – c70 ;

**** We want to retain the correct answers that were in the key for all of the records in the data set ;
**** Since we never put a new value in c1 – c70, they will stay the correct answers ;

raw_c = sum(of sc1 – sc70) ;
*** This sums the raw score ;

pct_c = raw_c/70 ;
*** This gives a percentage score :

proc means data=scored ;
var sc1-sc10 c1 -c10 ;

*** This is just a spot check. Does the mean for the scored items fall between 0 and 1? Is the minimum 0 and the maximum 1 ;
*** The correct answers should have a standard deviation of 0 because every record should be the same ;
*** Also, the mean should either be 1, 2, 3, 4 or 5 ;

proc corr data = scored ;
var raw_c pct_c raw pct_correct ;

*** Spot check 2. The raw score I calculated, the percent score I calculated ;
*** The original raw score and percent score, all should correlate 1.0 ;

data mydata.scored ;
set scored ;
if idnum ne “KEY” ;
drop c1-c70 q1-q70 ;

*** Here is where I save the data set I am going to analyze. I drop the answer key as a record. I also drop the 70 correct answer fields and the original answers, just keeping the scored items ;
proc corr alpha nocorr data= mydata.scored ;
var sc1 – sc70 ;

*** Here is where I begin my analyses, starting with the Cronbach alpha value for internal consistency reliability ;

I want to point something out here, which is where I think the professional statisticians are maybe distinguished from others. It’s second nature to check and verify. Even though this program should work perfectly – and it did – I threw in reality checks at a couple of different points. Maybe I spelled a variable name wrong, maybe there was a problem with data entry.

One thing I did NOT do was write over that original data. Should I decide I need to look at what the actual answers were, say, I wanted to see if students were selecting chi-square instead of t-test (my hypothetical correct answer), that would alert me to some confusion.

Incidentally, for those who think that all of the time they save grading is taken up by entering individual scores, I would recommend having your students take tests on the computer if you possibly can. I was at a school today where we had a group of fourth graders taking two math tests using Google chrome to access the test and type in answers. They had very little difficulty with it. I wrote the code for one of those tests, but the other was created using survey monkey and it was super easy.

I’d love to include pictures or video of the kids in the computer lab but the school told me it was not allowed )-: