### Re: Answers to Qs about: VM, Movies, Scores… UPDATED 6:56pm

Posted:

**Wed Jul 31, 2013 12:59 pm****Thanks for staying with us Evelyn! BTW, I ran this little hiccup regarding the new Skill Score formula by Dr. Westphal and he ended up doing a full analysis to convince himself of it. His report follows. Dan**

*I went back to rerun the Monte Carlo simulation to confirm that the skill score can go down even if one clicks correctly on a low-value movie. Here's what I did. For each trial, I drew randomly 101 power movies. For each movie, I assumed that the response was 100% correct if the powerscore was less than 70, and 100% wrong if the powerscore was more than 70. I then computed the score from power movies 1 through 100, and also for power movies 2 through 101. I did this for 10,000 trials. I found that in the 1366 cases is which the skillscore went down, 396 of those cases were for correct responses.*

I redid it for a threshold of 40 instead of 70 and the numbers are 3643 and 867, respectively.

I then just assumed that 50% of the movies were randomly clicked correctly, regardless of powerscore. Then the numbers are 4376 and 920, respectively.

To confirm that this has to do with the fact that we are now calculating from the previous 100 movies, rather than back to the beginning, I redid the calculation comparing the score from movies 1 through 100, to that from movies 1 through 101. In this case, there are no cases of the score going down. So as expected, this fluctuation has to do with the fact that we are computing on a moving window 100 movies wide. As John Smaje pointed out originally, the skill score cannot go down, regardless of the movie value, if you just add onto the whole sequence of responses. But if you have a window, responses are dropping out as you add more movies.

I redid it for a threshold of 40 instead of 70 and the numbers are 3643 and 867, respectively.

I then just assumed that 50% of the movies were randomly clicked correctly, regardless of powerscore. Then the numbers are 4376 and 920, respectively.

To confirm that this has to do with the fact that we are now calculating from the previous 100 movies, rather than back to the beginning, I redid the calculation comparing the score from movies 1 through 100, to that from movies 1 through 101. In this case, there are no cases of the score going down. So as expected, this fluctuation has to do with the fact that we are computing on a moving window 100 movies wide. As John Smaje pointed out originally, the skill score cannot go down, regardless of the movie value, if you just add onto the whole sequence of responses. But if you have a window, responses are dropping out as you add more movies.