Update 16Aug06: "cheaters" vs science!

Stardust@home project news.

Moderator: Stardust@home Team

Post Reply
albutterworth
Stardust@home Team
Stardust@home Team
Posts: 69
Joined: Wed May 17, 2006 12:24 pm
Location: UC Berkeley, CA
Contact:

Update 16Aug06: "cheaters" vs science!

Post by albutterworth »

Several colleagues pointed out on the forum that people could "cheat" by advancing rapidly through the real movies to the calibration movies. Although the "cheaters" only cheat themselves out of an opportunity to find a real interstellar dust particle, we did want to find them so that we could ignore their data and remove their artificially high scores and rankings.

To find cheaters, we compared the median evaluation time spent on real movies to the median evaluation time spent on calibration movies for the top 200 ranked searchers. Cheaters would have a very much smaller evaluation time for real movies (because they're just flipping through them quickly) than for calibration movies. This makes them easy for us to spot.

To demonstrate, we show two sets of histograms below -- one is for a cheater, the other for a true collaborator.
In each pair of histograms, the top figure shows the distribution of evaluation times for calibration movies, and the bottom shows the distribution of evaluation times for real data.

A real collaborator will treat calibration and real movies almost the same way -- if anything, the times will be shorter on average for the calibration data, because some tracks are obvious and can be discovered quickly. A cheater, on the other hand, will have much shorter evaluation times for the real data. Can you tell who is who?

As we had hoped, we found that out of the top 200 volunteers in the rankings, only eight were cheating. Everyone else was simply incredibly hard-working and talented!
For the cheaters, we are ignoring their data and have zeroed their scores. If they would like to do some real scanning, we invite them back, and suggest that they re-register under a different username.

1
Image

2
Image

Stardust@home team
Stardust@home
Useful Links: | Aerogel Scanning Progress | Forum Rules |
Post Reply