Scoring method
Moderators: Stardust@home Team, DustMods
Scoring method
The current scoring method is blatently open for major abuse. If you see ondeck_movie_type=testing in the link you know it is a test video. You can spend your time looking for a track. If it says real you are safe to just click no track.
Easily gaining a lot of points. I'm sure you could get even more creative with how the links work to give yourself testing movies on purpose. I suggest a system that doesn't let the user directly know if its a test video or real data.
Easily gaining a lot of points. I'm sure you could get even more creative with how the links work to give yourself testing movies on purpose. I suggest a system that doesn't let the user directly know if its a test video or real data.
true,
if you look at some participants rate of scoring you get suspicious..
imho some people already have made scripts that contain the answer to the testing movies and don't have to wait for the images to load before submitting the answer.
Not that I can think of any reason that people want to cheat such a project but weird people do exist.
Maybe some kind of trapping system to take care of automated responders.
Or some random real movies that contain a message: CLICK ON THIS MOVIE
if you look at some participants rate of scoring you get suspicious..
imho some people already have made scripts that contain the answer to the testing movies and don't have to wait for the images to load before submitting the answer.
Not that I can think of any reason that people want to cheat such a project but weird people do exist.
Maybe some kind of trapping system to take care of automated responders.
Or some random real movies that contain a message: CLICK ON THIS MOVIE
Re: Scoring method
Sasari I hope you are wrong about abusing the system, but I think you're right, the cheating has begun. It's very sad indeed.
Well until I read this post I wasnt aware of how to abuse and create higher scores not that I would want to anyway. But I am sitting at a genuine No6 and that would I am sure have been higher if the site had not `shut down`. At one point I got up to No 2 before access was denied....Damn!if you look at some participants rate of scoring you get suspicious..
Anyhow I am just just enjoying the fun of it, and the children are enjoying taking the P... at me, mind you the fact I have now declared myself a scientist its no wonder
regards
Juco
-
- Posts: 20
- Joined: Wed May 17, 2006 8:11 pm
- Location: Boise, ID, USA, North America, Earth, Solar System, Milky Way
Cheating is pointless. What do you have to gain? "Oh look, I'm #1!" but you know what that means to those of us who actually care about the project? It means that our results are not reliable and it makes more work all around.
Do you want to be #1, or do you want to make the people working to make this project heppen hate their lives and contract out the project? If you must cheat, you're not having fun.
Do you want to be #1, or do you want to make the people working to make this project heppen hate their lives and contract out the project? If you must cheat, you're not having fun.
It can be quite frustrating to people who do the real job at the best possible rate they can and still see people going twice or triple as fast. This makes people worry about quality of work. For example, someone with 800 points, and 1 cali movie in 4 means he has waited for 3200 movies to load, scrolled to each of their focus levels, analyzed them thoroughly and made an educated decision about wether there is a track, without making any errors in calibration movies..
If we take like 20 seconds for a full movie analysis, which is basically the amount you need to get everything loaded and analyzed in your brain.
That means 17 hours straight non-stop working... No human being can do that without error.
I wouldn't mind for the ranking system to be reviewed and after change of scoring method a reset would be cool.
Just my 2 cents.
If we take like 20 seconds for a full movie analysis, which is basically the amount you need to get everything loaded and analyzed in your brain.
That means 17 hours straight non-stop working... No human being can do that without error.
I wouldn't mind for the ranking system to be reviewed and after change of scoring method a reset would be cool.
Just my 2 cents.
I think the "score" formula needs to be changes. Have all five key values (correct cali, incorrect cali, total real, specificity, sensitivity) come into play.
Something along the lines of:
a=(spec+sens/2)
b=(correct-incorrect)*a
score=b+(a*real)
This way, regardless of ratio, the amount of real movies you have done correctly (correctness tested using the calibration movies) comes into play.
Something along the lines of:
a=(spec+sens/2)
b=(correct-incorrect)*a
score=b+(a*real)
This way, regardless of ratio, the amount of real movies you have done correctly (correctness tested using the calibration movies) comes into play.
Join us on the Unofficial IRC channel!
irc://irc.freenode.net/stardust@home
irc://irc.freenode.net/stardust@home
-
- Posts: 153
- Joined: Wed Aug 02, 2006 8:17 pm
And if someone is twice as fast as that fabricated value, your 17 hours becomes 8.5. And if they are three times as fast ast the fabricated value, then it's under 6 hours. Once again, maybe some people ARE just better at it than others.Whity wrote:If we take like 20 seconds for a full movie analysis, which is basically the amount you need to get everything loaded and analyzed in your brain.
Sounds like there are some issues (I for one haven't been paying attention to URLs in the address bars, I'm watching the movies) but I hope that the system doesn't get tweaked to take away from more efficient participants' progress - just because of some whiners.
Je ne peux pas regarder la Mer sans me demander qui vit au-delà de cela.
No need to get defensive. We all need to remember that people work faster than others and can work fewer or more hours than we do ourselves. While new leaders may be shocking now, let the project evolve for a week or two and I doubt we'll see runaway leads any more.Orion_0169 wrote:Sounds like there are some issues (I for one haven't been paying attention to URLs in the address bars, I'm watching the movies) but I hope that the system doesn't get tweaked to take away from more efficient participants' progress - just because of some whiners.
Join us on the Unofficial IRC channel!
irc://irc.freenode.net/stardust@home
irc://irc.freenode.net/stardust@home
-
- Posts: 153
- Joined: Wed Aug 02, 2006 8:17 pm
lol Not defensive. My integrity stands.
Actually, I've been paying more attention to the time involved. And the 20 seconds average gets absolutely blown away by many of the obvious callibration movies. Many of those I have done in a second or less. Often, if you start the focus at the bottom, you'll see the track for the callibration pop up right away. A quick check in the focus and you're done in no time. Doing a one or two second movie one in four times... well, it explains how people can be doing well.
Like I said, praise the people who are actually trying to help.
And like they say, if you're not having fun...
Actually, I've been paying more attention to the time involved. And the 20 seconds average gets absolutely blown away by many of the obvious callibration movies. Many of those I have done in a second or less. Often, if you start the focus at the bottom, you'll see the track for the callibration pop up right away. A quick check in the focus and you're done in no time. Doing a one or two second movie one in four times... well, it explains how people can be doing well.
Like I said, praise the people who are actually trying to help.
And like they say, if you're not having fun...
Je ne peux pas regarder la Mer sans me demander qui vit au-delà de cela.
-
- Posts: 3
- Joined: Fri Aug 04, 2006 2:48 am
- Location: Denmark
Sit back and relax
It seems to me that a LOT of people here are concentrating about "possible cheaters" or other thoughts regarding the scoring system.
To my knowledge the scores is of NO importance to the project.
What counts is how many people identifies a real particle track and submits this finding to the scientists for further investigation.
A particle caught in the Aerogel
To my knowledge the scores is of NO importance to the project.
What counts is how many people identifies a real particle track and submits this finding to the scientists for further investigation.
A particle caught in the Aerogel
"Even if you are on the right track, you will be run over if you don't move"
-
- Posts: 38
- Joined: Thu Aug 03, 2006 9:19 am
- Location: St. Petersburg Florida
- Contact:
Re: Scoring method
It looks like the scoring system has been turned off. My rank this morning was 597 and it's still the same after going through approx 200 movies. I would think it would have moved at least a little bit.Sasari wrote:The current scoring method is blatently open for major abuse. If you see ondeck_movie_type=testing in the link you know it is a test video. You can spend your time looking for a track. If it says real you are safe to just click no track.
Easily gaining a lot of points. I'm sure you could get even more creative with how the links work to give yourself testing movies on purpose. I suggest a system that doesn't let the user directly know if its a test video or real data.
Last edited by templar781 on Fri Aug 04, 2006 7:39 am, edited 1 time in total.
Winning isn't everything, but wanting to win is. - Vince Lombardi