Skill Score

Discuss your experiences with and ideas about Stardust@home here.

Moderators: Stardust@home Team, DustMods

tudorbug
Posts: 45
Joined: Sat Aug 29, 2009 7:48 pm

Re: Skill Score

Post by tudorbug » Sun Nov 06, 2011 5:48 am

In the meantime, 20 dusters have voted and a tie has resulted. Voting has been exteneded until November 13 to allow more votes in the hope of breaking the tie.

David

laserphil
Posts: 81
Joined: Sat Dec 02, 2006 9:28 pm
Location: Sydney Australia

Re: Skill Score

Post by laserphil » Wed Nov 16, 2011 4:06 am

Hi John,

As a matter of interest I thought I would conduct a statistical look at the PM's as received in the last few weeks just to see if what I thought was correct.

I believe that there are about 5000 different PM's so my sample is reasonably small but still significant and assuming that I am receiving a reasonably random sample the stats should be valid

Of the last 133 PM's I received:
Value Range . . . . . . . . Number of PM's
2 - 10 . . . . . . . . . . . . . 47
11 - 20. . . . . . . . . . . . .50
21 - 30. . . . . . . . . . . . .12
31 - 40. . . . . . . . . . . . . 8
41 - 50. . . . . . . . . . . . . 9
51 - 60. . . . . . . . . . . . . 3
61 - 70. . . . . . . . . . . . . 3
>70. . . . . . . . . . . . . . . .1

The average value of the PM's received is around 16!

I am now keeping a running tally of the value of each PM received and the larger the sample becomes the more it is skewed to the lower value end. I wonder what it will be like when I get say another 900 samples :lol:

jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: Skill Score

Post by jsmaje » Wed Nov 16, 2011 1:37 pm

Thanks for those interesting stats Phil.
Meanwhile, in response to a recent PM about this issue, Andrew Westphal replied: "I wouldn't count on any particular distribution, because it may change as things evolve".
John

PS: please note that my sim wasn’t capable of incorporating proposal F2 (still in contention with 6 votes) since it amounts to requiring the movie delivery strategy at the team’s end being personalised for each duster’s changing skill level, while all the other formulae assume (as at present) uniformly random delivery to every duster, independent of the issue whether the movie-difficulty range itself be random or biased in any way.

PPS: sorry if the one or two of you still interested in the Top 100 graphs and Skill Score Sim were unable to access them over the last week or so. Should be back now, but let me know if any problems.

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Mon Nov 28, 2011 3:09 pm

Hi Everyone,

Sorry to have been away for so long (really!) but I have been pulled by other pressing projects, proposals, etc. by the team I actually do most of my work with (90-95%) - including some projects where I essentially function as PI (and thus need to report to NASA on, etc.). Seems things have calmed down now though and I will do my best to get caught up.

Thanks very much for submitting and voting on the Skill Score proposals. I know Dr. Westphal has very seriously considered your ideas and concerns on this issue. Give me a week and I'll fill you in with what gets decided!

Dan

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Mon Nov 28, 2011 3:12 pm

jsmaje wrote:Agreed, laserphil & Elaine!
Given that the team officially designated this forum topic to Skill Score issues, at least a link to the News/Blog item and voting opportunity should have been posted here. I find it all too easy to skip the Home page, given the rarity of anything new of interest, instead going straight to the forum for the occasional gem.
It's essentially my fault. I was too involved in my other work and I sincerely apologize.

Dan

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Mon Nov 28, 2011 3:31 pm

jsmaje wrote:I also confess that each time I’ve naively delved into this project’s stats and other issues such as this over the last 5+ years, the more complicated they’ve turned out to be, and my sympathy must ultimately be with the team, whose priority has of course to be the basic interstellar particle science - of which It would be good to hear more!
It would appear you've been heard! Please see the most recent Blog post here.

Dan

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score - A Final Run Off!

Post by DanZ » Fri Dec 02, 2011 5:06 pm

Hi Everyone,

Thanks for your patience on the Skill Score, but there is still no clear winner. Therefore, if you HAVE NOT voted yet, or you DID NOT VOTE FOR either proposal B or G (the two in contention), please vote again for either proposal B or G (B or G ONLY please) at http://www.doodle.com/mzy32ewtxacs59s9

The deadline for voting in this FINAL run off is this Friday, Dec 9th at 3:00pm (15:00) Pacific Time (23:00 GMT). For your reference, the proposals can be reviewed at http://stardustathome.ssl.berkeley.edu/blog/?p=361 or http://stardustathome.ssl.berkeley.edu/ ... sFinal.pdf

Thank you so much!

Dan
On behalf of the entire Stardust@home team

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Fri Dec 16, 2011 12:48 pm

I think a decision will be made very soon so stay tuned!

Dan

SmithES
Posts: 186
Joined: Sun Apr 17, 2011 5:25 am

Re: Skill Score

Post by SmithES » Mon Jan 02, 2012 7:58 am

Hi, Dan Z, Hope you and crew have a Happy New Year coming in 2012. Haven't heard anything about update of Skill Score since Dec. 16. Any news due soon? I'm ready to get back to work---as long as I don't get penalized for doing it. Thanks, ERSTRS - (Evelyn)

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Tue Jan 03, 2012 6:55 pm

Happy New Year to you too! Yes indeed, we need to get back on this skill score issue. We had a team discussion just before the winter break to discuss the winning proposal. Now we just need to determine how we should implement it, including the start of a new phase (which can drag the process out a little as you can imagine). But IMHO, Phase 4 will probably be coming to an end fairly soon, so now's the time if you feel like you want to raise your final score.

Anyway, I'll see if I can get an update for everyone by this Friday.

Stay tuned!

Dan

SmithES
Posts: 186
Joined: Sun Apr 17, 2011 5:25 am

Re: Skill Score

Post by SmithES » Wed Jan 04, 2012 1:10 pm

Thanks, Dan Z. I plan to join the next phase of dusting. But, I have to comment on your statement, so now's the time if you feel like you want to raise your final score. I quit dusting because my score kept dropping each day! The early batches assigned to me had high-scoring tracks in them and I advanced by finding lots of 50-70 point tracks. Then, my batches began to have only small tracks in them---no big tracks---needed if I was going to raise my final score! I began to drop---even though I was faithfully dusting. So, I quit, rather than fall lower than #37 where I still am right now. (Interestingly, Even though I've not dusted since August 2011, my score has remained the same.) I'm afraid to try to raise my score----afraid my batches won't have any high-point tracks, and I'll drop further down. Thanks for your hard work. Looking forward to next Phase---and truly do wish you and crew a very different and happier New Year for 2012! Thanks for being patient with us. ERSTRS-Evelyn (Joined Aug. 2006)

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Re: Skill Score

Post by DanZ » Wed Jan 04, 2012 1:14 pm

I understand Evelyn - I guess I was thinking just Power Score.

But yes, hopefully 2012 will be a bit more problem free. At least on the science side, things are going great and you should stay tuned for announcements of new discoveries!

All the best,

Dan

jsmaje
Posts: 616
Joined: Tue Aug 15, 2006 8:39 am
Location: Manchester UK

Re: Skill Score

Post by jsmaje » Mon Jan 16, 2012 12:59 pm

DanZ wrote:I think a decision will be made very soon so stay tuned! Dan

It's now a month since the above (dated Dec 16 11).
Any news yet of what may be going on behind the scenes regarding the scoring issue, as well as "announcements of new discoveries" (dated Jan 4 12) ?

DanZ
Site Admin
Posts: 776
Joined: Fri Feb 27, 2009 3:44 pm
Location: Berkeley, CA
Contact:

Skill Score - We Have A Winner!

Post by DanZ » Tue Jan 17, 2012 11:49 am

Dear Dusters,

It is my pleasure to announce to you that we now have a winner in the Skill Score formula run-off. By the narrowest of margins, John Smaje's Proposal G beat out Proposal B by Mike Capraro. Congratulations John, and a huge THANK YOU from the Stardust team not only to you, but to all those who took the time to contribute (and vote too)!

In the next few weeks, we will be cleaning up the system so that the new formula works properly (in that it assumes that each PM was assigned its score with the utmost attention to detail and accuracy - which frankly, is not entirely true). But not only will we be launching the new formula, we will also be rolling out Phase V, which will include, FOR THE FIRST TIME, a separate competition for those who want to try their hand at an automated track detection system. More details will come on that to be sure, but for now, what I can tell you is that we'll basically be making our raw scan data available to the public with the hopes that someone can find a way to computerize track detection (something the team could not!).

More updates and news to come of course, so please, stay tuned.

Dan
On behalf of the entire Stardust@home team

caprarom
Posts: 336
Joined: Thu Aug 02, 2007 7:12 am
Location: Riverview, MI

Re: Skill Score

Post by caprarom » Wed Jan 18, 2012 6:12 am

Hey, congrats, John! Of course, if that computerized track detection effort pays of, we may be out of a "job" after Phase V.

Mike C.

Post Reply